It’s likely a sensor. Are there any “check engine” lights on? If the car sits for a while and you re-start it, does this solve the problem?
What you are describing is common in the early 2004-05 Quests. I would check if there any OBDII codes stored as a bad camshaft position sensor(s) (there are 2) can cause this problem. If there aren’t any codes stored and this problem is sporadic, it could be a problem with the Transmission Control Module.
Before replacing anything, get the car scanned to see if any codes are stored.