Obviously, just speculative but the electric front windows are separately controlled: the driver's through the Electric Window Module (ECM), featured in my article, and the driver's window switch; the passenger window is operated directly from its switch, and both obtain current via the window relay.
If the front driver's window works exactly as it should (ie with ignition on, engine not started, and within your 20 second period), I would suggest that the problem is likely to be a wiring or switch fault on the passenger side. In particular, look to see if the wire terminals at the switch for driving the passenger window motor have been accidentally swapped over - that would drive the window open when it should remain at the top. Alternatively, try another switch.
The window relay simply switches battery positive through three fuses to a) the ECM; b) the driver's window switch; c) the passenger's window switch. Since the passenger window operates (even though incorrectly) it shows that the window relay is delivering current, so works.
Additionally, the motor in the passenger door will have some form of sensor module to detect top, bottom and jamming. That's to avoid the consequences of the switch being on for an excessive time, or something preventing window movement such as a child's toy or head etc. It is possible the window motor sensor module could be faulty. I have no experience of that, unfortunately.