Probably motion sickness.
Here is what I've found from having many guests without VR experience try out my racing rig.
1. You need a solid frame rate. Holding 90fps helps.
2. A motion system helps dramatically. It makes you "feel" like you are moving and is enough to trick your brain into believing the motion is happening.
3. Transducers help incrementally. They mostly add texture, but feeling the engine revving and getting the bumps in the road helps sell the motion.
Initially I had an under powered computer and and no motion or transducers. In real life I've flown planes, like roller coasters, and can read for a while in a car without getting sick. It took me a week of half hour driving to get my VR legs. I got queasy each day for 5 days in a row and then I was basically over it.
If I had guests come over I would save the driving sims for last because the results were always the same.
Then I dramatically upgraded my rig with a Sim-Lab P1, NLRv3, more transducers with better placement and finally upgraded the CPU from an i5-4960 to an i9-9900K.
The frame rate was rock solid with SS, the motion system was synced up well and there appears to be no lag time with control inputs. It felt night and day better to me even with my VR "immunity".
This Spring I had a couple friends over who had tried it out 6 months earlier without motion or the CPU upgrade. They really enjoyed it this time and agreed it was much better. They spend the better part of two hours playing racing sims before going to room scale.
Last Friday I had a party with 24 guests and a number of them tried out my racing rig. No one got queasy. They loved it! There was a lot of laughing as people plowed into tire walls or drove off cliffs.
The Catch-22. I would argue that queasiness is not a right of passage for "most" people with the right setup. However I doubt many people would go all in on a motion system until they've already gotten hooked. Considering VR is less expensive to setup than a serious rig, I'm betting that a good number of people will have bad experiences in VR before setting up a motion system or transducers.
There are also exceptions. If you are a person who gets car sick easily in a real car and can't handle roller coasters in REAL LIFE, you can't expect the experience to be better in VR. My wife fits into this category. After all the upgrades she tried some cart racing in VR. She lasted for a while, but had to stop.