Yeah I agree with getting the sweet spot just right, like any new HMD it takes some getting used too but after a few races I'm adjusting it less and less mid race.
The clarity outside the sweet spot is unnoticeable during racing/flying/no man's sky (for me), it's when I'm in a Menu I notice it more but it's really not "bad" or anything, I just move my head so the sweet spot takes up the text I want to read and it's fine.
As above it takes a while to adjust, but the more I use it the more comfortable I am with the Reverb and it's sweet spot.