When I scroll down server list there is too much x sensitivity.
In VR it is very hard to scroll all the way down in one movement.
To reproduce in VR. Grab the slider maintain VR click and move the head down, if not perfectly Vertical you have to repeat 2 or 3 time.
You can also experiment with mouse but it is easier to go straight.

I think less x sensitivity after grabbing the slider should help a lot.