Thanks for all the great advice guys. What I did in the end is run a packet loss test, it showed significant jitter and packet loss, which was explained to be problematic for gaming. So I ran the test again, this time using a LAN connection and... hardly an jitter or packet loss!
The limited time I had on public servers to confirm that cable connection solved the problem showed promising, no warps so far =)