This is extremely frustrating to see. It also happens to the shores, no matter the settings.
I first started the game up in VR, and it worked fine. I then changed some visual settings, then reverted back after this issue arose, to no avail. The only thing it fixed was that my VR view finally synced with my headset translation (rotation was OK).