We are building a social VR platform and the latency is killing the experience. When players high-five or throw objects to each other, the lag makes it feel disjointed. Are there specific Networking Frameworks like Photon Fusion or Mirror that handle VR-specific constraints better, such as head and hand tracking synchronization?
3 answers
For VR, you need a networking model that supports Client-Side Prediction. I highly recommend Photon Fusion because it is a "tick-based" state synchronization engine. Tracking data (head/hands) should be sent at a high frequency but compressed to save bandwidth. You don't sync every frame; you sync the "transform" and let the client interpolate the movement smoothly. For interactions like high-fives, you should use "Local Ownership Transfer." When a player grabs an object, they become the master of that object's physics temporarily. This removes the round-trip delay to the server, making the interaction feel instant for the person doing the grabbing.
Have you looked into Inverse Kinematics (IK) to hide the jitter in network movement? Sometimes the "lag" is just poor animation interpolation.
We use Netcode for Entities in Unity’s DOTS. It’s a bit of a learning curve, but the performance for high player counts is significantly better than traditional Monobehaviors.
Jessica, DOTS is definitely the future for large-scale VR worlds, though it does require a complete shift in how you structure your data.
Steven, IK is a great mask! Samantha, by using a VRIK solver on the client side, you only need to sync the position of the head and hands. The client's local IK engine will "guess" where the elbows and shoulders are. This significantly reduces the amount of data you need to send over the wire. Also, for social VR, consider using Voice-Over-IP (VoIP) with spatial audio like Photon Voice. If the audio is perfectly synced with the mouth movements (Lip Sync), players will forgive a little bit of jitter in the body movement, as the social connection feels maintained.