HMD-only Controllable 3D Gaussian Avatars in VR: Face and Full-body Demonstration

Abstract

We present a VR research demo for practical telepresence with photorealistic 3D Gaussian Splatting (3DGS) avatars under HMD-only sensing. The demo consists of two parts that provide complementary experiences in a single session: a face part, where HMD-provided blendshape signals drive an expressive 3DGS head avatar rendered stereoscopically on the client, and a full-body part, where a user-specific 3DGS full-body avatar is created from a single image and driven using HMD signals. During the demo, participants first capture a single image for full-body avatar creation, experience facial control using a pre-built third-person head avatar while the full-body avatar is generated, and then control their personalized full-body avatar in VR. By enabling both facial and full-body controllability and contrasting two rendering-responsibility designs, client-side direct 3D rendering and server-side stereoscopic image rendering/streaming-the demo offers practical insights for designing responsive, realistic VR telepresence systems under HMD-only sensing.