Publications & Patents

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

We propose event-based vibrotactile feedback as an effective approach for enhancing bare-hand XR interaction, delivered through commodity smartwatches that users already wear. While controllers and haptic gloves provide rich tactile feedback, they introduce additional hardware that users must carry and wear throughout the day. Importantly, they interpose hardware between the user and the virtual world, occupying the hands, constraining finger motion, and undermining truly unobstructed bare-hand interaction. Our proposed approach delivers short discrete pulses at key manipulation moments such as object contact and state changes, providing tactile confirmation at the wrist without requiring additional hardware or disrupting finger movement. Two user studies with 26 participants demonstrate that this referred feedback significantly enhances user experience at either wrist placement, with 94% finding it helpful and particular benefits during complex tasks like knob rotation where haptic cues reduced visual attention demands. These findings establish design guidelines for integrating accessible haptics into everyday XR through personal devices, supporting broader adoption of natural hand interaction.

IEEE Transactions on Visualization and Computer GraphicsTop 10%

We present ForceCtrl, a novel 3D hand raycasting technique that enhances pointing precision based on control-display (CD) gain controlled with user-defined pinch force. We introduce a target-agnostic approach for refining raycasting precision, overcoming limitations in human motor accuracy. User-defined pinch force, detected with surface electromyography (sEMG), enables users to easily activate or deactivate CD gain during interaction. We propose three CD gain strategies and compare them through target selection and placement tasks. Our system reduces selection errors, placement jitters, and user workload, especially for distant targets in high-difficulty tasks. These results highlight the effectiveness of applying CD gain to hand raycasting and demonstrate the potential of user-defined pinch force as a robust input modality for precise hand interaction in AR/VR.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

We propose Int3DNet, a scene-aware network that predicts 3D intention areas directly from scene geometry and head-hand motion cues, enabling robust human intention prediction without explicit object-level perception. In Mixed Reality (MR), intention prediction is critical as it enables the system to anticipate user actions and respond proactively, reducing interaction delays and ensuring seamless user experiences. Our method employs a cross attention fusion of sparse motion cues and scene point clouds, offering a novel approach that directly interprets the user's spatial intention within the scene. We evaluated Int3DNet on MoGaze and CIRCLE datasets, which are public datasets for full-body human-scene interactions, showing consistent performance across time horizons of up to 1500 ms and outperforming the baselines, even in diverse and unseen scenes. Moreover, we demonstrate the usability of proposed method through a demonstration of efficient visual question answering (VQA) based on intention areas. Int3DNet provides reliable 3D intention areas derived from head-hand motion and scene geometry, thus enabling seamless interaction between humans and MR systems through proactive processing of intention areas.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

We propose OFERA, a novel framework for real-time expression control of photorealistic Gaussian head avatars for VR headset users. Existing approaches attempt to recover occluded facial expressions using additional sensors or internal cameras, but sensor-based methods increase device weight and discomfort, while camera-based methods raise privacy concerns and suffer from limited access to raw data. To overcome these limitations, we leverage the blendshape signals provided by commercial VR headsets as expression inputs. Our framework consists of three key components: (1) Blendshape Distribution Alignment (BDA), which applies linear regression to align the headset-provided blendshape distribution to a canonical input space; (2) an Expression Parameter Mapper (EPM) that maps the aligned blendshape signals into an expression parameter space for controlling Gaussian head avatars; and (3) a Mapper-integrated Avatar (MiA) that incorporates EPM into the avatar learning process to ensure distributional consistency. Furthermore, OFERA establishes an end-to-end pipeline that senses and maps expressions, updates Gaussian avatars, and renders them in real-time within VR environments. We show that EPM outperforms existing mapping methods on quantitative metrics, and we demonstrate through a user study that the full OFERA framework enhances expression fidelity while preserving avatar realism. By enabling real-time and photorealistic avatar expression control, OFERA significantly improves telepresence in VR communication. A project page is available at https://ysshwan147.github.io/projects/ofera/.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

We introduce SceneLinker, a novel framework that generates compositional 3D scenes via semantic scene graph from RGB sequences. To adaptively experience Mixed Reality (MR) content based on each user's space, it is essential to generate a 3D scene that reflects the real-world layout by compactly capturing the semantic cues of the surroundings. Prior works struggled to fully capture the contextual relationship between objects or mainly focused on synthesizing diverse shapes, making it challenging to generate 3D scenes aligned with object arrangements. We address these challenges by designing a graph network with cross-check feature attention for scene graph prediction and constructing a graph-variational autoencoder (graph-VAE), which consists of a joint shape and layout block for 3D scene generation. Experiments on the 3RScan/3DSSG and SG-FRONT datasets demonstrate that our approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations, even in complex indoor environments and under challenging scene graph constraints. Our work enables users to generate consistent 3D spaces from their physical environments via scene graphs, allowing them to create spatial MR content. Project page is https://scenelinker2026.github.io.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

We propose a spatial affordance-aware affine transformation method between heterogeneous spaces for continuous multi-object matching in shared Mixed Reality (MR) spaces. While previous redirection and spatial mapping approaches utilize physical objects and walkable areas, a critical gap remains in enabling continuous mapping between dissimilar physical environments that supports both precise object alignment and seamless locomotion in a shared space. Our method structurally segments heterogeneous spaces into interaction zones and constructs affine patches based on object adjacency and facing configuration, enabling continuous correspondence. We evaluate our method using a dataset of paired dissimilar spaces and demonstrate that, unlike conventional grid-based methods, our approach achieves broader spatial alignment and richer object matching. The results show that our method can serve as an effective mapping framework for shared environments requiring semantic continuity and structural coherence across diverse real-world spaces.

IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%

This study explores a streamlined facial data collection method for conversational contexts, addressing the limitations of existing approaches that often require extensive datasets and prioritize technical metrics over user perception and experience. We systematically investigate which facial expression data are essential for reconstructing photorealistic avatars and how they can be captured efficiently. Our research employs a two-phase methodology to identify efficient facial data collection strategies and evaluate their effectiveness. In the first phase, we conduct facial data acquisition and evaluate reconstruction performance using utterance data and emotional data. In the second phase, we carry out a comprehensive user evaluation comparing three progressive conditions: utterance only, utterance and emotional data, and a control condition involving extensive data. Findings from 24 participants engaged in simulated face-to-face conversations reveal that targeted utterance and emotional data achieve comparable levels of perceived realism, naturalness, and telepresence, while reducing training time and data usage when compared to the extensive data collection approach. These results demonstrate that targeted data inputs can enable efficient avatar face reconstruction, offering practical guidelines for real-time applications such as AR/VR telepresence and highlighting the trade-off between data quantity and perceived quality.