Papers & Patents
Publications & Patents
256 outputs
256 outputs
202618
IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%
We propose a spatial affordance-aware affine transformation method between heterogeneous spaces for continuous multi-object matching in shared Mixed Reality (MR) spaces. While previous redirection and spatial mapping approaches utilize physical objects and walkable areas, a critical gap remains in enabling continuous mapping between dissimilar physical environments that supports both precise object alignment and seamless locomotion in a shared space. Our method structurally segments heterogeneous spaces into interaction zones and constructs affine patches based on object adjacency and facing configuration, enabling continuous correspondence. We evaluate our method using a dataset of paired dissimilar spaces and demonstrate that, unlike conventional grid-based methods, our approach achieves broader spatial alignment and richer object matching. The results show that our method can serve as an effective mapping framework for shared environments requiring semantic continuity and structural coherence across diverse real-world spaces.
IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2026)Top 10%
This study explores a streamlined facial data collection method for conversational contexts, addressing the limitations of existing approaches that often require extensive datasets and prioritize technical metrics over user perception and experience. We systematically investigate which facial expression data are essential for reconstructing photorealistic avatars and how they can be captured efficiently. Our research employs a two-phase methodology to identify efficient facial data collection strategies and evaluate their effectiveness. In the first phase, we conduct facial data acquisition and evaluate reconstruction performance using utterance data and emotional data. In the second phase, we carry out a comprehensive user evaluation comparing three progressive conditions: utterance only, utterance and emotional data, and a control condition involving extensive data. Findings from 24 participants engaged in simulated face-to-face conversations reveal that targeted utterance and emotional data achieve comparable levels of perceived realism, naturalness, and telepresence, while reducing training time and data usage when compared to the extensive data collection approach. These results demonstrate that targeted data inputs can enable efficient avatar face reconstruction, offering practical guidelines for real-time applications such as AR/VR telepresence and highlighting the trade-off between data quantity and perceived quality.
IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR'26)Top 10%
We propose a method for generating task breakpoints based on an Origin-Centric Graph (OCG) to segment goal-oriented activity recordings into task units for adaptive playback in Virtual Reality (VR) environments. With the development of Augmented Reality (AR)/VR head-mounted displays (HMDs), research on adaptive tutorials and authoring tools has become active, but existing task segmentation methods mainly rely on manual annotation or are restricted to 2D video which limits their applicability to 3D VR contexts. In our approach, assembly scenarios with clearly defined task boundaries are recorded using a structured spatio-temporal scene graph (STSG), and the OCG is employed to track changes in the central object and the formation of new groups, thereby generating task breakpoints automatically. A user study collected user-perceived task breakpoints to establish ground truth (GT), and comparison with the algorithm-detected breakpoints demonstrated high agreement and confirmed accuracy in supporting adaptive playback. The proposed task segmentation method provides a foundation for dynamically adjusting VR playback according to user proficiency and progress, with potential for extension into automatic timeline segmentation systems for diverse VR recordings.
IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR'26)Top 10%
We present VRGaussianAvatar, an integrated system that enables real-time full-body 3D Gaussian Splatting (3DGS) avatars in virtual reality using only head-mounted display (HMD) tracking signals. The system adopts a parallel pipeline with a VR Frontend and a GA Backend. The VR Frontend uses inverse kinematics to estimate full-body pose and streams the resulting pose along with stereo camera parameters to the backend. The GA Backend stereoscopically renders a 3DGS avatar reconstructed from a single image. To improve stereo rendering efficiency, we introduce Binocular Batching, which jointly processes left and right eye views in a single batched pass to reduce redundant computation and support high-resolution VR displays. We evaluate VRGaussianAvatar with quantitative performance tests and a within-subject user study against image- and video-based mesh avatar baselines. Results show that VRGaussianAvatar sustains interactive VR performance and yields higher perceived appearance similarity, embodiment, and plausibility. Project page and source code are available at https://vrgaussianavatar.github.io.
HCI Korea 2026
KR 10-2924674-0000 · Korea · Granted
본 발명은, 이동 단말기의 전면에 입력을 가하는 펜 및, 사용자에게 증강현실의 3 차원 공간을 제공하는 증강현실 장치를 구비하고, 증강현실 장치는, 이동 단말기 및 펜과 관련된 정보를 디지털 데이터로 전환하여 3 차원 공간과 연결하고, 3 차원 공간에 3 차원 오브젝트를 출력하며, 이동 단말기의 전면을 3 차원 오브젝트와 연계하여 제어명령을 가하는 2 차원 받침면으로 설정하고, 2 차원 받침면에서 이동하는 펜의 움직임을 모니터링하여 제어명령을 생성하는, 증강현실 데스크톱 시스템을 제공한다.
202538
SIGGRAPH Asia 2025 XR
We propose a collaborative scene mood authoring system in a virtual reality (VR) environment, where multiple users design and control the multimodal feedback (i.e., audiovisual and haptic) of a shared scene using voice commands. To support this, the user’s command is processed by large language models (LLMs) with the scene metadata and mood presets to identify intended target objects and mood, and the system modifies the objects to match the desired atmospheric expression. To cover a wide range of affective moods in scene mood authoring, we devised four distinct mood presets that represent different combinations of arousal and valence across the affective space. Each preset is expressed through a set of labeled visual, auditory, and haptic feedback elements, enabling coherent and multimodal expression of affective states within the scene.
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) 2025
We present a spatial augmentation system designed for Augmented Reality (AR) exhibition experiences, focusing on the immersive reinterpretation of two-dimensional artworks. Existing AR exhibition applications primarily augment content related to the artwork, but fall short of its architectural elements, which are crucial for integrating the artwork into the viewer’s surrounding space and enabling immersive spatial alignment. This study proposes a spatial augmentation approach that takes an artwork image as input, generates floor patterns and architectural 3D models through 2D and 3D model generation pipelines, and semantically aligns and augments them in the physical space. This spatial augmentation intends to enhance viewer engagement and perceived presence by maintaining spatial coherence between virtual and real space. By activating architectural surfaces as part of the experience, the system repositions AR exhibitions as spatial narratives rather than conventional overlay-driven displays.
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) 2025
We propose a lifestyle quantification method for personalized augmented reality (AR) intervention in everyday life. Maintaining a healthy lifestyle accompanies extensive workforce and costs, because accurate health evaluation and tailored management is a prerequisite. However, research on assessing multiple dimensions of health and timely administration of personalized information has been lacking. To address these issues, first we deduced participant’s physical activity, nutrition, interpersonal relations, and stress management level through their mobile devices. Each participant’s week-long data were filtered by clinical features evidenced in high-level medical literature. Next, based on their health status, we presented personalized AR interventions in their simulated everyday environment. Statistical analysis and user experiment results verified the clinical capability of our method and its effect on health-promoting decision behaviors of participants. Through this feasibility study, we hope to contribute on tailored lifestyle management in our daily lives, for better health of all.
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) 2025
We present a fast and efficient method for transferring facial textures onto SMPL-X–based full-body avatars. Unlike conventional affine-transform methods that are slow and prone to visual artifacts, our method utilizes a barycentric UV conversion technique. Our approach precomputes the entire UV mapping into a single transformation matrix, enabling texture transfer in a single operation. This results in a speedup of over 7000x compared to the baseline, while also significantly improving the final texture quality by eliminating boundary artifacts. Through quantitative and qualitative evaluations, we demonstrate that our method offers a practical solution for personalization in immersive XR applications. The code is available online.1
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) 2025
We present a multimodal framework that uses few-shot language models to generate action scene graphs from raw video for structuring spatial experiences in extended reality (XR). The system integrates unsupervised action segmentation, multimodal captioning with a vision-language model (VLM) guided by speech and motion cues, and scene graph generation from captions using a large language model (LLM), both leveraging few-shot prompting. By combining visual, verbal, and motion inputs, the framework produces semantically rich and accurate representations of spatial experiences without requiring annotated data or model fine-tuning. The resulting outputs will support a range of downstream XR applications, including adaptive playback, intent-driven retrieval, and flexible content reconstruction.
IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2025Top-tier
As immersive virtual environments become more prevalent, avatars serve as critical social interfaces. This study explores how combinations of visual appearance, vocal characteristics, and informed identity influence users' initial impressions and partner preferences in four distinct XR scenarios: physical, intellectual, social, and romantic. A within-subject experiment with 40 participants assessed perceived warmth, competence, uncanniness, and selection preferences across diverse avatar configurations. Results indicate that vocal cues had a particularly strong impact on social perception, often shaping feelings of approachability and clarity in communication. While some cue alignments enhanced perceived social comfort and engagement, inconsistencies across gender-related cues occasionally led to increased perceptions of uncanniness, especially in emotionally sensitive contexts. These findings highlight the importance of designing avatars that thoughtfully adapt to different interaction contexts, supporting inclusive and responsive user experiences in social XR platforms.