Heejeong Ko

Question-asking is one of the key indicators of cognitive engagement. However, understanding how the distinct psychological affordances of presentation media shape learners’ spoken inquiries with embodied Intelligent Virtual Agents (IVAs) remains limited. To systematically examine this process, we propose a 5W1H-based framework for analyzing learner questions. Using this framework, we conducted a user study comparing an Augmented Reality-based IVA (AR-IVA) deployed in the physical environment with a screen-based IVA (Video-IVA) during cardiopulmonary resuscitation (CPR) instruction. Results showed that the AR-IVA elicited higher spatial and social presence and promoted more frequent and longer questions focused on clarification and understanding. In contrast, the Video-IVA encouraged questions regarding procedural refinement. Presence acted as a selective filter, shaping the timing and …

We present a multimodal framework that uses few-shot language models to generate action scene graphs from raw video for structuring spatial experiences in extended reality (XR). The system integrates unsupervised action segmentation, multimodal captioning with a vision-language model (VLM) guided by speech and motion cues, and scene graph generation from captions using a large language model (LLM), both leveraging few-shot prompting. By combining visual, verbal, and motion inputs, the framework produces semantically rich and accurate representations of spatial experiences without requiring annotated data or model fine-tuning. The resulting outputs will support a range of downstream XR applications, including adaptive playback, intent-driven retrieval, and flexible content reconstruction.

For the growth of the metaverse-related consumer electronics market, it is essential that users from diverse backgrounds can easily create and share content on metaverse platforms. Generative AI (GenAI) has the potential to transform the metaverse into a major consumer industry by innovating asset acquisition and editing within authoring tools. This article analyzes the current applications of GenAI in metaverse content creation and proposes strategies for its effective use across various authoring stages, including data integration, content creation, and modification. In addition, it suggests solutions to address the lack of training data and enhance content quality.

We proposed viewpoint-tolerant shared depth perception without individual tracking by leveraging human cognitive compensation in universally 3D rendered images on a wall-sized display. While traditional 3D-perception-enabled display systems have primarily focused on single-user scenarios-adapting rendering based on head and eye tracking-the use of wall-sized displays to extend spatial experiences and support perceptually coherent multi-user interactions remains underexplored. We investigated the effects of virtual depths (dv) and absolute viewing distance (da) on human cognitive compensation factors (perceived distance difference, viewing angle threshold, and perceived presence) to construct the wall display-based eXtended Reality (XR) space. Results show that participants experienced a compelling depth perception even from off-center angles of 23°-37°, and largely increasing virtual depth worsens depth perception and presence factors, highlighting the importance of balancing extended depth of virtual space and viewing distance from the wall-sized display. Drawing on these findings, wall-sized displays in venues such as museums, galleries, and classrooms can evolve beyond 2D information sharing to offer immersive, spatially extended group experiences without individualized tracking or wearables.

본 발명은 입체적 지각 기반의 확장 가상 공간 제공 시스템 및 방법에 관한 것으로서, 특히 대형 월 디스플레이 상에 정적 이미지 기반의 XR 콘텐츠를 제공하되, 사용자 시점에서 깊이 단서를 인식하게 하여 가상 공간의 입체감과 확장성을 체험할 수 있도록 하는 기술에 관한 것이다.

We introduce the concept of a meta-object, a next-generation virtual object that inherits the form, properties, and functions of its real-world counterpart, enabling seamless synchronization, interaction, and sharing between the physical and virtual worlds. While plenty of today's virtual objects provide some sensory feedback and dynamic behavior, meta-objects fully integrate interactive and multisensory features within a structured data framework to enable real-time immersive experiences in a postmetaverse intelligent simulation platform. Three key components underpin the utilization of meta-objects in the postmetaverse: property-embedded modeling for physical and action realism, adaptive multisensory feedback tailored to user interactions, and a scene graph-based intelligence simulation platform for scalable and efficient ecosystem integration. By leveraging meta-objects through wearable augmented reality/virtual reality devices, the postmetaverse facilitates seamless interactions that transcend spatial and temporal barriers, paving the way for a transformative reality-virtuality convergence.