Ikbeom Jeon

We present a novel natural user interface framework, called Meta-Gesture, for selecting and manipulating rotatable virtual reality (VR) objects in egocentric viewpoint. Meta-Gesture uses the gestures of holding and manipulating the tools of daily use. Specifically, the holding gesture is used to summon a virtual object into the palm, and the manipulating gesture to trigger the function of the summoned virtual tool. Our contributions are broadly threefold: 1) Meta-Gesture is the first to perform bare hand-gesture-based orientation-aware selection and manipulation of very small (nail-sized) VR objects, which has become possible by combining a stable 3-D palm pose estimator (publicly available) with the proposed static-dynamic (SD) gesture estimator; 2) the proposed novel SD random forest, as an SD gesture estimator can classify a 3-D static gesture and its action status hierarchically, in a single classifier; and 3) our novel voxel coding scheme, called layered shape pattern, which is configured by calculating the fill rate of point clouds (raw source of data) in each voxel on the top of the palm pose estimation, allows for dispensing with the need for preceding hand skeletal tracking or joint classification while defining a gesture. Experimental results show that the proposed method can deliver promising performance, even under frequent occlusions, during orientation-aware selection and manipulation of objects in VR space by wearing head-mounted display with an attached egocentric-depth camera (see the supplementary video available at: http://ieeexplore.ieee.org).

There have been studies on personalized augmented reality (AR) systems taking users’ contexts and histories into account. However, there is insufficient research on incorporating real-time user behavior and interactions into the personalized recommendations in spatial transitions, which can be used for new users without user history data. The spatial transitions, distances between two Point of Interests (POIs) of an AR tour trajectory through which users should pass to visit AR contents, need to be filled using the personalized AR contents to reduce the discontinuity experience. This paper aims to propose a user behavior model to recommend personalized contents in the AR tour trajectory to create a personalized experience. First, we model three interactions, including staring, skipping, and liking, using mathematical methods; and two spatial behaviors, including standing and moving, using a rule engine. Second, a content filtering-based recommendation method is presented to apply the proposed user model to recommend personalized AR contents in the spatial transitions. The experiment results showed that the proposed real-time user behavior model brought notable improvements in creating a personal experience of the AR tour system. The proposed method outperformed the conventional method in terms of recommendation performance metrics, so that it achieved an average of 15% more precision and recall scores and an average of 32% more personalization scores than those of the conventional method. Furthermore, the level of personalization perceived by the proposed user model demonstrated significant positive relationships with the level of decision effectiveness, system trustworthiness, and perceived recommendation transparency. The findings showed the effectiveness of the proposed framework in solving the cold-start recommendation problem.

There have been attempts to provide new cinematic experiences by connecting TV or movie content to suitable locations through augmented reality (AR). However, few studies have suggested a method to manage breakdowns in continuity due to spatial transitions. Thus, we propose a method to manage the spatial transition that occurs when we create a TV show trajectory by mapping TV show scenes with spatiotemporal information to the real world. Our approach involved two steps. The first step is to reduce the spatial transition considering the sequence, location, and importance of TV show scenes when creating the TV show trajectory in the authoring tool. The second is to fill the spatial transition with additional TV show scenes considering sequence, importance, and user interest when providing the TV show trajectory in the mobile application. The user study results showed that reducing spatial transition increases narrative engagement by allowing participants to see important content within the trajectory. The additional content in spatial transition decreased the physical demand and effort in terms of the perceived workload, although it increased the task completion time. Integrated spatial transition management improved the overall cinematic augmented reality (CAR) experience of the TV show. Furthermore, we suggest design implications for realizing the CAR of TV shows based on our findings.

사용자에게 착용된 깊이 카메라에 의해 촬영되는 손의 제 1 시퀀스 영상을 획득하는 단계; 획득한 제 1 시퀀스 영상 내 복수의 프레임 각각의 손에 복수의 복셀(voxel)로 나뉘어진 사각 큐브(rectangular cube)를 매핑하는 단계; 복수의 프레임 각각에 대해, 사각 큐브를 구성하는 복수의 복셀들의 정적 복셀 특징(static voxel feature)을 추출하고, 추출된 정적 복셀 특징을 인코딩하여 제 1 LSP 특징(layer shape pattern feature)을 획득하는 단계; 및 제 1 LSP 특징을 랜덤 포레스트에 입력하여, 제 1 시퀀스 영상에서 손의 홀딩 제스쳐에 대응하는 물체의 종류와 손의 조작 제스쳐에 대응하는 조작의 종류를 판단하는 단계를 포함하는 것을 특징으로 하는 본 발명의 일 실시예에 따른 제스쳐 감지 장치에 의한 제스쳐 감지 방법이 개시된다.

We investigate how path context, encompassing both comfort and attractiveness, shapes user experiences in outdoor locative storytelling using Augmented Reality (AR). Addressing a research gap that predominantly concentrates on indoor settings or narrative backdrops, our user-focused research delves into the interplay between perceived path context and locative AR storytelling on routes with diverse walkability levels. We examine the correlation and causation between narrative engagement, spatial presence, perceived workload, and perceived path context. Our findings show that on paths with reasonable path walkability, attractive elements positively influence the narrative experience. However, even in environments with assured narrative walkability, inappropriate safety elements can divert user attention to mobility, hindering the integration of real-world features into the narrative. These results carry significant implications for path creation in outdoor locative AR storytelling, underscoring the importance of ensuring comfort and maintaining a balance between comfort and attractiveness to enrich the outdoor AR storytelling experience.