Experience Graph using Spatio-Temporal Scene Data for Replaying Mixed Reality Interaction

Abstract

This study proposes an experience graph, a Mixed Reality (MR) experience replaying method for recording and playing spatially adaptive interaction in a mixed reality environment. Many MR immersive interactions are experienced in a space different from the original recorded space, so it is important to adaptively experience the interaction in the local space. As a solution to the collision and mismatch during the interactions in local space, there is research on scene generation or adaptive object placement, but the change of interactions dependent on space and the resulting experience transformation were not considered. This study structures spatial information and dependent interactions into an experience graph and proposes a spatial adaptive replay method that minimizes interaction deformation. Recorded interactions are evaluated quantitatively through loss rate and transformation rate, played interactions are evaluated qualitatively through user experiments, and finally, indicators important to the user experience are reflected in the experience graph. Utilizing the results of this study will contribute to experiencing immersive spatial MR where users actively record and share spatial experiences beyond the limitations of time and space.