Current
MetaFrame
A high-level, component-based interaction system built upon Oculus and openXR APIs. This toolkit intends to establish embodied interaction modalities with the virtual environment and objects as well as related measurement methods. Interaction modes include ones beyond the physical realm like distance grab, eye gaze, and gesture pointing, as well as tracked hand gestures with their physics-based interface with virtual objects.
Adaptive Selection Interface
Project Leader: Tobias Höllerer, Four Eyes Lab, UCSB
Implement multimodal interaction (hands, gaze, voice, etc) into object-interaction tasks in Augmented Reality (AR), using the passthrough mode on Oculus Quest 3 headset. Applies Machine Learning models to learn from user features (preference, gesture pattern, intention) and task features (virtual/physical, size, distance) to predict and recommend the optimal interaction modality for interacting with a new object.
Understanding Cognitive Capacities During Active, Wide Area Exploration
An investigation on human cognitive variation across screen-based video games and virtual reality (VR) experiences. Designed controlled laboratory experiments including pathfinding tasks, developed user study applications using Unity, and implemented treadmill for natural embodied navigation in VR.
Detached Within: a game mechanics based on depersonalization-derealization disorder to reduce immersive horror in Virtual Reality
Develop techniques for eliciting and intensifying fear within immersive Virtual Reality (VR) horror-themed experiences. This includes detection methods of emotions (such as boredom, anxiety, frustration, distress, fear, and aversion) through VR headsets. The study also addresses the attenuated emotional effect of depersonalization and derealization symptoms. Furthermore, it introduces a novel game mechanic inspired by these symptoms, adaptively adjusting the cognitive impact on emotional perception of horror games.
Exhibition
May – December 2023 | “Digital Colonization”, Best Data Visualization Award, Art of Science, Santa Barbara Museum of Art, Santa Barbara
May 2023 | “Visualizing the Invisibles”, latent•ville, SBCAST, Santa Barbara
May 2023 | “LivingRoom”, latent•ville, SBCAST, Santa Barbara
May – July 2020 | “Digital Colonization”, Rijeka 2020 European Capital of Culture, Rijeka
May 2018 | “The Mediator”, Dwight T. Black Memorial Scholar Award, IIT COA Open House, IIT, Chicago