A high-level, component-based interaction system built upon Oculus and openXR APIs. This toolkit intends to establish embodied interaction modalities with the virtual environment and objects as well as related measurement methods. Interaction modes include ones beyond the physical realm like distance grab, eye gaze, and gesture pointing, as well as tracked hand gestures with their physics-based interface with virtual objects.

Adaptive Selection Interface

Project Leader: Tobias Höllerer, Four Eyes Lab, UCSB

Implement multimodal interaction (hands, gaze, voice, etc) into object-interaction tasks in Augmented Reality (AR), using the passthrough mode on Oculus Quest 3 headset. Applies Machine Learning models to learn from user features (preference, gesture pattern, intention) and task features (virtual/physical, size, distance) to predict and recommend the optimal interaction modality for interacting with a new object.

Understanding Cognitive Capacities During Active, Wide Area Exploration

Project Leader: Barry Giesbrecht, Mary Hegarty, Tobias Höllerer, Institute for Collaborative Biotechnologies, UCSB

An investigation on human cognitive variation across screen-based video games and virtual reality (VR) experiences. Designed controlled laboratory experiments including pathfinding tasks, developed user study applications using Unity, and implemented treadmill for natural embodied navigation in VR.

Detached Within: a game mechanics based on depersonalization-derealization disorder to reduce immersive horror in Virtual Reality

Develop techniques for eliciting and intensifying fear within immersive Virtual Reality (VR) horror-themed experiences. This includes detection methods of emotions (such as boredom, anxiety, frustration, distress, fear, and aversion) through VR headsets. The study also addresses the attenuated emotional effect of depersonalization and derealization symptoms. Furthermore, it introduces a novel game mechanic inspired by these symptoms, adaptively adjusting the cognitive impact on emotional perception of horror games.



May – December 2023 | “Digital Colonization”, Best Data Visualization Award, Art of Science, Santa Barbara Museum of Art, Santa Barbara

May 2023 | “Visualizing the Invisibles”, latent•ville, SBCAST, Santa Barbara

May 2023 | “LivingRoom”, latent•ville, SBCAST, Santa Barbara

May – July 2020 | “Digital Colonization”, Rijeka 2020 European Capital of Culture, Rijeka

May 2018 | “The Mediator”, Dwight T. Black Memorial Scholar Award, IIT COA Open House, IIT, Chicago