SoundBloom

Github link:    https://github.com/Psychea-Dai/XRFinal

Overview

Sound Bloom is an immersive spatial music experience designed for Apple Vision Pro. The project explores how spatial computing can transform music from something you listen to into something you actively build and interact with.

In this experience, the user is placed at the center of a dark environment, surrounded by a dormant organic structure. Five floating sound objects represent different musical layers. By interacting with these objects, users gradually activate and compose a dynamic audio-visual environment.

Development Process

The project was developed in Unity using PolySpatial for Apple Vision Pro. The main focus was on building a fully gesture-based interaction system without traditional UI.

We implemented hand tracking interactions using XR Interaction Toolkit, including grabbing, tapping, and repositioning objects in space. A modular audio system was created with multiple layers and variations, allowing dynamic composition. A centralized state management system was used to track user progress and trigger visual changes.

The development process involved iterative prototyping, testing interaction responsiveness, and optimizing performance to reach the target frame rate of 90fps.

Successes

One of the main successes of the project was creating a seamless interaction system with zero UI, relying entirely on natural gestures.

Other successes include strong integration between audio and visual feedback, emergent musical compositions based on user interaction, and a cohesive immersive aesthetic inspired by organic forms. The experience allows users to become active creators rather than passive listeners.

Challenges

Several challenges were encountered during development.

There were technical issues with scripts and builds, especially when preparing for deployment on  Vision Pro. Ensuring stable gesture detection and avoiding interaction conflicts was also difficult. Managing multiple audio layers while maintaining performance required optimization. Synchronizing visual effects with audio activation and keeping the experience running at 90fps were additional challenges.

What We Learned

Through this project, we learned how to design interactions specifically for spatial computing environments and the importance of simplicity in interaction design.

We also learned how audio can drive user experience in XR, how to manage cross-platform development challenges, and how to structure systems using modular design and state management.

Most importantly, we learned how to design experiences that feel natural in 3D space rather than relying on traditional UI.

Future Revisions

If we had more time, we would expand the number of musical layers and sound variations, improve visual complexity and responsiveness, and refine gesture precision and feedback.

We would also explore multi-user collaboration and investigate how this system could evolve into a more advanced creative tool rather than a single-user experience.