
Audio-visual experiments: an Arduino spectrum analyzer, physically based audio synthesis at Pixar, records that hide images in their grooves, and audio-driven shorts.
Arduino Audio Spectrum Analyzer
A fun video demonstration of the analyzer in action with some look into the internal components.
For this project, 8-bit shift registers, loads of colored LEDs with their respective resistors, a microphone, and an Arduino Uno were used to create a spectrum analyzer. The encasing is wooden laser cut design with some sneaky soldering work in the back and a base that houses the electrical components.
The first step in this project was to adjust the sensitivity levels of the microphone to the environment to essentially filter out background noise. After getting successful input from the microphone and mapping the amplitude to a set of 5 LEDs, now it was a matter of how to create a spectral analysis on an 8x8 grid of colored LEDs. This is where the 8-bit shift registers came in handy as it allows one to input, shift, and output values one bit at a time, controlled by a clock signal. This is useful for tasks like data storage, manipulation, and efficient control of multiple outputs in electronics projects. In this particular case, it was used to shift the amplitude data over time, acting like a wave of the input signals that were coming into the microphone.
Physically Based Audio Synthesis System
An Independent Pixar Research & Development Project
During my time at Pixar in 2016, I had the opportunity to conduct an independent research project of my choosing. Seeing how there were practically no automated audio tools (dare I say audio-mated?) in the studio, I wanted to explore the possibilities. By computing simulation properties of a scene such as velocity, point of collision, and distance from camera, I was able to use this data to create physically based audio. The user of this program has the option to select the material type of the objects in the scene and can either define its volume, or that can be calculated as well through the system. All these components drive the final output. As an added layer, the user can define regions that won't calculate any sounds or lower their contribution to the end result. I also expanded the tool so that reverb can be added either by simulating bounces in the scene or driving it with a post-process filter, this feature is not demoed in the video as the content has not been released.
Record of a Thousand Words
Taking the concept of varying line weight in order to create an image, this experiment takes it a step further by mapping the audio signals of a song along a record. When viewed at an angle the audio signals form an image by their weight distribution through the duration of the sampled audio. Since this is currently limited to a one signal per a sample point, the audio frequency ranges that are accumulated and put into the record are customizable.
Below are some stills showing details that might have been lost in video compression.



The next steps of this project is to make these records customizable for any user and 3D print them with material that would be able to have the fidelity needed to create clear images with this methodology.
Real-Time Experimentations
[WORK IN PROGRESS] This is the first iteration at an attempt to hear images. For testing purposes, I am using this painting a friend of mine made and sampling the color and average luminance of the pixels in the sample area. Those values are converted into samples from a sine wave. The application modulates throughout the image but the user can scan the image themselves for a more creative control.

Animated Audio-Driven Shorts
Some fun I was having with animated audio-driven shorts to songs that I couldn't get out of my head.








