Week 8 - Visualizing Sound

Pitchforks, Musical Instrument, LED Light Source (Projector)

Computers, Humans, Physics

Electricity Misconceptions Spread by K-6 Textbooks

Light

The World -> EM Waves -> Eye -> Visual Processing -> Mind

Richard Feynman talks about light

The Electromagnetic Spectrum

Spectrum

Visual Spectrum

Cone typeNameRangePeak wavelengthColors
S Shortβ400–500 nm420–440 nmBlue
M Mediumγ450–630 nm534–555 nmGreen, Yellow
L Longρ500–700 nm564–580 nmGreen, Yellow, Red

Wikipedia: Electromagnetic Spectrum

How Good Our Eyes Are

Wikipedia: Naked Eye Wikipedia: Color Vision CDC: The Burden of Vision Loss

Color

Purple

Emissive Color

Sunlight contains electromagnetic radian in many wavelengths. Sunlight

An LED provides electromagnetic radian in a very specific wavelength range.

An LED computer display has LEDs of three colors. It can vary the intensity of those three colors, but can’t provide electromagnetic radiation in the wavelengths between them.

We perceive the mix of the three colors as a single color.

Reflective Color

A reflective object doesn’t reflect color of single wavelength. Instead it reflects/absorbs all wavelengths at different amounts.

We perceive the reflections as a single color.

A reflective color cannot be brighter than the lighting in any wavelength.

We adjust our perceived color of an image based on our understanding of the lighting.

How Good our Visual Processing Is

Thoughts

Our understanding of color/color theory is informed from the anatomy of our eye and the way our mind processes vision.

We talk about what color something is a single thing: dark blue, pink, vivid green. We don’t think about the color of something as a little bit green, a little bit blue, and a lot red. We definitely don’t think of color as the sum of the many in between wavelengths.

Sound

What Sound Is

A wave of air pressure and displacement. Something in contact with air vibrates. As that thing pushes forward, it pushes the particles air in front of it forward into the the particles of air in front of them. Making an area of higher pressure. This high pressure area pushes out in all directions, and a wave of pressure begins to propagate though the air.

This pressure wave can push on other things like microphones and our ears. Our ears are able to detect very rapid and subtle changes in this pressure. And we are then able to understand the amplitude, frequency, and even shape of these changes. Because we have two ears, spaced a few inches apart, we can compare what each ear hears to gain spacial information as well.

How Hearing Works

Wikipedia: Sound

How Good Our Ears Are

Our Ears Make Mistakes Sensitivity of Human Ear

How Good Our Audio Processing Is

Vision VS Hearing

Light, Sound, And Virtual Reality

“To give you just one example of how much better visuals can get; in order for Crescent Bay to deliver the same pixel density as a monitor at a normal viewing distance, it would have to have a resolution of about 5K by 5K per eye, something like 20 times as many pixels as it currently has. In order for it to have retinal resolution at a field of view of 180 degrees, it would have to have something on the order of 16K by 16K resolution, roughly 200 times as many pixels.”

— Michael Abrash

Our audio recording and playback capabilities are much closer to saturating the sensitivity of our ears.

Even two perfect microphones, recordings, and speakers can do a pretty good job of fooling you into thinking a recording sound is a real sound. But a recording not enough on its own. We can move our heads, and we can use our understanding of space in interpreting sounds. For VR sound, the computer must process the sound to place sounds three-dimensional, acoustic space.

Lets Make Noise

A Wave Propagated On a Wave

Additive Synthesis

The Fourier transform

P5 + FFT

P5 FFT Reference Class Example Code

QC + FFT

Class Example Composition

Links

Thermal Infrared Photography