Audio data as video data representation

Additive synthesis, is the reverse of a Fourier transform. A Fourier transform converts an audio representation (the measure of air pressure over time that we call sound), to a spectral representation of the same sound also over time. Different ways to represent the same thing.

In DSP lingo they call this going from the time domain to the frequency domain, if you ever read this somewhere.

Additive synthesis goes the opposite direction. It take the spectral representation and creates a regular audio representation that can make a speaker vibrate that you can hear.

Additive synthesis works by adding all the various intensities of different frequency sine waves at a particular instant together to create a more complex audio wave at that instant, and then by stringing separate different instances together to create audio over time.

If you know about wavetable synthesis, you can think of each wave in a wavetable each packed with different intensities of sine waves, and each as a particular instance. If you have a very long wavetable, that you run through fast enough, it is somewhat like running through an audio “sample”.

Wavetables in practice also generally have the ability to be repitched, playing the wave instances faster or slower, but running through the table at the same rate. Or you can change speed, playing the wave instances at the same speed, but running through the table faster. This then also starts to resemble some aspects of granular synthesis.

[ I wrote a very simple Python program, ~50 lines of code, that takes a simple collection of numbers that represents a combination at different intensities of various different frequency sine waves and adding them all together, normalizing that sum, and putting it in one position of a wavetable. ]

The brain and our sense organs also convert audio pressure waves, into other formats, one for neural transmission and others to represent what was heard in the neural system of the brain. We can learn how to do some of that in different ways, some of that is just built-in.

A good example of hardware that does additive synthesis is the Rossum Panharmonium. Thread It actually includes both sides, so can go audio to spectral, and then back from spectral to audio. It can add up to 33 oscillators, and allows you to play with the data in between.

More posts to follow.

1 Like