What sound processing algorithm allows you to do such visualizations?

Question

I am interested in creating an OpenGL visualizer for MP3 as a pet project.

I came across this youtube video that demonstrates that someone is demonstrating a visualizer that is used along with augmented reality.

http://www.youtube.com/watch?v=SnshyLJSpnc#t=1m15s

Please watch this video, but do not consider the extended aspect of this video. I'm only interested in creating a visualizer, not augmented reality.

What algorithms were used to create these patterns in relation to music? If you look, you can see what looks like several different visualization methods. The first has a clear view:

The first looked like waves moving in the rendering area:

Another “mode” seemed to visualize centered movement in concentric circles:

Anyone who is well versed in the audio program, what algorithms can be used to create similar visualizations? Which algorithm did the first use? Or with concentric circles?

Any help in pointing out which algorithms were used to create these visualizations based on music would help me a lot!

Answer 1

Firstly, all of them are apparently based on FFT (Fast-Fourier Transforms) algorithms that can receive a sound wave for a specific time fragment and divide it into a linear graph of the XY spectrum, where X represents the frequency spectrum (usually log-based from 20 to 20,000 Hz), and Y represents the amplitude or volume of sound at each different frequency.

If you look at the very first visualizations (flat, colorless earlier in the video), you will see exactly this in its unvarnished form. You will notice that the lower notes appear as peaks and peaks on the left side, while the higher notes appear in the middle and on the right, which is the classic Fourier transform. (Actually, the biggest mistake in this video is that in the second half after the introduction, FFT mapping from left to right is wrong, so most of the highest and smallest notes cut off the left and right edges of the visualization).

From here, he simply adds different and gradually more sophisticated decorations to this basic trick. First he adds a very basic color matching: the height of the waveform is mapped directly to its color: from red (lowest) to dark blue / indigo (highest), following the classic ROYGBIV pattern (red, orange, yellow, green, blue, indigo , Violet). Remember that pitch also corresponds to volume at that frequency. As far as I can tell, it uses the same color display without changing.

All subsequent decorations and variations seem to be just different ways of playing with progressive time display. Initially, it simply displays waveforms in the front of the visual area, and then gradually displaces them, so that it effectively creates a continuous three-dimensional surface graph, the frequency of which works from left to right, volume (and color) going from bottom to top and time running from front to back. This is what you have in the first shot.

Everything else is more and more complex versions of it, matching time (and time) in only more complex ways. For example, in the circular one, which you show secondly, I believe that it displays time in a very fast radial sweep pattern around the obvious pole in the middle.

Answer 2

Take a look at Princeton Soundlab sndpeek - some basic but effective waterfall visualization effects implemented in OpenGL, and the source is pretty educational.