3D augmented STFT reality

ARTalƒ is real-time augmented music visualization of an album cover synchronized with the music of the album itself. It is the results of a combination of simple music information retrieval and image segmentation algorithm with augmented reality.

In practice:

  1. Colors and shapes are detected in the album picture
  2. Pitch, rhythm timber information is extracted by the audio
  3. A new multimedia content is added to the real image
  4. Such contents are selected according to the colors and animated according to the music

This project was realized as project for the Multimedia Programming course of the MSc SMC2016 at Aalborg University.

Music: Massive by Mandela Idea: Meet the soundtrack in Fantasia by Disney + fever trip

Other Resources