The Man Behind The "Scrambled Hackz"

I saw a video the other day that really stood out from the rest of the links making the rounds.

It depicts a man demonstrating software that appears to parse what he's saying fast enough to reassemble the same words by pulling and reordering bits from a recorded Michael Jackson interview. The result: Jackson appears to speak the same sentence right back to him.

The man goes on to explain how the software behind this process works, and his video closes with a live performance of the software in which a performer appears to employ the beat-box method to control the playback of audio and video on a large video screen behind him, in front of what I can only imagine must be a dazzled crowd.

If you haven't seen it yet, you can watch the video here.

It might look like a fancy parlor trick, but if the Scrambled Hackz (or "sCrAmBlEd?HaCkZ!") software does what it appears to do in the video, its ramifications extend far beyond. Sven König does a fine job of explaining how the software works in the above video, but here's the general idea:

1) Scrambled Hackz analyzes the audio portion of a video file to determine the tempo of the incoming audio, and then slices it up into discrete chunks of a quarter note, eighth note, sixteenth note and so on (a process also used by audio editing programs such as Ableton Live and Sony's Acid software).

2) Using a large number of vectors, those slices are classified into a database according to their sonic characteristics.

3) When you send new audio information to the program (using, say, your voice and a microphone), it follows approximately the same process, becoming classified in the database. The software then outputs the pre-analyzed sample that is most similar to that newly cached sample.

4) The result, as you can see in the video, is that König is able to reconfigure a Michael Jackson interview or any number of '80s music videos on the fly, so that they produce a sound similar to whatever he inputs. On screen, the software plays the frames of video that accompany the selected audio.