Month: February 2010

A few weeks back I saw an interesting music app written up on lifehacker, and I bookmarked it to download and play around with during some of my ample free time. That app was RGBMusicLab, and it is no longer available.

Why is it no longer available? Because someone contacted the programmer, Kenji Kojima, and notified him that they had patented the act of turning the colors Red, Green, and Blue into music and that they planned on pursuing legal action if Kojima didn’t purchase a license and credit the patent holder in any public exhibitions of artworks generated by the code that Kojima himself wrote.

The patent in question is here. I’m not an Intellectual Property lawyer, so I won’t claim any insight into the validity of these claims, but my general feeling on the matter is: that sucks. If the act of converting a visual state into an aural state is patentable, then why, for example, hasn’t someone patented the act of reading aloud in English?

Of course, this kind of thing happens every day, but the distinguishing factor of this particular instance, for me, was the way in which Kojima communicated the removal. He removed the program’s page, and replaced it with the following simple message:

“Dear people,
I was informed that I had to remove RGB MusicLab from the web site from a person who had a patent that enabling the interpretation of color as music. I have never known it was a patent technology. And I believed such a basic process belonged to all humankind. Moreover RGB MusicLab was realized by many other great technologies of anonymous. Though I have to remove it by the US law.
Kenji Kojima”

The link to the zip file, which had by this time migrated to all sorts of download sites, was replaced by a new zip, containing only the above message as a text file.

I will admit: I am definitely guilty of having Idea Inaction Remorse (I just made that up, but it sounds real) – meaning I have ideas, don’t do anything with them and eventually see them realized by others. The leap I cannot understand is taking that next step – to aggressively pursue shutting down another person who had the perseverance to carry things through to fruition.

It occurs to me that anyone with an approved and adequately vague patent can sit around all day searching the latest feeds for new creative projects to try to extort licensing fees from, and that’s just kind of sad. As Kojima said in an email:

“Color to music is intellectual property of art/science research. And I shared my studies with people. It must not be regulated by a US patent. All artist in 21st century have be a refugee from US.”

I did eventually get my hands on the software (I played around with the mac version), and you know what? It’s great. It’s a solid, feature-rich, educational program that was being offered for free across multiple platforms (Kojima used Runtime Revolution to port his code between mac and PC versions). Too bad.

Below is a screen shot of what the interface looks like encoding a photo:

You turn your image into a mosaic of adjustable resolution, and choose an instrument, volume and panning to assign to the colors red, blue and green. The music is generated by iterating through each pixel of the mosaic – row by row, left to right. For each pixel, a three note harmony is played, with the length of the note determined by the brightness of the pixel. It’s interesting to try to follow along with the generated music by ‘sight-reading’ the mosaic based on the colors of each square, and the user can adjust the tempo and add rests between lines to make this easier.

Since the program is unavailable and you can’t try it for yourself, here’s a simplified version of the mosaic above, immediately followed by the MP3 I generated. In this example, the part of Red was played by an Acoustic Grand Piano, Blue was a Slap Bass, and Green was a Trumpet:

[audio:http://www.kempa.com/wp/wp-content/uploads/2010/02/RGBMusicLab-Demo.mp3|titles=RGBMusicLab Demo]

The good news: Kenji Kojima has tons of interesting multimedia works on his website, and he says he’s working on a similar program to interpret 3d models. Unfortunately, an echo of the chilling effect that intellectual property ugliness can yield came through loud and clear in an email Kojima sent me describing his current work: “I hope nobody hold unreasonable patent of it.”

Update: The patent owner who was threatening action failed to pay maintenance fees on the patent in question and it has now expired, so RGBMusicLab is back up! Yay!