The Quartz Composer Music Visualizer Protocol was introduced by Apple in Mac OS X 10.5 Leopard. It provides a mechanism by which an audio player can communicate with a composition. Using this mechanism a composition may react to various input parameters provided by the audio player, such as peak and spectrum data, as well as track information.

AudioCodex is one of the first apps to implement the Music Visualizer Protocol. At this time, the only other app we are aware of that does so is iTunes. To make use of the protocol, an application needs to support both audio playback and Quartz Composition hosting. Presumably Apple devised the protocol primarily to allow Quartz Composer users to design visualizers for iTunes. Prior to Leopard, MachineCodex was planning to implement a similar music visualizer protocol to let compositions respond to our internal audio stream without using the microphone input. Leopard enabled us to simply adopt Apple's implementation, saving us a lot of work.

However, like most of Apple's Quartz Composer material, there is a glaring lack of documentation for users and developers who want to make use of this interesting technology. It's almost as if Apple really isn't sure how to present Quartz Composer, beyond its immediate coolness factor. This page exists to hold all the information we can glean about the protocol.

What is the Music Visualizer Protocol?

In simple terms, the protocol specifies a set of input patch keys that are required by all compositions that comply with the protocol. Several of the protocol keys are optional, and compositions may choose to implement those. See Protocol Input Keys provided by AudioCodex (below) for a list of the required and optional keys.

Using Music Visualizers within iTunes

Drop a composition which conforms to the protocol into /Library/Compositions, and re-start iTunes. It will appear in the View → Visualizer sub-menu.

Note that in order to be usable in iTunes, a composition must publish only the Music Visualizer Protocol keys and no others. iTunes appears to check if a protocol strictly conforms to the protocol before loading it.

Another limitation seems to prevent the use of compositions containing non-Apple patches within iTunes, though we can't confirm this. We have been unsuccessful in getting iTunes to recognise compositions containing third party plug-ins at this time.

Using Music Visualizers within AudioCodex

Drop a composition which conforms to the protocol into ~/Library/Application Support/AudioCodex/Compositions (or any of the default composition locations) and choose Rescan Composition Folders from the Visualizer Inspector menu. It will appear in the Module list.

As AudioCodex supports loose compliance with the protocol, compositions which publish additional keys are recognised. Compositions which contain non-Apple patches are supported as well.

Creating a Music Visualizer

It is very straightforward to create your own music visualizers using Quartz Composer. This discussion assumes you have Quartz Composer installed on your Mac.

Creating a default Music Visualizer

Open Quartz Composer and create a New composition (depending on your configuration, this may happen automatically). In the Choose a template dialog that appears, select the Music Visualizer and click Choose.

You now have a composition that will display level meters and track info. Save it to one of the default locations (described above) and you can use it immediately from AudioCodex or iTunes.

Adding Music Visualizer Protocol compliance to an existing composition

Quartz Composer makes it easy to add protocol compliance to an existing composition by providing a simple way to automatically add the required and optional input patches.

Open your existing composition in Quartz Composer. From the Editor menu, select Edit Protocol Conformance...

In the dialog sheet that appears, check Music Visualizer and click Done.

You now have a set of input patches which you can use to provide level, spectrum, and track info. Save your composition to one of the default locations (described above) and you can drive it from AudioCodex or iTunes.

Testing Music Visualizers

Testing music visualizers during development is possible, as Quartz Composer provides an audio input from the currently playing iTunes track. This gives you live data to see the effects of your work as animation occurs in the Viewer.

To use this feature, launch iTunes and begin playing a track. Then turn on the Visualizer from the view menu, and choose one of the Quartz Composition based visualizers. Switch back to Quartz Composer and you will have audio data from iTunes. Spooky, undocumented, but incredibly cool.

Protocol Input Keys provided by AudioCodex

• _protocolInput_AudioSpectrum (Required) - Updated with spectral data derived from AudioCodex's audio output stream. Currently a preliminary implementation - expect improvements to our spectral data in the future.
• _protocolInput_AudioPeak (Required) - Updated with amplitude data derived from AudioCodex's audio output stream
• _protocolInput_TrackPosition (Optional) - Updated with the current track position while the track plays
• _protocolInput_TrackInfo (Optional) - Updated with track metadata such as track name, artist, album, year and genre at the start of playback of each track
• _protocolInput_TrackSignal (Optional) - A signal (boolean) value at the start of playback of each track

Loose conformance

Because only loose compliance is required by AudioCodex, even compositions created on or for OSX 10.4.x (Tiger) can theoretically take advantage of this feature by simply adding the required input ports with the correct key names, described above, manually.

As a consequence of this, AudioCodex users on Tiger can create and use music visualizers, although they are not officially supported.

Apps that implement the Music Visualizer Protocol

• iTunes
• AudioCodex

Availability

Available in Mac OS X 10.5 or later

References

Apple provides no documentation at present relating to Music Visualizers.

I was however able to find Apple's Patent Application.