Designing an Audio Graph

In previous articles about the Jamoma Platform and the Jamoma DSP Library, there have been references to Jamoma Audio Graph (also previously known as Jamoma Multicore).  Up to this point, Jamoma Audio Graph has not been significantly documented or written about.  The authoritative information has been an Electrotap blog post showing the initial prototype in 2008.

At a workshop in Albi in 2009 we attempted to further expand Jamoma Audio Graph — and we failed.  The architecture was not able to handle N multichannel inputs and M multichannel outputs.  So we had to redesign a major portion of the inner-workings.  Get out your pipe wrench; it’s time to take a look at some plumbing…

What Is Jamoma Audio Graph ?

Let’s back up for moment to get the big picture.  The Jamoma Platform is essentially a layered architecture implementing various processes for interactive art, research, music, etc.  At the lowest level, the Jamoma Foundation delivers basic components for creating objects, passing values, storing values in lookup-tables, etc.  The Jamoma DSP library then extends the Foundation classes and provides a set of pre-built objects for audio signal processing.

Jamoma Audio Graph then gives us the ability to create Jamoma DSP objects and combine them into a graph.  In other words, we can connect the objects together like you might connect modules together on a Moog synthesizer.

A Moog Modular patch. Photo: Maschinenraum

A Moog Modular Patch. Photo: Maschinenraum

Unlike the Moog synthesizers of old, however, we can do a few new tricks.  Instead of sending a single channel of audio through a connection, we can send any number of channels through a connection.  While Jamoma Audio Graph does not currently implement any particular features for parellel processing on multiple cores/processors, the design of the system is ideal for such parallelization in the future.

The Audio Graph In Action

At the time of this writing, Jamoma Audio Graph has bridges to make it available in the Max and Ruby environments.  Most of the work is done on making it available to Pd as well (though if you are really interested in this then let us know so we can put you to work!).

In Ruby, you can code scripts that are executed in a sequence.  This provides a static interface to Jamoma Audio Graph even though all of the synthesis and processing is typically happening in real-time.  Alternatively, the irb environment allows you to type and execute commands interactively.  Jamoma Audio Graph, together with irb, then functions much like the ChucK environment for live coding performance.

Example

If you’ve been jonesin’ for an Atari/Amiga/Commodore fix then this might be your perfect example of Jamoma Audio Graph in Ruby:

# This is the standard require for the Jamoma Platform's Ruby bindings
require 'TTRuby'

# Create a couple of objects:
dac = TTAudio.new "multicore.output"
osc = TTAudio.new "wavetable"

# connect the oscillator to the dac
dac.connect_audio osc

# turn on the dac
dac.send "start"

# play a little tune...
osc.set "frequency", 220.0
sleep 1.0
osc.set "frequency", 440.0
sleep 1.0
osc.set "frequency", 330.0
sleep 0.5
osc.set "frequency", 220.0
sleep 2.0

# all done
dac.send "stop"

It’s a pretty cheesy example, but it should give you a quick taste.  If you want a flashback to kinds of music you could make with MS-DOS, be sure you set the oscillator to use a square waveform.

After creating a couple of objects, you connect two objects by passing the source object to the destination object using a connect message.  If you provide no further arguments, then the connection is made between the first outlet of the source object and first inlet of the destination object.  The inlets and outlets are numbered from zero, so the connect message in our example could also have been written as

dac.connect osc, 0, 0

The sleep commands are standard Ruby.  They tell Ruby to pause execution for the specified number of seconds.  Everything else is performed with the basic Jamoma Ruby bindings.  These provide the send method for sending messages and the set method for setting attribute values.

If you want to know the messages or attributes that an object possesses, you can use the messages? or attributes? methods.  This is particularly useful when coding on the fly in irb.  In the following example, I requested the list of attributes for the oscillator in the previous example:

>> osc.attributes?
=> ["gain", "mode", "size", "processInPlace", "maxNumChannels", "frequency", "mute", "interpolation", "sr", "bypass"]

How It Operates

If you create a visual data-flow diagram of the objects in a graph, like you would see in Max or PureData, then you would get a good sense of how audio starts at the top and works its way through various filters until it gets to the bottom.  The same is true for a Jamoma Audio Graph.  However, what is happening under the surface is exactly the opposite.

Pull I/O Model

Multicore Graph Flow

The flow of a Jamoma Audio Graph.

Jamoma Audio Graph is based on a “Pull” I/O Model.  Some other examples of audio graph solutions using a similar model include ChucK and Apple’s AUGraph.  In this model a destination, sink, or terminal node object sits at the bottom of any given graph — and this is the object driving the whole operation.  In Max, on the other hand, messages (e.g. a ‘bang’ from a metro) begins at the top of the graph and pushes down through the objects in the chain.

The image to the left visualizes the operation of the audio graph.  Let’s assume the the destination object is an interface to your computer’s DAC.  The DAC will request blocks of samples (vectors) every so often as it needs them.  To keep it simple, we’ll say that we are processing samples at a sample rate of 44.1KHz and a block size of 512 samples.  In this case, every 11 milliseconds the DAC will tell our destination object that it needs a block of samples and the process begins.

The process flows through the light blue lines.  The destination asks the limiter for a block of samples, which then asks the overdrive for a block of samples which then asks both the source and the multitap delay for samples, and then the multitap delays asks the source for a block of samples.  To summarize it: each object receives a request for a block of samples, and in response it needs to produce that block of sample values, possibly pulling blocks of samples from additional objects in the process.

One Object At A Time

To understand in finer detail what happens in each object, the graphic below zooms-in to view a single instance in the graphic above.  Here we can see that we have the actual unit generator, which is a Jamoma DSP object, and then a host of other objects that work to make the interface for the audio graph.

Anatomy of a Multicore Object

Jamoma Audio Graph class structure

The text in graphic explains each of the classes contained in a Jamoma Audio Graph object.  Implied in both of the figures, is the ability to handle “fanning” connections where many inlets are connected to an outlet, or an inlet is connected to many outlets.

In essence, the outlets are only buffers storing samples produced by the unit generator.  Each time a block is processed the unit generator is invoked only once.  Subsequent requests for the object’s samples then simply access the samples already stored in the outlet buffers.

As explained in the graphic, the inlets have more work to do, as they need to sum the signals that are connected.  And remember, each connection can have zero or more channels!

Benefits

The most obvious  benefit is the ability to easily handle multiple channels in a single connection.  So imagine that you create a Max patcher for mono operation.  It can then function in stereo or 8-channel or 32-channel without a single modification.

But there’s a lot more than that here.  The number of channels is dynamic and can change at any time.  One place this is valuable is in ambisonic encoding and decoding where the order of the encoding can dramatically alter the number of channels required for the encoded signal.  If you want to try changing the ambisonic order on-the-fly, which changes the number of channels passed, you can.

Likewise, the vectorsize can be altered dynamically on a per-signal basis.  The benefit here may not be immediately obvious, but for granular synthesis, spectral work, and analysis based on the wave length of an audio signal (e.g. the kinds of things in IRCAM’s Gabor) this can be a huge win.

Writing the objects is also very simple.  If you write a Jamoma DSP object, then all you have to do to make it available in Jamoma Audio Graph is…

Nothing!

That’s right.  In Ruby, for example, all Jamoma DSP classes are made available with no extra work.  If you want to make a Max external for a particular object then you can use a class wrapper (1 line of code) to create the Max external.

Interested in join the fun?  Come find us!

Accessing buffer~ Objects in Max5

One thing that has always been a bit tricky, and perhaps a bit under-documented, has been writing good code for accessing the contents of a buffer~ object in Max.  What has made the situation a bit more confusing is that the API has changed slowly over a number of versions of Max to make the system more robust and easier to use.  This is certainly true of Max 5, and the most recent version of the Max 5 Software Developer Kit makes these new facilities available.

I’ll be showing the favored way to access buffer~ objects for Max 5 in the context of a real object: tap.buffer.peak~ from Tap.Tools.  I’ll show how it should be done now, and in some places I’ll show how it was done in the past for reference.

Getting a Pointer

The first thing we need to do is get a pointer to the buffer~ bound to a given name.  If you know that there is a buffer~ object with the name “foo” then you could simply do this:

t_symbol* s = gensym("foo");
t_buffer* b = s->s_thing;

However, there are some problems here.  What if “foo” is the name of a table and not a buffer~?  What if there is a buffer~ named foo in the patcher, but when the patcher is loaded the buffer~ is instantiated after your object.  What if you execute the above code and then the user delete’s the buffer~ from their patch?  These are a few of the scenarios that happen regularly.

A new header in Max 5 includes a facility for eleganty handling these scenarios:

#include "ext_globalsymbol.h"

Having included that header, you can now implement a ‘set’ method for your buffer~-accessing object like so:

// Set Buffer Method
void peak_set(t_peak *x, t_symbol *s)
{
	if(s != x->sym){
		x->buf = (t_buffer*)globalsymbol_reference((t_object*)x, s->s_name, "buffer~");
		if(x->sym)
			globalsymbol_dereference((t_object*)x, x->sym->s_name, "buffer~");
		x->sym = s;
		x->changed = true;
	}
}

By calling globalsymbol_reference(), we will bind to the named buffer~ when it gets created or otherwise we will attach to an existing buffer.  And when I say attached, I mean it.  Internally this function calls object_attach() and our object, in this case tap.buffer.peak~, will receive notifications from the buffer~ object.  To respond to these notifications we need to setup a message binding:

class_addmethod(c, (method)peak_notify,		"notify",		A_CANT,	0);

And then we need to implement the notify method:

t_max_err peak_notify(t_peak *x, t_symbol *s, t_symbol *msg, void *sender, void *data)
{
	if (msg == ps_globalsymbol_binding)
		x->buf = (t_buffer*)x->sym->s_thing;
	else if (msg == ps_globalsymbol_unbinding)
		x->buf = NULL;
	else if (msg == ps_buffer_modified)
		x->changed = true;

	return MAX_ERR_NONE;
}

As you may have deduced, the notify method is called any time a buffer~ is bound to the symbol we specified, unbound from the symbol, or any time the contents of the buffer~ are modified.  For example, this is how the waveform~ object in MSP knows to update its display when the buffer~ contents change.

Accessing the Contents

Now that you have a pointer to a buffer~ object (the t_buffer*), you want to access the contents.  Having the pointer to the buffer~ is not enough, because if you simply start reading or writing to the buffer’s b_samples member you will not be guaranteed of thread-safety, meaning that all matter of subtle (and sometimes not so subtle) problems may ensue at the most inopportune moment.

In Max 4 you might have used code that looked like the following before and after you accessed a buffer~’s contents:

    saveinuse = b->b_inuse;
    b->b_inuse = true;

    // access buffer contents here

    b->b_inuse = saveinuse;
    object_method((t_object*)b, gensym("dirty"));

The problem is that the above code is not entirely up to the task.  There’s a new sheriff in town, and in Max 5 the above code will be rewritten as:

    ATOMIC_INCREMENT((int32_t*)&b->b_inuse);
    // access buffer contents here
    ATOMIC_DECREMENT((int32_t*)&b->b_inuse);
    object_method((t_object*)b, gensym("dirty"));

This is truly threadsafe. (Note that you only need to call the dirty method on the buffer to tell that it changed if you wrote to the buffer).

Here is the code from tap.buffer.peak~ that access the buffer~’s contents to find the hottest sample in the buffer:

{
	t_buffer	*b = x->buf;		// Our Buffer
	float		*tab;		        // Will point to our buffer's values
	long		i, chan;
	double		current_samp = 0.0;	// current sample value

	ATOMIC_INCREMENT((int32_t*)&b->b_inuse);
	if (!x->buf->b_valid) {
		ATOMIC_DECREMENT((int32_t*)&b->b_inuse);
		return;
	}

	// FIND PEAK VALUE
	tab = b->b_samples;			// point tab to our sample values
	for(chan=0; chan < b->b_nchans; chan++){
		for(i=0; i < b->b_frames; i++){
			if(fabs(tab[(chan * b->b_nchans) + i]) > current_samp){
				current_samp = fabs(tab[(chan * b->b_nchans) + i]);
				x->index = (chan * b->b_nchans) + i;
			}
		}
	}

	ATOMIC_DECREMENT((int32_t*)&b->b_inuse);
}