Custom Data-Types in Max Part 4: Passing Object Pointers

How do you pass data between objects in Max?  If the data is a simple number or a symbol then the answer is easy.  What happens when you are trying to pass around audio vectors, dictionaries, images, or some other kind of object?  The implementation of Jamoma Multicore for Max deals with these issues head-on as it provides an illustration of how this problem can be tackled.

This is the fourth article in a series about working with custom data types in Max.  In the first two articles we laid the groundwork for the various methods by discussing how we wrap the data that we want to pass.  The third article demonstrated the use of Max’s symbol binding as means to pass custom data between objects.  This article will show an example of passing pointers directly between objects without using the symbol table.  In this series:

  1. Introduction
  2. Creating “nobox” classes
  3. Binding to symbols (e.g. table, buffer~, coll, etc.)
  4. Passing objects directly (e.g. Jamoma Audio Graph)
  5. Hash-based reference system (similar to Jitter)

A Peer Object System

Jamoma Audio Graph for Max is implemented as what might be called a Peer Object System.  What is meant is that for every object that a user creates and manipulates in a Max patcher, there is a matching object that exists in a parallel system. As detailed in Designing an Audio Graph, a Jamoma Audio Graph object has inlets and outlets and maintains connections to other objects to create a graph for processing audio through the objects.  The implementation of Jamoma Audio Graph for Max then has the task of creating and destroying these objects, sending them messages, and making the connections between them.  Once the objects are connected Jamoma Audio Graph will take care of itself.   The end result is that no audio processing actually happens in the Max objects for Jamoma Audio Graph — instead the Max objects are a thin façade that helps to set up the relationships between the objects as they exist in something akin to a parallel universe.

A Patcher

A Jamoma Multicore patcher in Max For context, let’s take a look at a Max patcher using Jamoma Audio Graph. In this patcher we have 4 Jamoma Audio Graph objects, identified by the ≈ symbol at the tail of the object name.  Each of these Max objects have a peer Audio Graph object internal to themselves.  Each Audio Graph object then contains a Jamoma DSP object that performs the actual signal processing. For example, the jcom.overdrive≈ object contains a pointer to a  Jamoma Audio Graph object that contains an instance of the Jamoma DSP overdrive class.  The attributes of the overdrive class, such as bypass, mute, and drive are then exposed as Max attributes so that they can be set in the patcher. Remember that each connection may carry N channels of audio.  The jcom.oscil≈ is, in this case, producing a stereo signal which is then propagated through the processing graph down to the jcom.dac≈ object.

Configuring the Graph

The exciting work doesn’t begin until the start message is sent to the jcom.dac≈ object.  As with all Jamoma Audio Graph externals, jcom.dac≈ Max external has a peer object.  In this case the peer object that it wraps is the multicore.output object.  This is the same multicore.output object that is shown in the Ruby examples in the Designing an Audio Graph article. When the start message is sent, the jcom.dac≈ object performs the following sequence:

  1. Send a multicore.reset message to all objects in the patcher.  This message sends a reset message to the peer objects underneath, which tells them to forget all of their previous connections.
  2. Send a multicore.setup message to all objects in the patcher.  This message tells the objects to try and connect to any object below it in the patcher.
  3. Tell the audio driver to start running.  When it is running it will periodically request blocks of samples from us, which in turn means that we will ask the other objects in the graph to process.

The processing happens completely within the Jamoma Multicore objects, thus not involving the Max objects at all.  It is the set up of the network of objects in the graph (steps 1 and 2) that involve our passing of custom data types in Max.

Diving into the code

For a full source listing of the jcom.dac≈ object, you can find the code in Jamoma Audio Graph source code repository.  We’ll abstract the important parts from that code below.  Let’s start with the method that is executed when the start message is sent:

TTErr DacStart(DacPtr self)
{
	MaxErr			err;
	ObjectPtr		patcher = NULL;
	long			vectorSize;
	long			result = 0;
	TTAudioGraphInitData	initData;

	self->multicoreObject->mUnitGenerator->getAttributeValue(TT("vectorSize"), vectorSize);

 	err = object_obex_lookup(self, gensym("#P"), &patcher);
 	object_method(patcher, gensym("iterate"), (method)DacIterateResetCallback, self, PI_DEEP, &result);
 	object_method(patcher, gensym("iterate"), (method)DacIterateSetupCallback, self, PI_DEEP, &result);

 	initData.vectorSize = vectorSize;
 	self->multicoreObject->init(initData);
 	return self->multicoreObject->mUnitGenerator->sendMessage(TT("start"));
}

As previously discussed, the last we thing we do is send a start message to our peer object, the multicore.output, so that the audio driver will start pulling audio vectors from us. Prior to that we iterate the Max patcher recursively (so the messages go to subpatchers too) to send the multicore.setup and multicore.reset messages.   To do this, we send the iterate message to the patcher and pass it a pointer to a method we define.  Those two methods are defined as follows.

void DacIterateResetCallback(DacPtr self, ObjectPtr obj)
{
	TTUInt32	vectorSize;
	method		multicoreResetMethod = zgetfn(obj, gensym("multicore.reset"));

	if (multicoreResetMethod) {
		self->multicoreObject->mUnitGenerator->getAttributeValue(TT("vectorSize"), vectorSize);
		multicoreResetMethod(obj, vectorSize);
	}
}

void DacIterateSetupCallback(DacPtr self, ObjectPtr obj)
{
	method multicoreSetupMethod = zgetfn(obj, gensym("multicore.setup"));

	if (multicoreSetupMethod)
		multicoreSetupMethod(obj);
}

These functions are called on every object in the patcher.  If we start with the last function, we can see that we first call zgetfn() on the object, obj, which is passed to us.  If that object possesses a multicore.setup method then we will receive a pointer to that method.  Otherwise we receive NULL.  If that method exists then we call it. The multicore.reset method works the same way.  The only difference is that the method takes an additional argument — the vector size at which the jcom.dac≈ is processing.

The Other End

At the other end of this calling sequence are the remaining objects in the patcher.  The full jcom.oscil≈ source code will show how this Max object is implemented.  In brief, we have two message bindings in the main function:

	class_addmethod(c, (method)OscilReset, "multicore.reset",	A_CANT, 0);
	class_addmethod(c, (method)OscilSetup, "multicore.setup",	A_CANT,	0);

These two methods respond to the those called by the jcom.dac≈ object.  They both have an A_CANT argument signature, which is how you define messages in Max that use function prototypes different than the standard  method prototypes.  These messages can’t be called directly by the user, and they are not listed in the object assistance, but we can send them from other parts of Max such as our jcom.dac≈ object.  The reset message (for forgetting about all previous connections) is simply passed on to the oscillator’s Multicore peer object:

TTErr OscilReset(OscilPtr self)
{
	return self->multicoreObject->reset();
}

The setup method, as we discussed, tells our object that we need to try and make a connection to any object below us in the patcher. To do this we wrap our peer Multicore object’s pointer up into a Max atom.  That, together with the outlet number (zero), are passed as arguments to the multicore.connect message which is sent out our outlet.

TTErr OscilSetup(OscilPtr self)
{
	Atom a[2];

	atom_setobj(a+0, ObjectPtr(self->multicoreObject));
	atom_setlong(a+1, 0);
	outlet_anything(self->multicoreOutlet, gensym("multicore.connect"), 2, a);
	return kTTErrNone;
}

One More Time…

That took care of the jcom.oscil≈ object.  Once it sends the multicore.connect message out its outlet, its work is done.  But what happens with that message when it is received?

In our example it is going to a jcom.overdrive≈ object.  The source code for jcom.overdrive≈ isn’t going to be very helpful though.  It uses a magic class-wrapper that wraps any Jamoma DSP object as a Multicore object using 1 line of code.  That’s really convenient for coding, but not for seeing how all of the parts communicate.  So for our discussion, we will look at the jcom.dcblocker≈ source code instead — beginning with the main() function.

	class_addmethod(c, (method)DCBlockerReset,	"multicore.reset",	A_CANT, 0);
	class_addmethod(c, (method)DCBlockerSetup,	"multicore.setup",	A_CANT, 0);
	class_addmethod(c, (method)DCBlockerConnect,	"multicore.connect",	A_OBJ, A_LONG, 0);

You should recognize the multicore.reset and multicore.setup messages.  Those are exactly the same as they were for our oscillator.  We now also have a multicore.connect message.  The oscillator was generating a signal but has no signal inputs, so it had no need for a multicore.connect message.  Any object that requires an input, however, will require this message binding.  How that method is implemented?

TTErr DCBlockerConnect(DCBlockerPtr self, TTMulticoreObjectPtr audioSourceObject, long sourceOutletNumber)
{
	return self->multicoreObject->connect(audioSourceObject, sourceOutletNumber);
}

We simply wrap a call to our peer object’s connect method, sending the audioSourceObject (which is the peer object that the jcom.oscil≈ object sent us), and the outlet number from which that object was sent.  If you compare this to the connect message from the Ruby example in Designing an Audio Graph, it may illuminate the process.

Some Final Details

The example code that we’ve seen from Jamoma Audio Graph demonstrates the passing of custom data (pointers to C++ objects) from one object to the next through the multicore.connect message. Because we are sending this custom data type, and not all inlets of all objects will understand this data type, it would be nice if we could protect users from hooking up the objects in a way that will not function.  For this task, Max makes it possible to give outlets type information.  When the type of an outlet is specified, a user will not be able to connect the patch cord to any inlet that doesn’t accept the specified message. To get this functionality, in DCBlockerNew(), we create our outlet like this:

	self->multicoreOutlet = outlet_new(self, "multicore.connect");

So instead of the customary NULL for the argument to outlet_new(), we specify that this outlet will be sending only multicore.connect messages.

Surfacing for Air

Jamoma Audio Graph provides a fairly intense example of passing custom data types in Max.  However, it presents not just the basics of how you would pass a pointer, but rather a context for why you might want to pass a custom type, and a real-world example to show what you can do.  I think that objective has been accomplished.

Designing an Audio Graph

In previous articles about the Jamoma Platform and the Jamoma DSP Library, there have been references to Jamoma Audio Graph (also previously known as Jamoma Multicore).  Up to this point, Jamoma Audio Graph has not been significantly documented or written about.  The authoritative information has been an Electrotap blog post showing the initial prototype in 2008.

At a workshop in Albi in 2009 we attempted to further expand Jamoma Audio Graph — and we failed.  The architecture was not able to handle N multichannel inputs and M multichannel outputs.  So we had to redesign a major portion of the inner-workings.  Get out your pipe wrench; it’s time to take a look at some plumbing…

What Is Jamoma Audio Graph ?

Let’s back up for moment to get the big picture.  The Jamoma Platform is essentially a layered architecture implementing various processes for interactive art, research, music, etc.  At the lowest level, the Jamoma Foundation delivers basic components for creating objects, passing values, storing values in lookup-tables, etc.  The Jamoma DSP library then extends the Foundation classes and provides a set of pre-built objects for audio signal processing.

Jamoma Audio Graph then gives us the ability to create Jamoma DSP objects and combine them into a graph.  In other words, we can connect the objects together like you might connect modules together on a Moog synthesizer.

A Moog Modular patch. Photo: Maschinenraum

A Moog Modular Patch. Photo: Maschinenraum

Unlike the Moog synthesizers of old, however, we can do a few new tricks.  Instead of sending a single channel of audio through a connection, we can send any number of channels through a connection.  While Jamoma Audio Graph does not currently implement any particular features for parellel processing on multiple cores/processors, the design of the system is ideal for such parallelization in the future.

The Audio Graph In Action

At the time of this writing, Jamoma Audio Graph has bridges to make it available in the Max and Ruby environments.  Most of the work is done on making it available to Pd as well (though if you are really interested in this then let us know so we can put you to work!).

In Ruby, you can code scripts that are executed in a sequence.  This provides a static interface to Jamoma Audio Graph even though all of the synthesis and processing is typically happening in real-time.  Alternatively, the irb environment allows you to type and execute commands interactively.  Jamoma Audio Graph, together with irb, then functions much like the ChucK environment for live coding performance.

Example

If you’ve been jonesin’ for an Atari/Amiga/Commodore fix then this might be your perfect example of Jamoma Audio Graph in Ruby:

# This is the standard require for the Jamoma Platform's Ruby bindings
require 'TTRuby'

# Create a couple of objects:
dac = TTAudio.new "multicore.output"
osc = TTAudio.new "wavetable"

# connect the oscillator to the dac
dac.connect_audio osc

# turn on the dac
dac.send "start"

# play a little tune...
osc.set "frequency", 220.0
sleep 1.0
osc.set "frequency", 440.0
sleep 1.0
osc.set "frequency", 330.0
sleep 0.5
osc.set "frequency", 220.0
sleep 2.0

# all done
dac.send "stop"

It’s a pretty cheesy example, but it should give you a quick taste.  If you want a flashback to kinds of music you could make with MS-DOS, be sure you set the oscillator to use a square waveform.

After creating a couple of objects, you connect two objects by passing the source object to the destination object using a connect message.  If you provide no further arguments, then the connection is made between the first outlet of the source object and first inlet of the destination object.  The inlets and outlets are numbered from zero, so the connect message in our example could also have been written as

dac.connect osc, 0, 0

The sleep commands are standard Ruby.  They tell Ruby to pause execution for the specified number of seconds.  Everything else is performed with the basic Jamoma Ruby bindings.  These provide the send method for sending messages and the set method for setting attribute values.

If you want to know the messages or attributes that an object possesses, you can use the messages? or attributes? methods.  This is particularly useful when coding on the fly in irb.  In the following example, I requested the list of attributes for the oscillator in the previous example:

>> osc.attributes?
=> ["gain", "mode", "size", "processInPlace", "maxNumChannels", "frequency", "mute", "interpolation", "sr", "bypass"]

How It Operates

If you create a visual data-flow diagram of the objects in a graph, like you would see in Max or PureData, then you would get a good sense of how audio starts at the top and works its way through various filters until it gets to the bottom.  The same is true for a Jamoma Audio Graph.  However, what is happening under the surface is exactly the opposite.

Pull I/O Model

Multicore Graph Flow

The flow of a Jamoma Audio Graph.

Jamoma Audio Graph is based on a “Pull” I/O Model.  Some other examples of audio graph solutions using a similar model include ChucK and Apple’s AUGraph.  In this model a destination, sink, or terminal node object sits at the bottom of any given graph — and this is the object driving the whole operation.  In Max, on the other hand, messages (e.g. a ‘bang’ from a metro) begins at the top of the graph and pushes down through the objects in the chain.

The image to the left visualizes the operation of the audio graph.  Let’s assume the the destination object is an interface to your computer’s DAC.  The DAC will request blocks of samples (vectors) every so often as it needs them.  To keep it simple, we’ll say that we are processing samples at a sample rate of 44.1KHz and a block size of 512 samples.  In this case, every 11 milliseconds the DAC will tell our destination object that it needs a block of samples and the process begins.

The process flows through the light blue lines.  The destination asks the limiter for a block of samples, which then asks the overdrive for a block of samples which then asks both the source and the multitap delay for samples, and then the multitap delays asks the source for a block of samples.  To summarize it: each object receives a request for a block of samples, and in response it needs to produce that block of sample values, possibly pulling blocks of samples from additional objects in the process.

One Object At A Time

To understand in finer detail what happens in each object, the graphic below zooms-in to view a single instance in the graphic above.  Here we can see that we have the actual unit generator, which is a Jamoma DSP object, and then a host of other objects that work to make the interface for the audio graph.

Anatomy of a Multicore Object

Jamoma Audio Graph class structure

The text in graphic explains each of the classes contained in a Jamoma Audio Graph object.  Implied in both of the figures, is the ability to handle “fanning” connections where many inlets are connected to an outlet, or an inlet is connected to many outlets.

In essence, the outlets are only buffers storing samples produced by the unit generator.  Each time a block is processed the unit generator is invoked only once.  Subsequent requests for the object’s samples then simply access the samples already stored in the outlet buffers.

As explained in the graphic, the inlets have more work to do, as they need to sum the signals that are connected.  And remember, each connection can have zero or more channels!

Benefits

The most obvious  benefit is the ability to easily handle multiple channels in a single connection.  So imagine that you create a Max patcher for mono operation.  It can then function in stereo or 8-channel or 32-channel without a single modification.

But there’s a lot more than that here.  The number of channels is dynamic and can change at any time.  One place this is valuable is in ambisonic encoding and decoding where the order of the encoding can dramatically alter the number of channels required for the encoded signal.  If you want to try changing the ambisonic order on-the-fly, which changes the number of channels passed, you can.

Likewise, the vectorsize can be altered dynamically on a per-signal basis.  The benefit here may not be immediately obvious, but for granular synthesis, spectral work, and analysis based on the wave length of an audio signal (e.g. the kinds of things in IRCAM’s Gabor) this can be a huge win.

Writing the objects is also very simple.  If you write a Jamoma DSP object, then all you have to do to make it available in Jamoma Audio Graph is…

Nothing!

That’s right.  In Ruby, for example, all Jamoma DSP classes are made available with no extra work.  If you want to make a Max external for a particular object then you can use a class wrapper (1 line of code) to create the Max external.

Interested in join the fun?  Come find us!

The Jamoma Platform

In the series Custom Data-Types in Max there is frequent reference to Jamoma.  Jamoma is “A Platform for Interactive Art-based Research and Performance”.  Sounds great, right?  But what does it mean by “A Platform”?  How is  it structured?  What are the design considerations behind Jamoma’s architecture?  Many people are aware of some of Jamoma’s history or what it was in the past, but it has come a long way in the last couple of years.

The Jamoma Platform

The Jamoma Platform comprises a group of projects addressing the needs of composers, performers, artists, and researchers.  These projects are orchestrated in a number of layers with each layer dependent on the layers below it, but the layers below not dependent upon the layers above them.

jamoma-layers

The modular framework is built on top of the Max environment while others are completely independent of Max.  For example, the Jamoma DSP layer is actually used to write objects for Pd and SuperCollider, plug-ins in the VST and AU formats, C++ applications in addition to creating objects for Max or user by the Jamoma Modular Framework.

The modular layer also bypasses some intermediary layers, which is indicated in this graphic with the lines that directly connect the layers.

Let’s take a look at each of these layers (bypassing the System Layer).

Jamoma DSP Layer

At the bottom of the stack is the Jamoma DSP Layer, also known as TTBlue for historical reasons.  The DSP layer, logically enough, is where all signal processing code for Jamoma is written in C++.  There is a library of processing blocks and utilities from which to draw.  The library is extensible and can load third-party extensions to the system dynamically or at start-up.  Finally, the DSP Layer is more than just a bunch of DSP processing blocks: it includes an entire reflective OO environment in which to create the processing blocks and send them messages.

All by itself the Jamoma DSP Library doesn’t actually do anything, because it is completely agnostic about the target environment.  The Jamoma DSP repository includes example projects that can wrap or use the DSP library in Max/MSP, Pd, SuperCollider, VST and AU plug-ins, etc.  In some cases there are class wrappers that will do this in one line of code.  In all of these examples, the DSP library is used, but no other part of Jamoma is required, nor will it ever be required, as we keep a clear and firm firewall between the different layers.

Jamoma Multicore Layer

Jamoma Multicore, hereafter we’ll simply say ‘Multicore’, is built on top of the DSP layer.  Multicore creates and manages graphs of Jamoma DSP objects to produce signal processing chains.  One can visualize this as an MSP patcher with lots of boxes connected to each other, patchcords fanning and combining, generator objects feeding processing objects etc.  Multicore does not, however, provide any user interface or visual representation; it creates the signal processing graph in memory and performs the actual operations ‘under-the-hood’.

At this time I would describe the status of the Multicore layer as “pre-alpha” – meaning it is not very stable and is in need of further research and development to fulfill its vision.

Jamoma Modular

When most people say Jamoma, they typically are referring to the Jamoma Modular Layer, and more specifically the Jamoma Modular Framework.  The Modular framework provides a structured context for fully leveraging the power of the Max/MSP environment.  The modular layer consists of both the modular framework and a set of components (external objects and abstractions).  The components  are useful both with and without the modular framework.

To exemplify the Modular Components, we can consider the jcom.dataspace external.  This is an object that converts between different units of representation for a given number (e.g. decibels, linear gain, midi gain, etc.).  This is a useful component in Max/MSP regardless of whether the modular framework is being used for patcher construction or management.

The Modular Framework, on the other hand, is system of objects and conventions for structuring the interface of a Max patcher – both the user interface and the messaging interface.

9a608

The screenshot above (from Spatial sound rendering in Max/MSP with ViMiC by N. Peters, T. Matthews, J. Braasch & S. McAdams) demonstrates the Jamoma framework in action.  There a number of modules connected together in a graph to render virtual microphone placements in a virtual space.  The module labeled ‘/cuelist’ communicates remotely with the other modules to automate their behavior.

Digging Deeper

In future articles I’ll be treating the architecture of each of these layers in more detail.  I also will be demoing Jamoma at the Expo ’74 Science Fair next week.  If you are going to be at Expo ’74, be sure to stop by and say hello.

Custom Data-Types in Max Part 1: Introduction

The Max API makes it easy to pass a handful of standard data types in Max: ints, floats, symbols, lists of the aforementioned.  But what happens when you want to pass a frog from one object to the next?  A frog is not a standard data type.  Instead it is something made up that we want to send hopping through our Max patch from one green object box to the next.

Where do we start?

Before we can pass the frog from one object to another, we first need to define the frog type.  What is it?  Is it an object (meaning a proper Max class with a t_object as it’s first member)?  Or is it a naked struct or C++ class?  Or something else entirely?  Are we passing the data by value, or by reference (meaning a pointer)?

That last question may be more difficult that it seems at first glance.  Answering the question may help to determine the answer to the other questions.  If we pass by value then we have a certain amount of simplicity, but for anything other than rudimentary types it quickly becomes a very computational expensive situation.  So the obvious answer here is to pass by pointer, right?  Not so fast…  Consider the following patcher topology:

simple-patcherIf we pass by value from the first number object, then we get the results that are shown.  If we simply pass a pointer to the value (pass by reference) without some sort of management in place then we will get very different results.  The result could be the following:

  1. The address of the data (2) is passed to the [+ 5] object.
  2. 5 is added to 2, the data now has a value of 7 and this new value is passed to the lower-right number box.
  3. The address of the data (which now has the value 7!) is now passed to the [+ 7] object.
  4. 7 is added to 7, the data now has a value of 14(!) and this new value is passed to the lower-left number box.

Indeed.  A subtle problem with real life consequences.  In our example the problem may seem trivial, but when you are operating on more complex structures (e.g. Jitter or FTM) then there needs to be a system in place that allows for the graph to bifurcate without downstream operations corrupting the output of other operations happening ‘in parallel’.

Series Overview

This introduction to the problem is the first of a multi-part series. Over the next several weeks I will be writing about several different approaches to passing custom data types in Max, and I’ll be using some real-world examples to demonstrate how and why these various strategies are effective.

  1. Introduction
  2. Creating “nobox” classes
  3. Binding to symbols (e.g. table, buffer~, coll, etc.)
  4. Passing objects directly (e.g. Jamoma Multicore)
  5. Hash-based reference system (similar to Jitter)