Inventing a New Project Format for C/C++

This past spring we rolled-out a new way compile our source code in the Jamoma project. After a number of months of real-world use in managing our code, the transition has been big success. I here propose that this system be moved beyond Jamoma for use by other software developers who care about portability, ease of use, ease of maintenance, and requisite flexibility.

The Maintenance Nightmare

We had been using Xcode projects on the Mac, Visual Studio on Windows, Makefiles on Linux, etc. The problem with maintaining all these independent projects is that a change in one requires changes in the others, which typically are changes that need to be made by someone running a different operating system. In the mean-time, the build is broken on that other platform. Or maybe instead the developer was extra conscientious and checked-in the change on a branch, using this work-flow:

  1. create new branch
  2. make change (add a file to an Xcode project)
  3. check-in the change
  4. check out the branch on a windows computer
  5. make the change in visual studio
  6. check-in the change
  7. check out the branch on a linux computer
  8. fix and test the Makefile
  9. check-in the change
  10. merge the branch to master
  11. delete the temporary branch
  12. delete the temporary branch on the Mac
  13. delete the temporary branch on the Windows machine
  14. delete the temporary branch on the Linux box
  15. Realize that, oops, forgot to update the iOS target
  16. etc.

In an open source project like Jamoma most of the work is done on an unpaid/volunteer basis.  That means a lot of these steps don’t happen in a timely manner and the result is a project that is in a state of perpetual brokenness.  (One side-effect of that state is that the project is hostile to potential new developers joining).

With our the new YAMLProj format, you simply add the file in the project on one platform, and it is added for all of the other platforms.

This is example is the YAMLProj for Jamoma’s TrajectoryLib extension.

sources:
  - TrajectoryLib.cpp
  - TTTrajectory.cpp
  - sources/Bean2D.cpp
  - sources/Hypocycloid2D.cpp
  - sources/Bspline2D.cpp
  - sources/Limacon2D.cpp
  - sources/Scarabaeus2D.cpp
  - sources/Butterfly2D.cpp
  - sources/Linear2D.cpp
  - sources/Slinky3D.cpp
  - sources/Linear3D.cpp
  - sources/Spiral3D.cpp
  - sources/Circular2D.cpp
  - sources/Lissajous2D.cpp
  - sources/Torus3D.cpp
  - sources/CircularHelix3D.cpp
  - sources/Lissajous3D.cpp
  - sources/Viviani3D.cpp
  - sources/Epitrochoid2D.cpp
  - sources/LogSpiral2D.cpp
  - sources/Gear2D.cpp
  - sources/Rose2D.cpp

includes:
  - "."
  - "includes"
  - "../../library/includes"
  - "../../../Foundation/library/includes"

libraries:
  - FOUNDATION
  - DSP

To drive the point home, here is sequence of steps for adding a file to a project, as compared to the 16+ step behemoth presented at the outset:

  1. make change (add a source file to the project)
  2. check-in the change

No need to create a branch to prevent breaking the build on other platforms (though you could if you thought there were going to be problems).

Ease of Use

In addition to easing the maintenance burden, the system must also be easy for developers of all experience levels to use.  Face it: not all developers are command-line junkies; a lot of people develop software by dragging files from the Mac Finder into Xcode and clicking the icon.  That’s a good workflow if you want a non-intimidating experience that gets a quick result and makes you feel good.

Without writing an app for umpteen platforms, some of which wouldn’t even make sense, we’re not going to have drag-n-drop gui windows.  Drag-n-drop gui windows can also be incredibly difficult to use and present the user with a complete cognitive cacophony that does not feel easy to use (e.g. Eclipse).  So dispensing with that we have text files.

Text files can be intimidating.  They have been used, misused, and abused in various ways.  Some people even have baggage associated with them.  The text file needs some sort of structure to it.  Some text file formats for creating structure are hideous (ever looked at pbxproj file?), some are unforgiving (JSON), others are just tedious (XML).  We chose YAML.  It doesn’t require massive amounts of decoration, it is clear, it is easy and straightforward.

But what about CMake?

CMake sucks.  There.  I said it.  I’ll upset some people.  Whatever.

It’s too complicated.  We don’t need our projects to fly the International Space Station.  We need to compile a few source files and have it in a text file format that is not intimidating.  In order to use CMake you probably have to compile some other open-source software, which might fail with errors that only the devs understand.  Then you need it to work for compiling a lib for the iOS or some new platform that we haven’t heard about yet and you’re screwed.

Sorry.  No CMake.  We went down that road with Jamoma and threw it out in disgust. Don’t bother trying say anything contrary.  I won’t be listening.

To the contrary, YAMLProj is super-simple, exceptionally clear, uncomplicated, and unintimidating.  All the while still addressing the cross-platform maintenance issues in a reasonable way.

Making it Go

The implementation is a simple Ruby script.  The Ruby script reads the YAMLProj, parses the YAML to generate the intermediate format and the compiles from the intermediate format.  On Windows the intermediate format it generates is a VisualStudio project.  On the Mac and Linux it generates a good old Makefile.

For completeness’ sake, here is the Makefile generated on a Mac for the aforementioned TrajectoryLib in Jamoma.

NAME = TrajectoryLib

CC_32 = g++-4.2 -arch i386
CC_64 = g++-4.2 -arch x86_64

#########################################

SRC = TrajectoryLib.cpp
SRC += TTTrajectory.cpp
SRC += sources/Bean2D.cpp
SRC += sources/Hypocycloid2D.cpp
SRC += sources/Bspline2D.cpp
SRC += sources/Limacon2D.cpp
SRC += sources/Scarabaeus2D.cpp
SRC += sources/Butterfly2D.cpp
SRC += sources/Linear2D.cpp
SRC += sources/Slinky3D.cpp
SRC += sources/Linear3D.cpp
SRC += sources/Spiral3D.cpp
SRC += sources/Circular2D.cpp
SRC += sources/Lissajous2D.cpp
SRC += sources/Torus3D.cpp
SRC += sources/CircularHelix3D.cpp
SRC += sources/Lissajous3D.cpp
SRC += sources/Viviani3D.cpp
SRC += sources/Epitrochoid2D.cpp
SRC += sources/LogSpiral2D.cpp
SRC += sources/Gear2D.cpp
SRC += sources/Rose2D.cpp

#########################################

INCLUDES = -I.
INCLUDES += -Iincludes
INCLUDES += -I../../library/includes
INCLUDES += -I../../../Foundation/library/includes

#########################################

LIBS = ../../../Foundation/library/build/JamomaFoundation.dylib
LIBS += ../../../DSP/library/build/JamomaDSP.dylib

#########################################

OPTIMIZATION_DEBUG = -O0
OPTIMIZATION_RELEASE = -O3

OPTIONS = -shared -msse3 -mfpmath=sse -gdwarf-2

WARNINGS = -Wall -Wno-unknown-pragmas -Wno-trigraphs
DEFINES = -DTT_PLATFORM_MAC

#########################################

CFLAGS = $(OPTIONS) $(DEFINES) $(INCLUDES) $(WARNINGS)
LDFLAGS = $(LIBS)

#########################################

Debug:
	mkdir -p build
	mkdir -p /usr/local/jamoma/extensions
	touch /usr/local/jamoma/extensions
	$(CC_32) $(SRC) $(LDFLAGS) $(CFLAGS) $(OPTIMIZATION_DEBUG) -o build/$(NAME)-i386.ttdylib
	$(CC_64) $(SRC) $(LDFLAGS) $(CFLAGS) $(OPTIMIZATION_DEBUG) -o build/$(NAME)-x86_64.ttdylib
	lipo build/$(NAME)-i386.ttdylib build/$(NAME)-x86_64.ttdylib -create -output build/$(NAME).ttdylib
	cp build/$(NAME).ttdylib /usr/local/jamoma/extensions

Release:
	mkdir -p build
	mkdir -p /usr/local/jamoma/extensions
	$(CC_32) $(SRC) $(LDFLAGS) $(CFLAGS) $(OPTIMIZATION_RELEASE) -o build/$(NAME)-i386.ttdylib
	$(CC_64) $(SRC) $(LDFLAGS) $(CFLAGS) $(OPTIMIZATION_RELEASE) -o build/$(NAME)-x86_64.ttdylib
	lipo build/$(NAME)-i386.ttdylib build/$(NAME)-x86_64.ttdylib -create -output build/$(NAME).ttdylib
	cp build/$(NAME).ttdylib /usr/local/jamoma/extensions

clean:
	rm -f *.o
	rm -rf build

install:
	cp build/$(NAME).ttdylib /usr/local/jamoma/extensions

This is using the Apple Developer Tools-installed GCC version 4.2.  If the Ruby script had detected a newer version of GCC or the Intel (ICC) compiler it would have used that instead.  The compiler can be defined in the YAMLProj if a specific compiler is required or considered more desirable.

What about Editing and Debugging?

On Windows a VisualStudio project is created, so there’s nothing to worry about there.  Developers on the Mac are used to being spoiled with the Xcode debugger and it first glance it looks like we’re throwing out Xcode projects here to use Makefiles.

In fact, we don’t have to throw out Xcode projects — we just need to change them a bit.  Xcode projects can be configured to use as a source code editor that builds using the Makefile generated by our YAMLProj.  It can then debug the project, complete with breakpoints.

Here is a quick video showing how to do it.

Next Steps

The existing implementation is primarily in a single Ruby method called generate_makefile which you can read on Github. The problem with this implementation is that it is tied directly to Jamoma.  To be of general use it needs to be factored-out and made to standalone.  It needs a group of people peer-reviewing it.  It needs real-world use outside of Jamoma.

So who will do it?  YAMLProj awaits your arrival.

The scale~ object

Max 6 introduced some big changes when it was released last fall.  Flying under-the-radar were a number of new objects added to the collection.  A few of these objects came from Tap.Tools, including the join/unjoin objects and the XmlParse class for MXJ.  In the MSP world, the scale~ object entered the fray.

On the surface, scale~ is relatively straightforward.  It maps one range onto another.  With the addition of the exponent to provide non-linear functions the object becomes much more powerful, and little more difficult to quantify as a mathematical formula.  More difficult still is a backwards-compatibility mode called “classic” which implements an idiosyncratic exponential base function from back in the ancient days of the ISPW.  You probably don’t want to use this mode, but I’ve included it here for thoroughness sake.

So let’s take a technical look at scale~ beginning with the equation it implements.

scale~ equationsThis rendering was typeset in TextMate using the MacTex distribution of Latex.  Here is the Latex source:

\section{modern scale$\sim$} %(fold) \begin{equation} y = out_{low} +   \left( out_{high} - out_{low} \right)  \left( \frac{ x - in_{low} }{ in_{high} - in_{low} } \right) ^{exp} \text{,} \qquad \frac{ x - in_{low} }{ in_{high} - in_{low} } > 0 \end{equation} \begin{equation}
y = out_{low} + \left( out_{high} - out_{low} \right) \left( -1 \right)   \left( \frac{ -x + in_{low} }{ in_{high} - in_{low} } \right) ^{exp} \text{,} \qquad \frac{ x - in_{low} }{ in_{high} - in_{low} } < 0 \\ \end{equation} \section{classic scale$\sim$} %(fold) \begin{equation} y = out_{low} + \left( out_{high} - out_{low} \right)   \bigg[ (out_{high} - out_{low})  \exp(-(in_{high}-in_{low}) \log(exp))  \exp(x \log(exp)) \bigg]   \text{,} \qquad  out_{high} - out_{low} \geq 0 \end{equation} \begin{equation} y = -\bigg(out_{low} + \left( out_{high} - out_{low} \right)   \bigg[ (out_{high} - out_{low})  \exp(-(in_{high}-in_{low}) \log(exp))  \exp(x \log(exp)) \bigg]   \bigg) \text{,} \qquad  out_{high} - out_{low} < 0 \end{equation}

Finally, we have something functional to use: code for use in Octave or Matlab.

function [ y ] = scale_modern( x, in_low, in_high, out_low, out_high, exp )
  if (((x-in_low)/(in_high-in_low)) > 0)
    y = out_low + (out_high-out_low) * ((x-in_low)/(in_high-in_low))^exp;
  elseif (((x-in_low)/(in_high-in_low)) < 0)
    y = out_low + (out_high-out_low) * -((((-x+in_low)/(in_high-in_low)))^(exp));
  else
    y = out_low;
  end
end

function [ y ] = scale_classic( x, in_low, in_high, out_low, out_high, power )
  if (out_high-out_low >= 0)
    y = out_low + (out_high-out_low) * (  (out_high - out_low) * 
        exp(-1*(in_high-in_low)*log(power)) * exp(x*log(power)) );
  else
    y = (-1) * ( out_low + (out_high-out_low) * (  (out_high - out_low) * 
        exp(-1*(in_high-in_low)*log(power)) * exp(x*log(power)) ) );
  end
end

Scaling signals isn’t particularly sexy, but it’s certainly handy item to have in the tool belt.  One more of those little sneaky features in Max 6 that makes it an awesome upgrade.

Custom Data-Types in Max Part 4: Passing Object Pointers

How do you pass data between objects in Max?  If the data is a simple number or a symbol then the answer is easy.  What happens when you are trying to pass around audio vectors, dictionaries, images, or some other kind of object?  The implementation of Jamoma Multicore for Max deals with these issues head-on as it provides an illustration of how this problem can be tackled.

This is the fourth article in a series about working with custom data types in Max.  In the first two articles we laid the groundwork for the various methods by discussing how we wrap the data that we want to pass.  The third article demonstrated the use of Max’s symbol binding as means to pass custom data between objects.  This article will show an example of passing pointers directly between objects without using the symbol table.  In this series:

  1. Introduction
  2. Creating “nobox” classes
  3. Binding to symbols (e.g. table, buffer~, coll, etc.)
  4. Passing objects directly (e.g. Jamoma Audio Graph)
  5. Hash-based reference system (similar to Jitter)

A Peer Object System

Jamoma Audio Graph for Max is implemented as what might be called a Peer Object System.  What is meant is that for every object that a user creates and manipulates in a Max patcher, there is a matching object that exists in a parallel system. As detailed in Designing an Audio Graph, a Jamoma Audio Graph object has inlets and outlets and maintains connections to other objects to create a graph for processing audio through the objects.  The implementation of Jamoma Audio Graph for Max then has the task of creating and destroying these objects, sending them messages, and making the connections between them.  Once the objects are connected Jamoma Audio Graph will take care of itself.   The end result is that no audio processing actually happens in the Max objects for Jamoma Audio Graph — instead the Max objects are a thin façade that helps to set up the relationships between the objects as they exist in something akin to a parallel universe.

A Patcher

A Jamoma Multicore patcher in Max For context, let’s take a look at a Max patcher using Jamoma Audio Graph. In this patcher we have 4 Jamoma Audio Graph objects, identified by the ≈ symbol at the tail of the object name.  Each of these Max objects have a peer Audio Graph object internal to themselves.  Each Audio Graph object then contains a Jamoma DSP object that performs the actual signal processing. For example, the jcom.overdrive≈ object contains a pointer to a  Jamoma Audio Graph object that contains an instance of the Jamoma DSP overdrive class.  The attributes of the overdrive class, such as bypass, mute, and drive are then exposed as Max attributes so that they can be set in the patcher. Remember that each connection may carry N channels of audio.  The jcom.oscil≈ is, in this case, producing a stereo signal which is then propagated through the processing graph down to the jcom.dac≈ object.

Configuring the Graph

The exciting work doesn’t begin until the start message is sent to the jcom.dac≈ object.  As with all Jamoma Audio Graph externals, jcom.dac≈ Max external has a peer object.  In this case the peer object that it wraps is the multicore.output object.  This is the same multicore.output object that is shown in the Ruby examples in the Designing an Audio Graph article. When the start message is sent, the jcom.dac≈ object performs the following sequence:

  1. Send a multicore.reset message to all objects in the patcher.  This message sends a reset message to the peer objects underneath, which tells them to forget all of their previous connections.
  2. Send a multicore.setup message to all objects in the patcher.  This message tells the objects to try and connect to any object below it in the patcher.
  3. Tell the audio driver to start running.  When it is running it will periodically request blocks of samples from us, which in turn means that we will ask the other objects in the graph to process.

The processing happens completely within the Jamoma Multicore objects, thus not involving the Max objects at all.  It is the set up of the network of objects in the graph (steps 1 and 2) that involve our passing of custom data types in Max.

Diving into the code

For a full source listing of the jcom.dac≈ object, you can find the code in Jamoma Audio Graph source code repository.  We’ll abstract the important parts from that code below.  Let’s start with the method that is executed when the start message is sent:

TTErr DacStart(DacPtr self)
{
	MaxErr			err;
	ObjectPtr		patcher = NULL;
	long			vectorSize;
	long			result = 0;
	TTAudioGraphInitData	initData;

	self->multicoreObject->mUnitGenerator->getAttributeValue(TT("vectorSize"), vectorSize);

 	err = object_obex_lookup(self, gensym("#P"), &patcher);
 	object_method(patcher, gensym("iterate"), (method)DacIterateResetCallback, self, PI_DEEP, &result);
 	object_method(patcher, gensym("iterate"), (method)DacIterateSetupCallback, self, PI_DEEP, &result);

 	initData.vectorSize = vectorSize;
 	self->multicoreObject->init(initData);
 	return self->multicoreObject->mUnitGenerator->sendMessage(TT("start"));
}

As previously discussed, the last we thing we do is send a start message to our peer object, the multicore.output, so that the audio driver will start pulling audio vectors from us. Prior to that we iterate the Max patcher recursively (so the messages go to subpatchers too) to send the multicore.setup and multicore.reset messages.   To do this, we send the iterate message to the patcher and pass it a pointer to a method we define.  Those two methods are defined as follows.

void DacIterateResetCallback(DacPtr self, ObjectPtr obj)
{
	TTUInt32	vectorSize;
	method		multicoreResetMethod = zgetfn(obj, gensym("multicore.reset"));

	if (multicoreResetMethod) {
		self->multicoreObject->mUnitGenerator->getAttributeValue(TT("vectorSize"), vectorSize);
		multicoreResetMethod(obj, vectorSize);
	}
}

void DacIterateSetupCallback(DacPtr self, ObjectPtr obj)
{
	method multicoreSetupMethod = zgetfn(obj, gensym("multicore.setup"));

	if (multicoreSetupMethod)
		multicoreSetupMethod(obj);
}

These functions are called on every object in the patcher.  If we start with the last function, we can see that we first call zgetfn() on the object, obj, which is passed to us.  If that object possesses a multicore.setup method then we will receive a pointer to that method.  Otherwise we receive NULL.  If that method exists then we call it. The multicore.reset method works the same way.  The only difference is that the method takes an additional argument — the vector size at which the jcom.dac≈ is processing.

The Other End

At the other end of this calling sequence are the remaining objects in the patcher.  The full jcom.oscil≈ source code will show how this Max object is implemented.  In brief, we have two message bindings in the main function:

	class_addmethod(c, (method)OscilReset, "multicore.reset",	A_CANT, 0);
	class_addmethod(c, (method)OscilSetup, "multicore.setup",	A_CANT,	0);

These two methods respond to the those called by the jcom.dac≈ object.  They both have an A_CANT argument signature, which is how you define messages in Max that use function prototypes different than the standard  method prototypes.  These messages can’t be called directly by the user, and they are not listed in the object assistance, but we can send them from other parts of Max such as our jcom.dac≈ object.  The reset message (for forgetting about all previous connections) is simply passed on to the oscillator’s Multicore peer object:

TTErr OscilReset(OscilPtr self)
{
	return self->multicoreObject->reset();
}

The setup method, as we discussed, tells our object that we need to try and make a connection to any object below us in the patcher. To do this we wrap our peer Multicore object’s pointer up into a Max atom.  That, together with the outlet number (zero), are passed as arguments to the multicore.connect message which is sent out our outlet.

TTErr OscilSetup(OscilPtr self)
{
	Atom a[2];

	atom_setobj(a+0, ObjectPtr(self->multicoreObject));
	atom_setlong(a+1, 0);
	outlet_anything(self->multicoreOutlet, gensym("multicore.connect"), 2, a);
	return kTTErrNone;
}

One More Time…

That took care of the jcom.oscil≈ object.  Once it sends the multicore.connect message out its outlet, its work is done.  But what happens with that message when it is received?

In our example it is going to a jcom.overdrive≈ object.  The source code for jcom.overdrive≈ isn’t going to be very helpful though.  It uses a magic class-wrapper that wraps any Jamoma DSP object as a Multicore object using 1 line of code.  That’s really convenient for coding, but not for seeing how all of the parts communicate.  So for our discussion, we will look at the jcom.dcblocker≈ source code instead — beginning with the main() function.

	class_addmethod(c, (method)DCBlockerReset,	"multicore.reset",	A_CANT, 0);
	class_addmethod(c, (method)DCBlockerSetup,	"multicore.setup",	A_CANT, 0);
	class_addmethod(c, (method)DCBlockerConnect,	"multicore.connect",	A_OBJ, A_LONG, 0);

You should recognize the multicore.reset and multicore.setup messages.  Those are exactly the same as they were for our oscillator.  We now also have a multicore.connect message.  The oscillator was generating a signal but has no signal inputs, so it had no need for a multicore.connect message.  Any object that requires an input, however, will require this message binding.  How that method is implemented?

TTErr DCBlockerConnect(DCBlockerPtr self, TTMulticoreObjectPtr audioSourceObject, long sourceOutletNumber)
{
	return self->multicoreObject->connect(audioSourceObject, sourceOutletNumber);
}

We simply wrap a call to our peer object’s connect method, sending the audioSourceObject (which is the peer object that the jcom.oscil≈ object sent us), and the outlet number from which that object was sent.  If you compare this to the connect message from the Ruby example in Designing an Audio Graph, it may illuminate the process.

Some Final Details

The example code that we’ve seen from Jamoma Audio Graph demonstrates the passing of custom data (pointers to C++ objects) from one object to the next through the multicore.connect message. Because we are sending this custom data type, and not all inlets of all objects will understand this data type, it would be nice if we could protect users from hooking up the objects in a way that will not function.  For this task, Max makes it possible to give outlets type information.  When the type of an outlet is specified, a user will not be able to connect the patch cord to any inlet that doesn’t accept the specified message. To get this functionality, in DCBlockerNew(), we create our outlet like this:

	self->multicoreOutlet = outlet_new(self, "multicore.connect");

So instead of the customary NULL for the argument to outlet_new(), we specify that this outlet will be sending only multicore.connect messages.

Surfacing for Air

Jamoma Audio Graph provides a fairly intense example of passing custom data types in Max.  However, it presents not just the basics of how you would pass a pointer, but rather a context for why you might want to pass a custom type, and a real-world example to show what you can do.  I think that objective has been accomplished.

Designing an Audio Graph

In previous articles about the Jamoma Platform and the Jamoma DSP Library, there have been references to Jamoma Audio Graph (also previously known as Jamoma Multicore).  Up to this point, Jamoma Audio Graph has not been significantly documented or written about.  The authoritative information has been an Electrotap blog post showing the initial prototype in 2008.

At a workshop in Albi in 2009 we attempted to further expand Jamoma Audio Graph — and we failed.  The architecture was not able to handle N multichannel inputs and M multichannel outputs.  So we had to redesign a major portion of the inner-workings.  Get out your pipe wrench; it’s time to take a look at some plumbing…

What Is Jamoma Audio Graph ?

Let’s back up for moment to get the big picture.  The Jamoma Platform is essentially a layered architecture implementing various processes for interactive art, research, music, etc.  At the lowest level, the Jamoma Foundation delivers basic components for creating objects, passing values, storing values in lookup-tables, etc.  The Jamoma DSP library then extends the Foundation classes and provides a set of pre-built objects for audio signal processing.

Jamoma Audio Graph then gives us the ability to create Jamoma DSP objects and combine them into a graph.  In other words, we can connect the objects together like you might connect modules together on a Moog synthesizer.

A Moog Modular patch. Photo: Maschinenraum

A Moog Modular Patch. Photo: Maschinenraum

Unlike the Moog synthesizers of old, however, we can do a few new tricks.  Instead of sending a single channel of audio through a connection, we can send any number of channels through a connection.  While Jamoma Audio Graph does not currently implement any particular features for parellel processing on multiple cores/processors, the design of the system is ideal for such parallelization in the future.

The Audio Graph In Action

At the time of this writing, Jamoma Audio Graph has bridges to make it available in the Max and Ruby environments.  Most of the work is done on making it available to Pd as well (though if you are really interested in this then let us know so we can put you to work!).

In Ruby, you can code scripts that are executed in a sequence.  This provides a static interface to Jamoma Audio Graph even though all of the synthesis and processing is typically happening in real-time.  Alternatively, the irb environment allows you to type and execute commands interactively.  Jamoma Audio Graph, together with irb, then functions much like the ChucK environment for live coding performance.

Example

If you’ve been jonesin’ for an Atari/Amiga/Commodore fix then this might be your perfect example of Jamoma Audio Graph in Ruby:

# This is the standard require for the Jamoma Platform's Ruby bindings
require 'TTRuby'

# Create a couple of objects:
dac = TTAudio.new "multicore.output"
osc = TTAudio.new "wavetable"

# connect the oscillator to the dac
dac.connect_audio osc

# turn on the dac
dac.send "start"

# play a little tune...
osc.set "frequency", 220.0
sleep 1.0
osc.set "frequency", 440.0
sleep 1.0
osc.set "frequency", 330.0
sleep 0.5
osc.set "frequency", 220.0
sleep 2.0

# all done
dac.send "stop"

It’s a pretty cheesy example, but it should give you a quick taste.  If you want a flashback to kinds of music you could make with MS-DOS, be sure you set the oscillator to use a square waveform.

After creating a couple of objects, you connect two objects by passing the source object to the destination object using a connect message.  If you provide no further arguments, then the connection is made between the first outlet of the source object and first inlet of the destination object.  The inlets and outlets are numbered from zero, so the connect message in our example could also have been written as

dac.connect osc, 0, 0

The sleep commands are standard Ruby.  They tell Ruby to pause execution for the specified number of seconds.  Everything else is performed with the basic Jamoma Ruby bindings.  These provide the send method for sending messages and the set method for setting attribute values.

If you want to know the messages or attributes that an object possesses, you can use the messages? or attributes? methods.  This is particularly useful when coding on the fly in irb.  In the following example, I requested the list of attributes for the oscillator in the previous example:

>> osc.attributes?
=> ["gain", "mode", "size", "processInPlace", "maxNumChannels", "frequency", "mute", "interpolation", "sr", "bypass"]

How It Operates

If you create a visual data-flow diagram of the objects in a graph, like you would see in Max or PureData, then you would get a good sense of how audio starts at the top and works its way through various filters until it gets to the bottom.  The same is true for a Jamoma Audio Graph.  However, what is happening under the surface is exactly the opposite.

Pull I/O Model

Multicore Graph Flow

The flow of a Jamoma Audio Graph.

Jamoma Audio Graph is based on a “Pull” I/O Model.  Some other examples of audio graph solutions using a similar model include ChucK and Apple’s AUGraph.  In this model a destination, sink, or terminal node object sits at the bottom of any given graph — and this is the object driving the whole operation.  In Max, on the other hand, messages (e.g. a ‘bang’ from a metro) begins at the top of the graph and pushes down through the objects in the chain.

The image to the left visualizes the operation of the audio graph.  Let’s assume the the destination object is an interface to your computer’s DAC.  The DAC will request blocks of samples (vectors) every so often as it needs them.  To keep it simple, we’ll say that we are processing samples at a sample rate of 44.1KHz and a block size of 512 samples.  In this case, every 11 milliseconds the DAC will tell our destination object that it needs a block of samples and the process begins.

The process flows through the light blue lines.  The destination asks the limiter for a block of samples, which then asks the overdrive for a block of samples which then asks both the source and the multitap delay for samples, and then the multitap delays asks the source for a block of samples.  To summarize it: each object receives a request for a block of samples, and in response it needs to produce that block of sample values, possibly pulling blocks of samples from additional objects in the process.

One Object At A Time

To understand in finer detail what happens in each object, the graphic below zooms-in to view a single instance in the graphic above.  Here we can see that we have the actual unit generator, which is a Jamoma DSP object, and then a host of other objects that work to make the interface for the audio graph.

Anatomy of a Multicore Object

Jamoma Audio Graph class structure

The text in graphic explains each of the classes contained in a Jamoma Audio Graph object.  Implied in both of the figures, is the ability to handle “fanning” connections where many inlets are connected to an outlet, or an inlet is connected to many outlets.

In essence, the outlets are only buffers storing samples produced by the unit generator.  Each time a block is processed the unit generator is invoked only once.  Subsequent requests for the object’s samples then simply access the samples already stored in the outlet buffers.

As explained in the graphic, the inlets have more work to do, as they need to sum the signals that are connected.  And remember, each connection can have zero or more channels!

Benefits

The most obvious  benefit is the ability to easily handle multiple channels in a single connection.  So imagine that you create a Max patcher for mono operation.  It can then function in stereo or 8-channel or 32-channel without a single modification.

But there’s a lot more than that here.  The number of channels is dynamic and can change at any time.  One place this is valuable is in ambisonic encoding and decoding where the order of the encoding can dramatically alter the number of channels required for the encoded signal.  If you want to try changing the ambisonic order on-the-fly, which changes the number of channels passed, you can.

Likewise, the vectorsize can be altered dynamically on a per-signal basis.  The benefit here may not be immediately obvious, but for granular synthesis, spectral work, and analysis based on the wave length of an audio signal (e.g. the kinds of things in IRCAM’s Gabor) this can be a huge win.

Writing the objects is also very simple.  If you write a Jamoma DSP object, then all you have to do to make it available in Jamoma Audio Graph is…

Nothing!

That’s right.  In Ruby, for example, all Jamoma DSP classes are made available with no extra work.  If you want to make a Max external for a particular object then you can use a class wrapper (1 line of code) to create the Max external.

Interested in join the fun?  Come find us!

Writing DSP Objects

In my last article I talked about the structure of the Jamoma Platform.  That’s a bit abstract to be of much direct use.  A primer on how to write a DSP object seems to be in order.

So… let’s imagine we want to write a simple unit generator for audio processing.  One of the simplest filters we can write is a one-pole lowpass filter.  In pseudo code, it might look like this:

static float previous_output = 0.0;
static float feedback_coefficient = 0.5; // default is half way between 0 Hz and Nyquist

float processOneSample(float input)
{
    float output = (previous_output*feedback_coefficient) + ((1.0-feedback_coefficient)*input);
    previous_output = output;
    return output;
}

Simple, right?  Like most simple questions, the answer is only simple until you start asking more questions…  Let’s brainstorm a few practical questions about this simple filter:

  • How do we set the coefficient?
  • How do we efficiently process in blocks of samples instead of one sample at a time?
  • how do we handle multiple channels?
  • what if the number of channels changes on the fly?
  • is the audio processed in a different thread than the object is created and deleted on?  how do we handle thread safety?
  • do we want to have a “bypass” built-in so we can audition the effect?
  • How do we wrap this for a host environment like Max/MSP?
  • How do we wrap this as an AudioUnit plug-in?
  • What if we want to swap this unit generator out for another in real-time, without having to recompile any code?
  • How do we handle denormals and other similar gremlins that can cause performance headaches in realtime DSP code?

One more question: how do you get all of this without it sucking the life and love out of making cool DSP code? Funny you should ask, because that’s the very reason for the Jamoma DSP framework. Let’s look at an how we would write this object using Jamoma DSP.

Example Class: TTLowpassOnepole

First, the header file: TTLowpassOnepole.h:

#include "TTDSP.h"

class TTLowpassOnePole : TTAudioObject {
	TTCLASS_SETUP(TTLowpassOnePole)
	TTFloat64		mFrequency;	///< filter cutoff frequency
	TTFloat64		mCoefficient;	///< filter coefficients
	TTSampleVector		mFeedback;	///< previous output sample for each channel

	TTErr updateMaxNumChannels(const TTValue& oldMaxNumChannels);
	TTErr updateSr();
	TTErr clear();
	TTErr setfrequency(const TTValue& value);
	inline TTErr calculateValue(const TTFloat64& x, TTFloat64& y, TTPtrSizedInt channel);
	TTErr processAudio(TTAudioSignalArrayPtr inputs, TTAudioSignalArrayPtr outputs);
};

The TTDSP.h header includes everything needed to create a subclass of TTAudioObject. We will see some of the magical joy of TTAudioObject shortly. In the class definition there is a macro called TTCLASS_SETUP. This creates prototypes for the constructor, destructor, and glue code for class registration, etc.

This class implementation thus follows as:

#include "TTLowpassOnePole.h"

#define thisTTClass		TTLowpassOnePole
#define thisTTClassName		"lowpass.1"
#define thisTTClassTags		"audio, processor, filter, lowpass"

TT_AUDIO_CONSTRUCTOR
{
	addAttributeWithSetter(Frequency,	kTypeFloat64);
	addAttributeProperty(Frequency, range, TTValue(2.0, sr*0.475));
	addAttributeProperty(Frequency, rangeChecking, TT("clip"));

	addMessageWithArgument(updateMaxNumChannels);
	addMessage(updateSr);
	addMessage(clear);

	// Set Defaults...
	setAttributeValue(TT("maxNumChannels"), arguments); // This attribute is inherited
	setAttributeValue(TT("frequency"), 1000.0);
	setProcessMethod(processAudio);
}

TTLowpassOnePole::~TTLowpassOnePole()
{
	; // Nothing special to do for this class
}

TTErr TTLowpassOnePole::updateMaxNumChannels(const TTValue& oldMaxNumChannels)
{
	mFeedback.resize(maxNumChannels);
	clear();
	return kTTErrNone;
}

TTErr TTLowpassOnePole::updateSr()
{
	TTValue	v(mFrequency);
	return setFrequency(v);
}

TTErr TTLowpassOnePole::clear()
{
	mFeedback.assign(maxNumChannels, 0.0);
	return kTTErrNone;
}

TTErr TTLowpassOnePole::setFrequency(const TTValue& newValue)
{
	TTFloat64	radians;

	mFrequency = newValue;
	radians = hertzToRadians(mFrequency);
	mCoefficient = TTClip(radians / kTTPi, 0.0, 1.0);
	return kTTErrNone;
}

inline TTErr TTLowpassOnePole::calculateValue(const TTFloat64& x, TTFloat64& y, TTPtrSizedInt channel)
{
	y = mFeedback[channel] = TTAntiDenormal((x * mCoefficient) + (mFeedback[channel] * (1.0 - mCoefficient)));
	return kTTErrNone;
}

TTErr TTLowpassOnePole::processAudio(TTAudioSignalArrayPtr inputs, TTAudioSignalArrayPtr outputs)
{
	TT_WRAP_CALCULATE_METHOD(calculateValue);
}

Breaking it Down

To understand what’s happening here, let’s start at the bottom and work our way back up toward the top.

processAudio

This method accepts an input and an output.  The input and output arguments are arrays of multichannel audio signals.  That is to say the each of the input and output can contain zero or more multichannel signals, and each of those signals may have zero or more channels.  The audio signal has a vector size which indicates how many samples are contained for each channel that is present.

In most cases an object is only functioning on one multichannel input signal and one multichannel output signal.  Also, in most cases, the number of input channels and output channels are the same (e.g. 2 inputs and 2 outputs).  Furthermore, it is quite common that each channel is processed in parallel, and can be considered independent of the other channels.

Given this set of somewhat common set of assumptions, we can avoid the work of handling all of this audio processing machinery and just call the TT_WRAP_CALCULATE_METHOD macro.  Calling that macro will invoke the named calculation method to be used for processing one sample on one channel of one signal.  The calculate method is inlined, so we do not give up the performance benefits of processing by vector.

calculateValue

As we had previously alluded, this method calculates one output value for one input value.  You can think of this method in the mathematical terms

y = f(x)

This method may be called directly or, as just discussed, called to crunch numbers for the vector-based audio processing method.

setFrequency

As we will see shortly, attributes can be set using a default setter method that works most of the time.  In this case we need to do some customized work when the “Frequency” attribute is set.  Namely, we need to calculate the feedback coefficient.  We want to do that here so that the coefficient isn’t calculated every time our audio processing method is called.

This is the first time we’ve seen the TTValue data type, but we’ll be seeing a lot more of it.  This is the standard way of passing values.  TTValue can contain zero or more of any common data type (ints, floats, pointers) or special types defined in the Jamoma Foundation (symbols, objects, strings, etc.).

clear

This method is quite simple: it resets all of the feedback samples for each audio channel to zero.  It can be invoked by a user if the filter ‘blows-up’.

updateSr

This method is slightly special.  Just as we have a “Frequency” attribute, we have an “sr” attribute, which is the sample-rate of the object.  The trick is that we inherit the “sr” attribute from TTAudioObject.

Some objects may ignore the sample rate, or will function properly when the sample rate changes by virtue of the face that the member variable changed values.  In our case we need to take further action because our coefficient needs to be re-calculated.  The “updateSr” method is a notification that we will receive from our base class when the base class’ “sr” attribute is modified.

updateMaxNumChannels

Just like the updateSr() method, this method is a notification sent to us by our base class.  In this case, the notification is sent when the base class has a change in its “maxNumChannels” attribute.

The “maxNumChannels” attribute is an indicator of the maximum number of channels the object should be prepared to process in the audio processing method.  As such, we use this notification to take care of memory allocation for anything in our instance that is related to the number of channels we process.

The Destructor

As the comment says, we don’t have anything special to take care of in this case.  We still define the destructor so that we can be explicit about what is happening regarding object life-cycle.

The Constructor

Obviously, to experienced C++ programmers anyway, the constructor is what gets called when a new instance of our class is created.  But what we do in this constructor is what makes everything else we’ve been through actually work.

First, we use a macro to define the function prototype.  We do this because it is the same every single time, and this ensures that we don’t screw up the initialization (or lack of initialization) of members or super-classes.

Next, we define attributes.  In our case we have only one attribute, and that attribute has a custom setter method (the setFrequency() method).  It is represented by the mFrequency member variable.  Attributes can be given properties.  In this case we limit the range of the values for our attribute to things that will actually work.

In addition to attributes, which have a state and are represented by data members, we have messages.  These are stateless methods that will be invoked any time our object receives the said message.  Messages might have no arguments, as in the case of the “sr” and “clear” messages.  If they do have arguments the arguments will passed as a TTValue reference, as in the case of the “updateMaxNumChannels” method.

Finally we set defaults.  This means default attribute values, but it also means the initial audio processing and value calculation methods.  These methods may be changed on the fly during operation, though in our case we only have one of each.

Gift Wrap

To summarize, we now have an object with the following features from our original list:

  • We set the coefficient using an attribute for cutoff frequency, which is automatically updated when the sample rate changes.
  • We efficiently process in blocks of samples (instead of one sample at a time) using the processAudio method.
  • processAudio also handles N channels of input and output transparently.
  • It is no problem if the number of channels changes on the fly, this is all handled properly.
  • The audio may be processed in a different thread than the one on which the object is created and deleted.  Thread safety for this circumstance is handled by the environment.
  • We did not discuss it, but we do have a “bypass” attribute that we inherited, among others, so we got this functionality for free.
  • We can swap any object inheriting from TTAudioObject for another in real-time.  The attributes and messages are called by dynamically bound symbols, so there are no linking problems or related concerns.
  • We did not discuss it but the calculateValue method handles denormals using a library function.

So now we just need to use the object.  TTAudioObject classes have been used directly and in combinations with each other to create Max/MSP objects, Pd objects, VST plug-ins, AudioUnit plug-ins, etc.  Some examples of these can be found in the Jamoma DSP Github repository.  Others include the Tap.Tools, sold by Electrotap.

The Magic Wand

One of the benefits of our dynamically-bound, message-passing TTAudioObjects is that we can use introspection on objects to find out about them at runtime.  That means we can load an object by name, ask what attributes it has and what types they are, and then create a new interface or adapter to the object.  One manifestation of this a class wrapper for Cycling ’74′s Max environment.

Given our TTAudioObject that implements a onepole lowpass filter, all that is required to make a full-blown Max/MSP object complete with Max attributes is this:

#include "TTClassWrapperMax.h"

int TTCLASSWRAPPERMAX_EXPORT main(void)
{
	TTDSPInit();
	return wrapTTClassAsMaxClass(TT("lowpass.1"), "jcom.onepole~", NULL);
}

The first symbol we pass is the symbol name of the TTAudioObject.  The second argument is the name of the Max class we generate.  It really is this easy.

At the time of this writing, no one that I’m aware of has written a similar class wrapper for PureData, SuperCollider, AudioUnits, etc.  but there is no reason that this kind of wrapper couldn’t work for any of those target environments.

It’s fun stuff!  As the Jamoma Foundation and DSP projects have evolved over the last six years the code for classes has become increasingly flexible and also increasingly clear.  It’s possible to really focus on the task in the code without having to worry about all of the glue and filler typically involved in writing audio code with C and C++ APIs.

Refactoring Jamoma

Over the last several months I haven’t been writing much in the way of prose.  Instead I have been working on the Jamoma 0.5 release, a consulting project built on top of the Jamoma DSP layer, and developing a graphics library to support a new version of Hipno.  Now, at the end of these projects we are doing much more than just dsp with the Jamoma “DSP” layer – in fact Jamoma underwent a major refactoring and restructuring.

The Jamoma Platform Revisited

Jamoma Layered TopologyIn my previous post about The Jamoma Platform, we introduced a graphic that looked like this first graphic.  The DSP Layer had a lot going on.  It was a runtime for a dynamically-bound object-oriented framework.  It was starting to grow graphics capabilities.  And Ruby language bindings.  None of this stuff has anything to do with audio or digital signal processing.

So it was time to break apart the monolithic beast to make the structure clearer, more focused, more flexible and to increase orthogonality and decoupling.

A New Pile

Jamoma's layered code architectureThe layers are now piled up on top of the Jamoma Foundation.  The Foundation is analogous to Cocoa’s Foundation classes.  It defines and implements the basic runtime architecture on which everything else relies.  It includes the base classes, message passing system, notifications, OS-specific glue, object instantiation, etc.

The DSP Layer, now, is actually focused on audio and signal processing.  It extends the Foundation’s classes to make it easy to create dynamically bound audio objects.  As before, there are example projects in Jamoma that compile into PureData externals and VST/AudioUnit plug-ins, but I’ve now taken the care to add them to the graphic.

The Multicore Audio Graph Layer is largely unchanged, as it was already separated from the DSP Layer.  With the new extensions, such as the Ruby language bindings, it becomes possible to create a graph of audio objects in real time using Ruby’s irb somewhat in the fashion that people use Ge Wang’s excellent Chuck environment.

Meanwhile, the new Graphics Engine creates both a platform-independent, and a host-independent, way to draw to the screen.  This means that GUI widgets can be created that will work both in Max/MSP and in other environments, such as AudioUnit plug-ins.  Internally the drawing is performed using the Cairo graphics library.  We extend the library to deal with other OS-related issues like fonts, mouse interaction, creating windows, etc.  We also make the Cairo API available through the Foundation’s message-passing system in addition to the old-fashioned static linking.  That means that we can perform drawing dynamically from environments such as Ruby.

Making it Happen

As an open source project, it is quite impressive how much a smallish group has been able to accomplish with limited resources and man-power.  While we may have missed on a few things, we have definitely gotten a lot of “bang for the buck”.  I mention this because we could use some help.  There are lots of areas where contributions are welcome.  We have a particular need for someone who is handy with C++ and oriented towards the Windows platform.  If you are interested, please make yourself known to us.  We don’t bite :-) .

Information can be found at the Jamoma website and over at our Github repository.

Less is Less

This month’s issue of Inc. Magazine features a profile of Jason Fried, founder of 37Signals. The part that caught my attention was the open:

You could sum up Jason Fried’s philosophy as “less is more.” Except that he hates that expression, because, he says, it still “implies that more is better.”

More clearly isn’t better. I wrote about a small bit about the ideas of Sarah Susanka a few months ago. Carried to an extreme, the idea of smaller houses results in the work of Jay Shafer, like in this video (via the 37Signals blog):

A happy coincidence occured, where I saw the above video during the same week that that I saw the video that follows: an etude for piano and electronics by fellow Jamoma developer Alexander Refsum Jensenius. As Alexander describes it:

Many performances of live electronics is based on large amounts of electronic equipment, cables, sound cards, large PA-speakers, etc. One problem with this is that the visual appearance of the setup looks chaotic. Another is that the potential for things that can go wrong seems to increase exponentially with the amount of equipment being used. The largest problem, though, at least based on my own experience of performing with live electronics, is that much effort is spent on making sure that everything is working properly at the same time. This leaves less mental capacity to focus on the performance itself, and sonic output.

I am currently exploring simplicity in performance, i.e. simplicity in both setup and musical scope.

I can attest to the problems Alexander relates, and I think the musical results he achieves are incredibly beautiful – in part because using less helps to focus the musical expression and make it more concise.

Making things simple, concise, and expressive, is incredibly difficult to do: whether it be music, prose, code, business, architecture, or hardware. It’s great to see examples of people finding the sweet-spot.

Introducing Max

So here I was, faced with the challenge of introducing Max to an insanely diverse group of painters, sculptors, ceramics majors, and others – some of whom used Max and some of whom had never even heard of it.  Last week I gave a pair of guest presentations at the Kansas City Art Institute (KCAI) to students of a class in the Interdisciplinary Arts department.  KCAI is currently teaching two sections of a course that introduces students of all disciplines to electronics, sensors, basic robotics, microcontrollers, debugging systems, etc.  (As a sidenote, I was honored to teach the inaugural semester of the course a few years ago with Leigh Rosser, and now the course is taught by Colin Leipelt).

A ring from the original Expo '74. Photo: Ian Broyles.

A ring from the original Expo '74. Photo: Ian Broyles.

The previous week I was in attendance during a panel session about introducing and teaching Max at Expo ’74 in San Francisco.  There are all sorts of contexts in which Max is taught: art schools, music conservatories, science laboratories, and even architecture departments.  Some have students with an assumed level of competence or exposure to related topics; others don’t.  There was a lot of discussion about how people deal with the challenges of teaching such diverse people about a tool that may be used in so many radically different ways.

There were also a lot of people trying to squeeze in their thoughts during the session, so I simply observed and took notes.  I do have some thoughts about introducing and teaching Max though.  And then we can answer the question I posed at the open: How in the world do you introduce Max to a diverse audience with all different quantities and qualities of experience and with whom you have no prior relationship or knowledge?

Driving a Car

When I first began studying computer music with Ichiro Fujinaga (then in Baltimore) he liked to use the example of driving a car.  You don’t have know very much about how the car works in order to drive around town.  But if you want to be a really good driver, to maximize fuel use or to race other drivers for example, then you need to understand engines and transmissions and gear ratios.  The point of the analogy is that if we want to really be in control of our computer music systems then we need to understand the concepts and details of what happens under the hood.

I agree with this analogy in the context of my work with Ich.  I think, however, people take this same approach to teaching Max.  And that is often a mistake.

Many (I would argue the vast majority of) users are encountering an entirely foreign construct.  It is completely overwhelming.  In this scenario, the key to teaching Max is the same as when you first learn to drive.  That is to say you don’t sweat details.  In the U.S. you probably don’t even start learning to drive on a manual transmission, so you don’t start learning to press the gas while popping the clutch, let alone memorizing the gear ratios and figuring out how horsepower and torque are related.  If someone tries to think about all these details and parts when trying to get used to the basic act of driving, they are going to wreck the car and may never drive again due to the trauma.

So my first key to teaching Max is this: It’s okay hide information or gloss over details – in fact, it’s essential to do so. To use a bicycling analogy, you can take the training wheels off later.

One of the ways I do this in a practical way is by presenting a number of things next to each other and indicate that they are all essentially functioning on the same principles – even if what happens underneath the surface is different.  This has been extremely effective.  I had actually forgotten that I formalized a bit of this until I had a conversation with Richard Boulanger about it at Expo ’74 when he was commenting on some of my older materials from teaching Max which were online.

Another way I do it is by not answering questions asked by students.  Rather than overwhelming a whole class, I simply say that we will cover it later, or it isn’t important right now, or to ask me after class is over.  I don’t always do this, but sometimes the risk of explaining a complex subject or an exception to the rules that you are in the middle of trying to re-inforce, outweighs the benefits.  You can always come back to it later.

From the Expo Floor

Mousetrap!  Photo: Valerie Reneé

Mousetrap! Photo: Valerie Reneé

There were a couple of ideas during the Expo ’74 panel session that I thought were interesting.  The first was from Gregory Taylor, who mentioned using the Mousetrap game to show the sequence of causality.

For teaching a full course, as opposed to a one-off lecture, there was an idea suggested (sadly, I don’t remember who it was) to give assignments to create a patcher that performs a 10 second piece that visually says hello on the monitor.  The next assignment is to hook up a game controller with the hi object, and use that to control the patch.  ”Now every student has an instrument and an installation”.

A notion expressed under several guises is that a good teacher/course will teach concepts rather than tools.  For example, teach basic problem solving  or debugging using the scientific method.  I completely agree, but I like to carry it a step further.  When a student has a problem we solve it together as a class — on the whiteboard with dry-erase markers.  The white board is a great:

  • it gets you away from the potential clumsiness of the mouse, keyboard, etc. and worrying about typing the correct thing.
  • it gets you away from the monitor and the light beaming into (and frying) your brain (in an emotional sense more than physical)
  • it  is comfortable for the whole class to see and contribute
  • there is no penalty for meta-patching (that is to say, creating a flow of objects that don’t really exists but represent some kind of concept)

The first couple of points may seem trivial, but they end up being important.  Students get nervous showing their problem in front of the whole class.  There is also a tendency when sitting in front the keyboard/mouse/monitor to do.  But solving problems is more about thinking before you do, and it is hard to slow down and do that when the machinery is under your finger-tips.  Solving problems requires a return from emotional patching to rational, reasoned, reflection.  So for me, the whiteboard is king.

Finally, something I never heard during the panel session, and which surprises me considerably, is that no one mentioned including a history/literature component to their courses teaching Max.  When I’ve taught full-semester courses on Max, I’ve generally split the time evenly between lecturing, tutoring/answering questions, and a history/literature section.  The History and Lit is absolutely crucial to giving students ideas on how to put “the meaningless numbers” to work.

Early in the course I use Risset’s Duets for One Pianist with scores and Max patches to show how, in some cases, simple relationships can yield extremely complex counterpoint.  Every week we have time allocated not only to see/hear/experience literature (music, installation footage, video works, etc) created using Max, but we have time to discuss the work and its artistic merits.  This grounds the course in something tangible, and starts the process of examining how one’s own aesthetics are challenged or expanded by this technology.  Part of understanding what you can do with Max is not just understanding the tool, but understanding the paradigms and contexts in which it is used. My experience is that without this, most artists and musicians will be lost and probably never return to Max after the semester is completed.

System Topography

Trying to understand Max is really a subset of another, more general, problem: understanding interactive systems (or any system for that matter).  I first saw this strategy put into action by Butch Rovan a number of years ago, during a visit to the University of North Texas (where he was then located). This is the approach I have found most effective when I’m confronted with an audience amongst whom I can assume little commonality of experience.  My sessions at KCAI last week were such a situation.

It goes like this: There 3 primary layers to any interactive system.

  1. Data Source Layer (analysis results, sensor data, generative algorithms, etc.)
  2. Mapping Layer
    1. one-to-one relationships
    2. one-to-many relationships
    3. many-to-many relationships
  3. Synthesis Layer (synthesis algorithms, effects processors, spatialization modules, etc.)

I begin by presenting this and offering a little overview of each layer.  The middle layer is really hard, and it’s where most artists working with machines spend their lives.  The outer layers are a bit more objective, and so we begin by looking at those.

The Data Source Layer

navierstokesFor last week’s foray into Max,  I began with a Max patcher that was shared by Gregory Taylor at last week’s Expo ’74.  In fact, I had it running before anyone even arrived and let it serve as my backdrop for the intro.  It was my hook.  Not only is the patcher really beautiful to watch as the motion ebbs and flows, but it is generating really useful data and demonstrates all of the basic principles of a Max patcher.

After showing Gregory’s patcher, and using it to explain the basics of how Max objects are connected (and how they do their work), I then shifted to talking about sensor inputs.  The class is using the Arduino and is actively working with sensors, so this where they could really start making connections between what they saw Max doing and the projects on which they are already working.

The Art Institute happens to also own a Teabox and bunch of sensors from Electrotap (used in their Max course taught by Dwight Frizzell).  So I explained some of the pros and cons of a Teabox vs. the Arduino.  They can do some of the same things, but ultimately the Teabox is easier to use and the Arduino is more flexible – take your pick.  It also gave me a chance to relay my various horror stories of problems in installations caused by relying on devices connected via USB, and thus the decision to have the Teabox connect using a more robust transport mechanism.

The Synthesis Layer

Time was very limited (I had only 1 hour), and so I didn’t discuss algorithms or technical details for synthesizing audio or video.  Instead, I used a couple of modules from Jamoma.  This allowed me to drop in pre-built black-boxes that did something easily grasped without any work at all.  I explained that Jamoma modules are really just Max patches, and that you drop them in connect them together.  In 30 seconds I was able to build a patcher that accepted a wide range of video sources (including files), a cheesy embossing effect, and a video output.

The Mapping Layer

kcai-patcherThen we come back and talk about mapping.  It’s pretty superficial in this context, but I show how mappings can be set up using Jamoma in Max.  First we create a mouse input module to represent our input layer, then we create one or more “continuous mapper” modules to do the mappings, and then start changing those mappings on the fly.  It’s really that easy, and everyone was able to grasp what I was talking about immediately when they saw the mouse position controlling the playback rate of a movie and the embossing of that movie.  And it was another bit of patching that only took about 30 seconds.

Building on the Work of Others

I like to end by showing some resources and telling them its okay to build on the work of others.  In fact, my opinion is that it is foolish to not build upon the work of others.

One interesting thing I’ve noticed a lot in the last few weeks is how shocked people are that Jamoma is free.  I can say ‘open source’ but it never registers.  I can get to the end of showing Jamoma (at KCAI or Expo ’74, for example) and say you can go download it for free and people are shocked and amazed.  It’s nice feeling like something is so obviously useful and well received that people would expect to pay for it.  The whole Jamoma team has something of which to be proud!

It can be difficult to gauge the success of presenting Max, especially in single-session scenarios. The nice thing about longer courses (such as a full semester) is that you can take the time to really establish some confidence in students that they are able to make something work.  I didn’t have that luxury.   An ending question/answer session, or conversations after-the-fact, can give some sense of whether the message was received or if people have been opened up to some new possibilities.

I think presenting Max this way, as I learned from Butch Rovan, really does do this for people regardless of their presuppositions and experience.  I was grateful for the opportunity to present and dialog with group at the Kansas City Art Institute last week, and give them my many thanks.

Poème Symphonique

Metronome 3.  Photo: Nigel Appleton.

Metronome 3. Photo: Nigel Appleton.

Last night I attended a concert of György Ligeti’s music hosted by newEar in Kansas City.  It was spectacular.  I don’t think the review in the Kansas City Star really did it justice.

Where the review was spot-on is in saying the the most conceptually interesting piece was the Poème Symphonique for 100 metronomes.  During the intermission all of the metronomes were wound and then released as the audience came back into the performance space.

The metronomes looked dapper for the performance – they were rented and all were of the same make and model.  This may seem trivial, but it really did add visually to the performance, and I think sonically as well.

I doubt this work could be effective in a recording.  For one thing, the spatial information (and how the metronomes interact with the space) provides a rich amount of information when hearing the piece live.  There is also a lot of visual information.  When looking at a particular group of metronomes, it became possible to really focus and hear what was happening within that group of metronomes as a foreground element, while the rest of the ‘metronome orchestra’ laid the backdrop.  In fact, the Cello Concerto functioned in much the same way — a piece that never felt compelling to me from recordings and was arresting to see and hear performed live because some much of the information in the performance is transmitted visually.

I was also pleasant surprised by the spectral diversity of the performance of Poème Symphonique. The clicks from the metronomes in the space produces a lot of phasing, and thus difference and summation tones were audible.

Metronome.  Photo: abbyladybug.

Metronome. Photo: abbyladybug.

The form of the piece was a bit like the form of a big rain storm.  Slowly it winds down as the metronomes slow and stop and various beating and phasing of the metronome beats maintain an organic unity/variety.  Eventually, down to just a few metronomes, really interesting rhythmic counterpoint emerges — again, much like dripping water in metal gutters after a big rain storm.  The rhythms here also strongly mirrored the polyrhythmic “Fanfares” from the Études pour Piano that closed the first half (and whose performance by Robert Pherigo was also mesmerizing).  The review in the Star complained about the one rogue metronome that kept playing for 7 minutes after the others had all wound down.  In fact, I thought it was quite an interesting way to end: that one dripping eave or gutter that just keeps going.

The ticking of the last metronome also transported me back to being a kid being kept away by a very large clock at night when we visited my grandparents.  So the passing of time, being performed by a device for marking time, was serving as an idée fixe of sorts for a variety of imagery brought to the performance by the individual audience members, and also provided a built-in moment in the piece for reflection.  Perhaps the author of the review in the Star didn’t have much to reflect upon.

Speaking with David McIntire afterwards, he relayed that there was a metronome in Thursday night’s rehearsal that went on for a long time at the end, so they specifically didn’t wind that one fully.  But in last night’s performance a different metronome, Metronome #9, was the rogue metronome instead.  I guess metronomes can have personalities too.

From the Expo

expo74badgeLast week I was a participant in Expo ’74, Cycling ’74‘s first Max user conference.  It was 120 Max-heads totally stoked-up about each other’s work and the work we’ve been doing at Cycling ’74 over the last while.  Unlike the academic conferences or trade shows to which I’m more accustomed, the attendees at Expo ’74 were exuding profound amounts of happiness.  It is really rare to experience the amount of joy and community that I saw this past week.

There are a number of topics from the Expo that I’d like to discuss in more length than I will do right now.  For the moment, I’ll give an overview and some highlights.

First, though, I will start with an admission.  I left for San Francisco with some doubts about the premise of Expo ’74.  The idea that a conference would be organized around a tool, rather than organized around a problem or research area, seemed problematic.  It ended up being quite the opposite.  I found that by all of the participants sharing a common platform that people were able to discuss the actual essence of their research, development, artistic practice, etc. without the distraction of having to preface everything with information about the technology.  Other participants understood the technology and tools, so they could fade away and allow for a clearer focus on the real artistic issues.

Day One

Lots of things were happening on day one.  The morning included some well done presentations by my Cycling ’74 colleagues Manuel Poletti and Darwin Grosse.  They did a nice job with Max 5 and Max for Live.  As someone already quite well acquainted their work, I was more interested by the afternoon where the presentations were about work with which I was less familiar.

cablecarPamela Z gave a history of her work with Max and showed the real practical aspects of how her performance system (not just Max) is structured.  This was capped off with a performance of Broom, which was both effective and inspiring too.  Robert Henke (of Ableton fame), showed a number of his Max patchers, many of them the search for the perfect step sequencer.  He also showed some Max patches that he uses for image processing to produce album covers (I will talk about this topic more on another day).  Finally, Barney Haynes showed us some documentation of his work.  There wasn’t any significant discussion of the role that Max played — and this is one of the things that made the conference really nice.  Because Max was assumed, it didn’t have to be discussed and there was no pressure to discuss it (or to hide it for that matter).

In the afternoon we split into groups named after Max objects.  My group was the Buddy group (as signified on the scan of badge).  In this first of two meetings we were to “collect data” from an excursion in the city.  Our group’s excursion consisted of a cable car ride and a visit to the San Francisco Cable Car Museum.  The data we collected was pretty impressive.  We had 4 video camera with time information embedded, we had a couple of people with professional portable audio recording gear, we had GPS and heart-rate-monitor data logged for the entire trip, and Joshua Goldberg (a fellow ‘buddy’) had his iPhone transmitting accelerometer data to his computer which logged the time-tagged accelerometer data into a coll object in Max (and yes, he carried his computer around open and running through for the whole experience).

Day Two

Days two and three split the morning into two tracks, so it is impossible to give an overview of everything that happened.  Ipresenation-stage happened to see presentations that included Gregory Taylor (discussing how he generates control data and, more importantly, why he does it that way), Andrew Pask (discussing time management in Max), and Andrew Benson giving an incredible introduction to writing OpenGL Shaders.  Gregory’s presentation has inspired me to start on some new work that includes some new chaotic generators for Jamoma and Andrew Benson’s presentation has got me writing shaders now, making this the session with the most immediate impact on what I’m doing.

In the afternoon there were presentations including Brad Garton showing the rtcmix object and a panel session on education.  I’ll talk more about the education panel in another post.  Brad’s talk was interesting, as I have long been a proponent of mixing graphical and text-based approaches in Max to leverage the strengths of each.  I feel pretty strongly that advocating graphical or text interfaces (all the way back to my work on Jade in 2001) for every purpose really puts you at a disadvantage because sometimes you end up with an inferior tool for a given task.

Day Three

group-projectI bounced between rooms in the morning to cover some advanced Max external developer topics and to see Luke Dubois present his work with Max.  Luke’s presentation was really well done and one of the highlights of the entire Expo for me.  I was familiar with a number of things Luke has done, and I’ve admired them before, but seeing a larger body of work together really brought it together for me in a new way.  It’s really impressive.  I was also nothing sort of stunned when at the end he gave away all of his Max patches.

We had an open mic session to solicit feature requests.  I had reservations about this, but no one threw any fruit at us and it seemed like we had an answer for about half of the feature requests that went like “good idea; it’s already done and will be released soon”.  There was something really funny about Robert Henke coming up to the mic and humbly submitting his feature requests for Max with everyone else.  I think it was a very democratizing experience.

The second meeting our ‘buddy’ group in the afternoon was provided so that we could take all of the data we had collected to create 5 minute performance/work for the rest of the Expo attendees (their groups had the same challenge).  We used a rope to physically connect the group as we moved through the space with out computers to provide a spatially shifting audio and visual performance.  I’m not convinced that our patches actually work though.  We only had 74 minutes to put it together, so given that constraint I think we did okay.

The conference ended on what may be the climax for me.  After the group projects we made our way up to Berkeley for a a barbeque and performance at CNMAT.  Bob Ostertag and Pierre Hébert put on a stunning show as a part of their Living Cinema collaboration.  The performance featured hand drawing created and then animated in real time.  The hand drawings had a visceral and organic feel to them.  The morphological qualities of these figures was explored through gestural sequences that changes the context of the figures, the meaning of the figures, and the meanings of how the figures related to each other.  Gestural energy and and visual articulation was beautifully at one with the sonic material.

On many levels this is one of the best, if not the best, real-time audiovisual performance collaboration I have ever experienced.  What great way to end the Expo ’74 event.  And I didn’t even mention the food, the wine, the sushi…  it was all extraordinary.  I hope there are more of these events in the future!