All audio processing is done through robotcowboy system software in realtime. Sounds and instrumental mappings can be specified and changed at will at the press of a button. Digital audio tools allow for mixing and recording in realtime during live performances. Realtime computer graphics sync live sound and visual generation.

robotcowboy runs on a minimal GNU/Linux Ubuntu operating system. Ubuntu is free and open source and ideally suitable for realtime audio. I have slimmed it down and removed all unnecessary software in the system allowing my outdated Pentium3 500 MHz to handle all audio and graphics in realtime. What is realtime? Imagine if you tried to play a piano and it took one whole second to sound the note after you pressed a key … that would make it nearly unplayable. It would not be a “realtime” instrument. Admittedly, no computer can be truly running “now”, but any system that can run as close to realtime as possible will yield a more responsive instrumentality.

The JACK audio connection kit is a realtime audio daemon. It handles sound card interfacing and midi routing via ALSA to Pure Data.

Audio processing and instrumental mappings are done in custom Pure Data patches. Pure Data is an object-oriented graphical programming environment for audio and visual processing. You create programs by connecting little boxes together. As opposed to “studio” production software such as Reason, FruityLoops, Logic, etc, there are no presets or pre-built instruments . You are given all of the elements you need to build your own digital synthesizers, effects, drum machines, and compositional objects. I feel far more creative and satisfied in building my own instruments as opposed to merely plugging in VST’s and software synths. I can add any sort of functionality I need such as network control and algorithmic midi processing.

Songs are individual Pure Data files or “patches”. My main PD patch loads the song patches when triggered and records the output. Musical control from joystick and midi devices are mapped with the song patches to audio generation. This is where the “instruments” live.

SpaceLight song score in PD

SpaceLight drum machine in PD

You can download Pure Data for free and use it, its open source. You can also download my personal patch library I use to create my music in PD.

Joystick and gamepad devices are controlled by my custom software which forwards all of event data over OSC (Open Sound Control) to the audio system in Pure Data. USB device insertions and removals are handled automatically and there is also Wiimote capability, although I have not yet finished implementing it.

The robotcowboy visual system is a simple sprite engine which is controlled over the network via OSC (Open Sound Control). I built it in about 3 days before the Consoles Afire Tour in Jan 08 and it reads simple text files which specify the sprites. It uses SDL (Simple Directmedia Layer) and is graphics accelerated and lightweight since it does not need to run through a graphical user environment such as OpenGL on the X Windows system.

Since visual is controlled over the network, multiple computers can send control data to a single instance of visual and a single computer can control multiple instance of visual. During the Consoles Afire Tour 08, my wearable computer ran its own visual and also controlled the visual running on my laptop hooked up to a projector. The helmet and giant projection were synced in realtime over the wireless network.