Linux Multimedia with PD and GEM: a User's Report

Now let's look at the possibility of combining our two example patches. Using straightforward cut/copy/paste editing, we easily can copy one patch's contents into another and start having some serious fun. Pd truly lives up to the promise of its name: data is purely data here, any data stream can be routed anywhere within a patch (with some restrictions), and we easily can set up a system in which one set of controllers simultaneously controls audio and video parameters.

Figure 6. A Complex Pd/GEM Patch

Figure 6 illustrates our complex combined audio/video patch. As you can see, the two sliders each simultaneously control an aspect of the video along with an aspect of the audio. Adjusting the audio amplitude results in an adjustment of the cube size, while moving the slider for the audio frequency control also will rotate the cube on its x/y axes. Multimedia artists will find great possibilities in Pd's support for such arbitrary attachments and correspondences. I also should note that GEM includes numerous other fascinating pixel-based effects (such as color convolution and pixel threshold control), but I leave their exploration to the interested reader.

Basing our work upon the examples shown here, we are currently planning our presentation for a two-man show. We hope to use no more than two Linux-powered laptops (ideally we would need only one) and a variety of external devices (video recorder/player, still-image projector, lighting displays, etc.). Ease of transportation is a concern because we would like to be able to take the show on the road in a single vehicle.

Our Pd audio explorations have been quite stable in performance, which is good news because we plan to use Pd's real-time audio processing throughout the piece. GEM 0.87_2 sometimes crashes Pd, but I'm using a beta version as I write this article. IOhannes Zmoelnig is dedicated to GEM's improvement, so we reasonably can expect flawless performance by the time we are ready to mount our first performance (targeted for late 2002). We also have seen that combining heavy audio and video processing creates a need for more powerful hardware than we currently employ. I hope to improve our video situation with the addition of a GeForce3 video card. Finally, we also intend to use Pd's shoutcast~/oggcast~ objects for broadcasting our performances live over the Internet.

Pd is incredibly easy to work with, permitting fast construction of relatively complicated patches. The examples shown here were designed merely as learning tools and proof-of-concept demonstrations, and I have already created considerably more complex patches for our project. This article is only a shallow indicator of the possibilities of Pd and GEM, and I encourage all Linux-based audio and video artists to get involved with this software.

Vast thanks to Miller Puckette and Mark Danks for creating and freely distributing Pd and GEM. Thanks also to the Pd community for their continuing assistance to this perpetual newbie, and especially to IOhannes Zmoelnig of IEM for his beta version of GEM 0.87 and for his unstinting aid while I learn more about GEM's video objects. Finally, great thanks also must go to Guenter Geiger for his initial work on the pix_movie object and for his many other contributions to the Linux versions of Pd and GEM.

Resources