AVSynthesis: Blending Light and Sound with OpenGL and Csound5
The artistic combination of sound and image is a common enough phenomenon. Movies, television and various Internet channels demonstrate the happy results from the blend of recorded sight and sound. However, these examples typically utilize sound in the role of an accompanist, perhaps greatly significant but still primarily an accompanist.
There is another way to consider the role of music and sound in video production—a way in which the sound itself informs the flow of images and their transformations. Although not a novel concept (see the Wikipedia entry on John Whitney), the practice has taken on a new richness of possibilities with the use of computers in the recording and editing of digital son et lumière.
Jean-Pierre Lemoine has been exploring these new riches at least since the late 1990s. I profiled his HPKComposer (coauthored with Didiel Debril) in my Book of Linux Music & Sound, which was written in 1999, and even then the HPKComposer Web page stated that the program was “... a 3D art composition tool for Csound”. At that time, the authors chose to use the Virtual Reality Modeling Language (VRML) for its graphics engine. I could meet the program's Java requirements and work with its Csound side, but I was unable to work with VRML under Linux then. Nevertheless, the Web site's screenshots made quite an impression, and I hoped that someday such a program would become useful under Linux.
Cut to the work of Csound developer Gabriel Maldonado: his CsoundAV for Windows is a true fork from the canonical Csound source tree, but Gabe is a genial fellow who freely offers all his code extensions to the community. Recent developments in canonical Csound have facilitated the adoption of some CsoundAV opcodes, though we await the inclusion of the CsoundAV opcodes for OpenGL, and this situation brings us to the latest work of Jean-Pierre Lemoine, titled simply AVSynthesis (Figure 1).
AVSynthesis embraces and extends many of the design concepts behind HPKComposer. The program blends sound and images to produce abstract non-representational works of art. It's written in Java, and Csound is still the audio engine of choice, but the VRML interface has been replaced by a set of image controls based on the OpenGL shading language (GLSL). The program creates radical associations and correspondences between image and sound, leveraging the powers of Csound and OpenGL for the arbitrary manipulation of digital audio and digital images.
Like many experimental applications, AVSynthesis is not a perfectly packaged program, and it is not ready for use right out of the box. It is a unique program, and as such, it has some unique requirements that may not be met by your distribution's package repositories. Building the required dependencies is not especially difficult, as long as you have a typical Linux development environment installed and properly configured for your system. I include here the particular instructions for compiling Csound and configuring AVSynthesis, with some notes on the requirements for building the application on a 64-bit system.
AVSynthesis demands a specific set of dependencies:
Java (1.5 or higher)
LWJGL (the Light Weight Java Game Library)
Csound (5.07 or higher)
Where they are noted, the versions are critical, and each component is subject to its own build prerequisites. As mentioned, Csound needs some special attention in order to use it with AVSynthesis.
Csound has its own set of necessary dependencies, but space restrictions here forbid a complete description of the program and its requirements. Fortunately, thorough and excellent documentation is available from www.csounds.com, so I focus here only on the configuration needed to compile the program for use with AVSynthesis.
The following options configure and compile the csound binary for double-precision floating-point numerics and create lib_jcsound.so, a Java “wrapper” library for Csound's audio synthesis and processing services:
scons useDouble=1 install=1 buildPythonOpcodes=1 buildInterfaces=1 ↪buildJavaWrapper=1 dynamicCsoundLibrary=1
The Python opcodes are not required by AVSynthesis, but I include the option for use with Steven Yi's blue, a superb environment for working with Csound. All other options in this build configuration must be included for work with AVSynthesis. If the build is successful, the lib_jcsound.so library will be at the top level of the Csound source tree. Install Csound (scons install), then copy lib_jcsound.so to the AVSynthesis native directory. That's it; you're finished with setting up the audio side of AVSynthesis.
The OpenGL and LWJGL libraries provide the interface's visual components and style. The various parameter control screens resemble the control panels seen in many OpenGL-based games, with visual effects, such as animated icons and mobile transparencies—niceties that liven the appearance of the program and improve its work flow.
The LWJGL libraries present a minor difficulty. The AVSynthesis package includes the LWJGL libraries as Windows-format DLLs but not the required native Linux libraries (that is, in .so format). The package includes these DLLs:
Those files must be replaced by the following native Linux equivalents:
The lib_jcsound.so library comes from the Csound build described above; the others come from the LWJGL binary package (downloaded from lwjgl.org). Alas, 64-bit users will need to build and install the LWJGL and the IL libraries themselves. As far as I could tell, packages for these libraries are not readily available in 64-bit format, but building them is trivial and requires no special instructions beyond adding --with-pic to the configuration step (./config --with-pic). After building or downloading the libraries, they must be copied to the AVSynthesis native directory. You then can move or delete the DLL versions.
Neither Java nor OpenGL requires any rebuilding or special runtime options. These are common packages now, so if you don't have them installed already, summon your package manager and install the latest versions (Java must be 1.5 or higher). AVSynthesis itself is launched from a .jar file that works equally well in a 32-bit or 64-bit environment.
In addition to these software requirements, your computer should have a fast CPU and a video system capable of accelerated 3-D graphics. I tested AVSynthesis on two machines: a 32-bit box with an AMD64 3800+ CPU (a 2.4GHz chip) and a 64-bit machine powered by an AMD64 3200+ CPU (2GHz). Both systems include NVIDIA graphics boards (GeForce 7300GS and GeForce 7600GS, respectively), with xorg.conf configured for NVIDIA's proprietary nvidia driver (that is, not the open-source nv module). The 32-bit iron runs the JAD distribution, based on OpenSUSE 10.2, and my 64-bit box runs 64 Studio, a Debian-based distro. Both systems are optimized for multimedia and include kernels optimized for real-time performance. However, programs such as AVSynthesis want resources, lots of them, and I consider my machines as rather low-end for AVSynthesis. Your mileage may vary, of course, but for the best results from this program, I recommend a 3GHz CPU, at least 2GB of RAM, a fast 3-D graphics card and a large, fast hard disk.
I also recommend a high-quality audio system. Cheaper desktop speaker arrays may be suitable for watching DVDs, but Csound is capable of audiophile-quality output, so you'll want a sound system as powerful as your graphics system. Here at Studio Dave, I have my JAD box connected to a relatively low-end 5.1 sound system (a combination of Creative Labs and Peavey hardware), while the 64 Studio machine is hooked up to a conventional small studio audio system with a Yamaha digital mixer, a standalone 100-watt power amplifier and a pair of high-quality monitor speakers.
Similis sum folio de quo ludunt venti.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- Paranoid Penguin - Building a Secure Squid Web Proxy, Part IV
- SUSE LLC's SUSE Manager
- Google's SwiftShader Released
- Managing Linux Using Puppet
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- SuperTuxKart 0.9.2 Released
- Parsing an RSS News Feed with a Bash Script
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide