The Use of Computers in the Creation
of Digital Sound Sculptures

Dr. Leonard V. Ball, Jr.

School of Music, The University of Georgia

lvball@rameau.music.uga.edu

One of the fundamental tasks facing any composition teacher is to assist students in broadening their understanding and application of timbre. It is not a new concern; indeed, it has always been a topic of discussion at formal and informal gatherings that I have attended with other teaching composers. And, if one will allow a simplistic, broad view of past treatises on theory and composition, each text might very well be summarized with the simple question: "What is the principal idea of this musical passage and how long might it last before it becomes overused and ineffective?" Composers have traditionally employed many different methods of imparting this information: 1) assigning a list of works, with appropriate listening instructions, intended to support a current project; 2) requiring a student search for available recordings related to a particular project, again based on appropriate instructions; and 3) in the case of inexperienced students with a strong background in popular music, assigning a study that encourages the use of particular timbres in relation to an actual timeline, rather than focusing on repeating meter as the motion-stimulating element. All of these tasks, and unspecified others, are essential to the teaching process, and it is not my purpose to suggest that all or any should be abandoned. In today’s technology-oriented world, however, it seems only natural–and vitally important–to include the computer as a means to the desired end. The purpose of this paper is to briefly explore the composition of musical works using a computer as the principal instrument. Underlying this principal focus, as suggested above, is the concept that work in this genre can be beneficial to composition students as they explore the world of sound.

To successfully implement the compositional process using digital audio on a computer, a software or hardware editing system would have to–at a minimum–provide a composition student with the capability to record, or sample, digital audio for use as source material in a composition; to process or manipulate that source material using available onboard software and, if possible, outboard equipment; to organize or arrange the audio according to a written score or, perhaps, by improvisation; and to perform the finished product by recording the work to stereo or multitrack magnetic tape or digital audio tape (DAT), to compact disc, or, in a live performance situation, routing the sound directly from the computer’s hard disk into the sound space. While digital audio works may include traditional harmony, melody, and rhythm, it is customary to encourage students to explore new and unique sound combinations through time. The end musical result of such a process, while correctly termed a magnetic tape piece, a digital tape piece, or even computer music, might best be described as a sound sculpture: an aural image or representation of sonic shapes or gestures molded from concrete sounds and combined, through time, for musical effect.

Pro Tools, the hybrid hardware/software editing system by Digidesign, Inc., successfully satisfies the four minimum criteria specified above … and more. This system–the industry-leading digital audio editor since its introduction–is currently available only on the Macintosh platform. It should be noted, however, that over the last decade several other programs have been introduced for both the Macintosh and Wintel platforms that possess characteristics and capabilities similar to Pro Tools. It is hoped that information presented here will be useful for educators who are interested in and considering the purchase of a digital audio software/hardware digital editing system for instructional use.

To appreciate the advantages of the Pro Tools technology, it is necessary to examine an earlier genre of composition that, in essence, served as the predecessor to digital audio editing, the magnetic tape piece. When listening to tape compositions from the early electronic period (ca. 1951 - 1968), five manipulation procedures are immediately identifiable. As a result, these techniques (rearrangement, reversal, speed variation, looping, and delay and echo) are usually characterized as basic to the production of music from that period. Indeed, these five techniques typically form the core for introductory electronic music courses that cover tape manipulation. Let’s briefly consider each. To provide the raw material for rearrangement, selected sounds were recorded many times and cut into segments. These segments were then arranged either in a predetermined or random order and ultimately joined together to form the desired musical statement. Reversing the tape direction was accomplished by cutting out the desired length of tape and flipping it horizontally. This procedure altered the amplitude envelopes of recorded sounds and, if the tape was stereo or multi-track, allowed varied placement of the sound in the stereo field. Speed variation resulted in pitch transposition of the recorded sounds. Controlling the tape transport mechanism with a tuned oscillator to obtain smooth glissandi, recording the audio at one speed and playing it back at another to obtain shifts of an octave, or lightly resting one’s finger on the supply reel of a tape during playback to obtain a random, fluctuating rendition of previously recorded material are merely three of the possible methods to obtain pitch transposition using tape. Repeating gestures were constructed by joining selected sound segments into a continuous loop that was then threaded through the tape player’s transport mechanism and recorded on a separate machine. Delay and echo–actually two different effects–could be achieved through the use of external devices such as spring or plate reverb, by recording in an ambient recording chamber, or by using two or more tape machines. Different effects that might result include quick head-slaps, single channel reflections, dual-channel multi-reflections or "simple" reverberation. To use any or all of these techniques, a tape composer had to acquire skill in separating and rejoining tape. Splicing–the commonly used name for this skill–was, by its very nature, a destructive technique requiring many hours of practice and, at times, much concern over the loss of desired audio. There exists, however, a significant archive of magnificent works that attest to the viability and effectiveness of these basic procedures and their more sophisticated offspring. Today, even with the availability and accessibility of computers containing software for digital audio editing, there appears to be a revival of interest in composing tape works.

Pro Tools 4, a professional , random-access, digital audio production tool, provides a virtually seamless transition for composers and students trained in basic tape techniques of the classic studio. The system is based on four integrated components: 1) the Pro Tools III system which provides basic software tools for recording, editing, and mixing digital audio; 2) Digidesign’s Time Division Multiplexing (TDM™) bus, a comprehensive hardware and software mixing environment created through the use of a single, internal, high-speed 24-bit data highway that connects all input/output, digital signal processing, and sample playback cards, and, in addition, allows the introduction of software plug-ins for advanced audio processing; 3) the Digidesign Audio Engine (DAE™), the audio operating system software responsible for disk operation management and other system-level tasks, as well as the Digital Signal Processing (DSP) plug-ins from Digidesign and third party developers; and, finally, 4) Pro Tools core hardware. In a typical core system, the basic hardware would include a Disk Input/Output card, a Digital Signal Processing Farm card containing four DSP chips, an external Input/Output Audio Interface providing eight channels of discrete analog to digital and digital to analog conversion and up to eight channels of simultaneous digital audio transfer, and at least one high-capacity SCSI drive approved by Digidesign for use with the Pro Tools system. This basic system may be expanded with additional I/O and Farm cards, sample player cards such as Digidesign’s SampleCell II, and more than one Input/Output Audio Interface.

Developed for and focused specifically on commercial projects, the current version of Pro Tools possesses the flexibility and power necessary to construct effective sound sculptures. Also, due to certain characteristics of digital editing and storage, the system offers some impressive, time-saving advantages over tape manipulation. First, and perhaps most obvious, the time spent rewinding or fast-forwarding to specific points in a tape editing session is virtually non-existent in Pro Tools. The program moves to a selected point instantaneously, and the provided transport tools are logical and familiar. Ample memory location points are provided and may be placed while the file is playing. Second, visual editing of audio files is a much preferred process when compared to the "rock and mark" technique used by tape composers to locate the desired editing point. Audio scrubbing capability is also provided and is extremely useful in precisely locating wave nodes. This process, while similar in concept to tape composition’s "rock and mark" technique, is much more user friendly than its tape counterpart and considerably easier to learn. Third, a two screen layout, with an easy toggle between the Edit and Mix windows, is logical and easily learned. An ideal system would use two monitors, however, one for each window for simultaneous viewing.

The most important advantage over the tape-based editing of the earlier genre, however, is Pro Tools’ non-destructive editing environment. Sound files can be added or removed from playback tracks, truncated, expanded, transposed, moved throughout the stereo field, have their amplitude adjusted or be muted altogether, all without affecting the actual source file. This amazing flexibility can be traced to another Digidesign program, Sound Designer II. SD II, as it’s commonly called, was Digidesign’s first professional digital audio processing software and it has been the industry standard since its introduction in 1988. An early innovation in the sound file manipulation process within Sound Designer II provided an "undo" function that recovered the original file if an effected procedure was unsuccessful. This function, however, from its earliest version to the current Sound Designer issue, always required sufficient disk space for the program to copy and preserve the original file for immediate recall. As sound is recorded into Pro Tools, an SD II file is created. This file is available to both Pro Tools and SD II once it is created … as well as any other program that uses SD II audio files. In Pro Tools, the undo function first introduced in SD II has been improved significantly through the use of performance "maps," i.e., session files that provide timing and digital signal processing instructions for real-time operations to be accomplished during playback of the recorded files. SD II files are usually stored once on a disk where they serve as source material for the entire sculpture. When a file is required in one of the session tracks, the composer drags the file to the point where play should initiate. Pro Tools "maps" the file’s location and length and, at the appointed time, calls up the file for playback. If the file’s location needs to be adjusted, the composer merely drags the waveform graphic to the necessary point in the track. Accurate timing is provided for immediate reference and may be adjusted incrementally to adjust the file in microseconds. Sessions are stored separately from the SD II audio files and other processing information that the composer chooses to use in the piece. Example 1 (below) shows the standard folder that is created when a Pro Tools session is initiated. The icon resembling a tape is the Pro Tools 4 session, while the audio files, any fade information, and plug-in parameters are given folders of their own.

Example 1:

With the latest software version of Pro Tools 4 (1997), an AudioSuite directory has been introduced that performs most of the processing functions originally offered in SD II. With that issue, Sound Designer II is no longer included as part of the Pro Tools package. The program is still functional, however, and may be purchased separately, if desired. As proof of its continued viability, Pro Tools maintains a menu selection under its audio region directory that allows a selected file to be opened in SD II for additional processing.

As stated previously, Pro Tools has two principal working environments: the edit window and the mix window. The edit window is the assembly area for digital audio files. The window will hold up to four virtual tracks of digital audio for each of the sixteen voices in a basic core Pro Tools system. When working in a Pro Tools’ editing session, a single file may be placed in the same or different tracks many times to achieve simultaneous statements or more complicated contrapuntal gestures. If the file is processed using Pro Tools onboard plug-ins, software designed to extend Pro Tools functionality using the DSP Farm cards, then this information is also stored and effected in real-time during playback. Thirteen plug-ins ship with the Pro Tools core system providing control over different AudioSuite elements and real-time manipulation. Even the Pro Tools Project system with DAE PowerMix–designed to run without hardware on all PowerPC Macintoshes including powerbooks–is capable of controlling five DSP functions. In addition, third party vendors provide a variety of plug-ins that greatly extend the capability of the hardware systems. With Pro Tools 4 and sufficient DSP power, incredible control over the audio in a Pro Tools session is possible. In all Pro Tools’ systems, each track has the ability to automate volume and muting, as desired, and–when stereo output mode is selected–panning as well. Figure two below provides a simple example with four session tracks containing several statements of the two files shown in the Audio Regions List at the upper right scrolling window. The original file is named "Crash", while the second is "Crash" altered to effect reverse playback. Not surprisingly the second file is assigned the name "Crash-RVRS-01-00" by Pro Tools once the processing is complete. Additionally, tracks one and two of Figure 2 are in automation mode; track one for pan and track two for volume.

Example 2:

While a thorough explanation of each window element is beyond the scope of this paper, it should be noted that the lower right, scrolling window is provided for any Musical Instrument Digital Interface (MIDI) region files that might be imported into the session. Pro Tools handles MIDI and digital audio in much the same fashion. Although its editing tools are optimized for audio files, the program does provide basic editing features for MIDI such as cut and paste and transposition. If more involved editing of MIDI information is required then a digital sequencer is recommended.

The second screen environment available to the Pro Tools composer is the Mix Window (Example 3). Here the composer sees the individual tracks in a configuration modeled after the standard audio mixer. Levels are viewed on plasma meters, sliders are automated and respond to any volume and pan information entered in the editing window, input and output channels may be assigned here as well as in the edit window, and the screen view may be augmented with plug-in and sends information. The use of assignment buses, available in the TDM-capable, hardware-based Pro Tools systems, ensures that all routings may be accomplished in the digital domain prior to transfer to recording media. In Example 3, the four audio tracks are routed internally to two master faders that, in turn, control the amplitude at stereo outputs 1 and 2 and outputs 3 and 4. These stereo outputs, when patched directly to two amplifiers, provide signal for a quadraphonic performance space. With the appropriate number of amplifiers and speakers, a core Pro Tools system is capable of providing signals for up to eight stereophonic fields or sixteen monophonic outputs.

Example 3:

Powerful, flexible, dependable, and firmly rooted in traditional recording, editing, and processing techniques, Pro Tools offers the student composer an excellent opportunity to experiment with timbre in a friendly, forgiving environment. Students introduced to the program at the University of Georgia are encouraged to record and manipulate "concrète" sounds in order to discover new and unusual timbres. Once these new timbres have been developed and organized for easy reference, the students "compose" the sculpture in the Pro Tools editing window, performing real-time processing using the available plug-ins as desired. With no traditional meter available, the students at first struggle with–and then later embrace–the movement of sound through time. To date, with only a few exceptions, interested students have explored the software, become proficient in the necessary skills and techniques, and produced a reasonable sculpture in a relatively short period of time. Additionally, these same students return to their acoustic compositions with a heightened appreciation for timbre and its development through time.

References

Digidesign. Pro Tools Software Guide for Pro Tools III Systems . Palo Alto: Digidesign, a division of Avid Technology, Inc., 1997.

Morris, William, ed. The American Heritage Dictionary of the English Language . Boston; Atlanta; Dallas; Geneva, Illinois; Hopewell, New Jersey; Palo Alto: Houghton Mifflin Company, 1980.