In a sample experiment, participants at the Sixth International Technological Directions in Music Learning conference were handed an answer sheet. They were asked to listen carefully to the following instructions:

After the six example tones were played, the participants were allowed to ask questions. At that point, the participants rated 15 consecutively played pairs of instrument tones.

This sample experiment is similar to experiments that have been conducted at UCLA (Hajda, 1995; Kendall, Carterette, & Hajda, in press). These experiments were multi-purposed. First, they tried to establish a palette for natural instrument timbres of the Western orchestra. Second, these experiments were attempts to uncover some of the salient–or perceptually important–criteria of musical timbre.

The basis for much of music theory has been the notational frame of reference. The derivation of rules of counterpoint, pitch-time structures, large- and small-scale forms, harmonic practices, and other stylistic characteristics are commonly learned through student exploration and interpretation–both reflective and creative–of the musical score. Although notation has proved invaluable to our understanding of these constituents of music, it has not been entirely helpful in revealing the underlying forms and structures of musical timbre.

Timbre, as it is used in musical practice, refers to the characteristic sound or sounds created by a particular musical instrument (including the voice) or combinations of instruments. It also refers to the range of qualities that a particular instrument group creates, such as the chalameua and clarino registers of the clarinet, or the coloristic effects of sul tasto or sul ponticello bowing. Finally, certain directions such as dolce and sotto voce imply a specific timbral performance (as well as other non-timbral features).

Many timbral parameters, such as the specification of instruments and playing style, are given in the musical score, and descriptions of the musical effect of such arrangements abound in orchestration texts. However, it is difficult to ascertain any kind of theoretical or predictive structures that might underlie the orchestration of a particular piece. Even today, many contemporary composers and students of composition have no formal system to guide them in their orchestrations; instead, they rely on past experiences and intuition (McAdams, 1996).

Roger Kendall, at UCLA, has continued to develop the Music Experiment Development System (MEDS, Kendall, 1998), a completely automated, computer program for designing and running perceptual experiments. MEDS allows object-oriented interactive computer design. Since MEDS experiments typically involve sound stimuli, the experimenter first creates play lists that contain sound filenames. In the Experiment module (Figure 1), one creates the structure of the experiment through the manipulation of icons. Parameters are passed to these icons, and, if there are no errors in programming, the experiment can be run. The subject sits a comfortable distance from the computer monitor, hears the sound stimuli through headphones, and responds entirely to the computer by using a combination of mouse and keyboard input.

In a similarity scaling experiment such as the one demonstrated at the beginning of this presentation, the subject hears a pair of consecutively presented tones. A horizontal scrollbar appears in the center of the screen with its poles labeled "similar" and "not similar." A cursor appears randomly along the scale. The subject clicks and drags the cursor to the point along the scale that corresponds to how similar the tones sounded. When the subject is finished moving the cursor, s/he clicks on an OK button at the button of the screen, and the next pair of stimuli are automatically presented. This continues until all possible pairs have been presented and rated. The order of presentation of the stimuli is randomized automatically by the program for each subject. MEDS automatically sorts and stores the subject data and presents it in useful forms for direct examination and export to commercial statistical software.

Figure 1. Experiment module of MEDS (Kendall, 1998). Used with permission.

Similarity scaling data is typically presented as a triangular matrix. As an example, I have created a matrix of approximate distances in miles between ten Texas cities (Figure 2):

Figure 2. Matrix of approximate mileage between all possible pairs of ten Texas cities.

This matrix is subjected to a multidimensional scaling (MDS) program. MDS algorithms convert raw data matrices into geometrical configurations, whereby stimuli that are more proximate (i.e. more similar) appear closer together than stimuli that are less proximate. The MDS of the Texas City matrix yields a "map" that corresponds to the relative positions of these cities in physical reality (Figure 3).

Figure 3. Two-dimensional MDS solution of the matrix of mileage between ten Texas cities.

However, there is no guarantee that the orientation of this map will conform to the standard orientation that we are used to in road maps; i.e., east-to-west is presented horizontally right-to-left and north-to-south is presented vertically up-to-down. So, an interpretation of the dimensions of an MDS configuration depends on intuition and the correlation of other variables onto the space. In the case of the Texas configuration, a correlation with latitude and longitude values would yield the common orientation of our "map." Still the extreme separation between Brownsville and Amarillo along Dimension 1 and Dallas-Fort Worth and El Paso along Dimension 2 in Figure 3 make intuitive sense.

Kendall, Carterette, & Hajda (in press) conducted a similarity scaling experiment, using 11 natural instrument tones played at concert Bb4: alto saxophone, bassoon, clarinet, English horn, flute, French horn, oboe, soprano saxophone, tenor saxophone, trumpet, and violin. The demonstration at the beginning of this paper used a subset of these tones. The MDS configuration (Figure 4) reveals certain geometric relationships that do not correspond to traditional instrument families. The clarinet, a single reed instrument, clusters with the flute, a lip reed and the French horn, a brass instrument. The traditional classifications that we use for orchestration–the reeds, brass, etc.–do not necessarily correspond to perception. This is especially evident for the saxophones, which do not cluster at all. On the other hand, Figure 4 does make musical sense: alto saxophone and flute blend very well and alto saxophone and French horn are often cross-cued in band scores.

An MDS configuration such as Figure 4 is an example of a palette of timbre that exists for any instrumental ensemble. This palette is a tonal analog to the visual palette of colors that a painter might use. However, unlike colors, the timbral palette does not correspond simply to a single physical or perceptual attribute. It is multidimensional, and the perceptual and acoustical correlates of these dimensions vary depending upon the stimuli that are being considered.

Additional acoustical and perceptual analyses have aided us in the interpretation of the dimensions of the MDS configuration in Figure 4. For example, acoustically, Dimension 1 yields a strong correlation with the relative amount of energy in the upper harmonics to the fundamental harmonic for each signal. Instruments on the left of the configuration have a relatively high amount of energy in the upper harmonics compared to instruments on the right. The instruments on the left also sound more nasal than the instrument on the right. So, the oboe is nasal and the flute is not nasal.

Figure 4. Two-dimensional MDS configuration for the similarity scaling of 11 orchestral instruments. From Kendall, Carterette, & Hajda, in press. Used with permission.

There are a number of experiments conducted since 1973 that have utilized MDS techniques to create palettes of musical or proto-musical stimuli (e.g. Wessel, 1973; Miller & Carterette, 1975; Grey, 1977; Krumhansl, 1989; Hajda, 1995; McAdams, Winsberg, Donnadieu, De Soete, & Krimphoff, 1995, Kendall, Carterette, & Hajda, in-press). In general, these studies have shown that the primary factor in separating musical stimuli is whether they are percussive (struck, plucked) or nonpercussive (bowed, blown). This makes intuitive sense; the percussive instruments stand out readily from the nonpercussive in the Western orchestra.

The application of timbral palettes to musical practice in Western art music is possible. Pierre Boulez (1987) has written and spoken of the composer's use of timbre throughout history. In the small ensemble of the Baroque, timbre was used to group or segregate the sounds of individual instruments based on their similarity (or dissimilarity). By the nineteenth century, however, the aural identity of a specific instrument in the wash of sound created by a much larger ensemble is sacrificed for new combinant sounds created through multiplication, superimposition, and accumulation. Boulez (1987) refers to the Baroque ensemble as the world of articulation; he refers to the nineteenth century orchestra as the world of fusion. Timbral palettes help to show how articulation and fusion are possible. In general, instruments that cluster together in Figure 4 will blend (fuse) when played concurrently, while the instruments that are far apart will not blend (segregate).

Although much work remains to be done, especially for more complex stimuli such as chords and melodies, a theory–or, more properly, theories–of musical timbre are possible and beneficial to both composers and music educators. It is unlikely, however, that such theories can be derived from traditional notation. Instead, it is necessary to invoke modern technology and empirical methods to achieve predictable models of how timbre is manipulated by composers and perceived by listeners.