Music technology is becoming increasingly accepted as a method of evaluating music performance. In studying piano performance, Musical Instrument Digital Interface (MIDI) allows the "capturing" of pianists’ performances and the subsequent indexing of precise measurements of all notes (keypresses) performed. Not only does MIDI provide the means for assessing technical performance (i.e., accuracy of pitches and rhythms), it also affords immediate access to measures of the expressive features of music, namely loudness, tempo, and articulation (relative note duration).

Some private studio music teachers are using MIDI technology in their work with students (e.g., Henderson, 1996). Often this is accomplished by having students practice on a MIDI-compatible acoustic piano, so they may then review detailed information about their performance. It is perhaps lesser known that this type of music technology is also of great use to researchers in cognitive psychology who are interested in getting "beneath" the external performance and investigating the internal processes of musical thinking and learning. Carefully devised research methods and procedures allow external behaviors—like musical performance—to serve as indicators of internal cognitive processes. When coupled with such research approaches, music technology is a powerful tool for studying and describing the "inner workings" of expressive music performance, and not just the performance itself.

Research findings in music cognition can be of tremendous practical value to music education. In fact, the technology and methods of study used in this type of research are, in many cases, immediately transferable to music instruction. This is largely due to an important similarity between cognitive research (in music) and music education … namely, the need to assess and evaluate how people produce music, how they interact in musical settings, and what they think about music.

In this paper, I begin by briefly illustrating how various types of technology have been used in past studies of expressive music performance. Next, drawing from the findings of such research, I address the task of exploring the cognitive processes involved in expressive music performance. In this section, I put forth a theoretical model of mental representations used in music performance. This model provides the background for the next section of the paper, which shares a research study recently conducted by the author. This study serves as an example of how technology can contribute much to investigations into music cognition. Finally, this paper will conclude by offering some conclusions regarding instruction in expressive music performance (the topic of the example study), as well as the promising role of technology in music cognition research.

At the present, MIDI is the most prevalent form of music technology used in studies of expressive music performance. However, before MIDI was popular and accessible to musicians and music educators, researchers were using similar antecedent technology to accomplish their research objectives. For example, in their studies of pianists’ uses of expressive timing, Shaffer (1981) and Sloboda (1983) used Bechstein grand pianos that were outfitted with photocell sensors and electronic circuits to detect and code movements of the piano key action. The same capabilities are now afforded with MIDI keyboard instruments, including specially equipped acoustic pianos such as the Yamaha Disklavier (e.g., Woody, 1999). The availability of MIDI technology has resulted in increased research activity in the area of music performance. It should be noted that most MIDI-assisted studies of expressive performance have primarily (if not exclusively) focused on the tempo aspects of expressivity, including Repp (1992), who examined the temporal lengthenings (i.e., ritardandos) at ends of musical phrases, and Clarke (1993), who analyzed musicians’ reproductions of real and artificially altered expressive performance models.

More specialized applications of MIDI technology have been utilized effectively in studies of expressive performance. One such example is the software Instant Pleasure, incorporated into a research study by Johnson (1998). This program allowed subjects to replay a MIDI preprogrammed piece of music by clicking a mouse button to indicate the onset of the next note of the piece. In effect, the person determined the basic performance tempo and was able to execute rhythmic nuances (e.g., hesitations, rubato). In his study, Johnson found that the performed expressive timing of music majors improved considerably—to become more like that of experts—after receiving direct instruction on rubato usage, presented graphically and in prose.

Other studies in music cognition have used technology that—although not specifically music technology—has been integral to the purposes of the research. The sentograph is a device that measures finger pressure on a pushbutton. At least two teams of researchers have used the sentograph to investigate whether music expressing different broad emotions is related to different basic motional patterns (Clynes & Walker, 1982; Gabrielsson & Lindström, 1995). In this research, subjects were asked to "press in a way that fits the music" or "express the pulses in the music." Results suggest that there are indeed characteristic pressure patterns for music expressing different broad emotions.

An unusual approach to studying expressive music performance was taken by Davidson (1993). Her study focused not on the sound, but the body movements of musicians engaged in expressive performance. In order to focus solely on body movement, Davidson employed the technology-assisted "point-light" technique. Musicians were videotaped while performing with ribbons of glass-bead retroflective tape attached to their major body joints. High powered theater lights were mounted close to the lens of the videocamera. On playback of the videotape, the brightness and contrast settings of the television monitor was adjusted in order to, in effect, "black out" all visual information except movement of the ribbon spots of light. As might be expected, Davidson concluded that movement is an important source of perceived expression in a live musical performance.

Most of the aforementioned researchers aspired to go beyond merely describing the properties of musicians’ expressive performances; their ultimate purposes involved testing theories that explain how musicians produce expressive performances. Of course, none of the technologies employed by these researchers allowed them to directly tap into the thought processes of musicians. (It is likely that the development of such technology is still a long way off!) Instead, these researchers relied on external performances of musicians to serve as indicators of internal cognitive processes. This was accomplished by incorporating technology into innovative research designs and experimental methods. Also necessary for using external performance to indicate internal cognition is a well-established theoretical framework.

One such theoretical model will now be presented and provides the context for the example research study covered in the next section of this paper. Lehmann (1997) and Ericsson (1997) proposed a model of mental representations used in music performance. This model suggests three component cognitive skills used by musicians when performing a given piece of music: (a) goal imaging or the ability to formulate a clear idea of what the music should sound like, (b) action production or the ability to generate the physical movements and fine motor skills required to produce the music, and (c) a self-monitoring or the ability to accurately hear the true sound properties of one’s own performance. These cognitive skills are interrelated and dependent on one another.

The example study presented in the following section focused primarily on the goal representation component. Access to musicians’ goal imaging thought processes was gained by combining the use of MIDI technology with verbal protocol research methodology and an aural modeling (imitation) task experimental design.

Recently, the present author conducted a study that sought to provide insight into musicians’ cognitive processes during expressive musical performance, focusing specifically on variations in dynamics or loudness (Woody, 1999). The main purpose of the study was to determine whether performing musicians use explicit planning (goal imaging) in expressive performance. This is an alternative explanation to the traditional notion that expressive performance results from musical intuition, and that it cannot be pre-calculated.

The subjects in the study were 24 university musicians with advanced piano performance skills. While seated at a Yamaha Disklavier, subjects heard expressive models of two brief piano excerpts and attempted to imitate them in their own performances. The models contained designed loudness features of two types: 1) idiomatic features, which would be considered appropriate by musicians in the specific musical context, and 2) nonidiomatic features that would be considered musically inappropriate.*1

Immediately after hearing each model, subjects were prompted to give retrospective verbal reports of their thoughts while listening to the model, in effect indicating the expressive (loudness) features that they identified (Ericsson & Simon, 1993). MIDI technology was used to record subjects’ performances as computer files, providing precise measurement of the loudness of notes played. The collected verbal protocols were transcribed and objectively coded in order to serve as indicators of subjects’ mental representation of goal performance.

Technology played an important role in the completion of this research in the following ways:

1. A Yamaha Disklavier was the musical instrument used. Subjects both heard the models and their own imitative performances on the Disklavier, contributing to the study’s internal validity (i.e., experimental control). The fact that the Disklavier is a real acoustic piano contributed to the the external validity, i.e., the generalizablity of the findings to the real world of piano performance.
2. An IBM-compatible desktop computer was interfaced with the Disklavier, using a MIDI sequencer software application. The sequencer was used to play MIDI files of the expressive models, as well as to record subjects’ imitative performances as MIDI files.
3. Audio recording (to cassette tape) was used to document subjects’ verbal reports of the expressive features heard in the models. The recorded verbal reports were subsequently transcribed into computer text files for further analysis.
4. A MIDI-to-text conversion software application was used to transform the data of subjects’ performances into a text format that could then be imported into a spreadsheet program and a statistical analysis application.
5. The software program Multiple Protocol Analysis System (MPAS) allowed for objective and context-free coding of subjects’ verbal reports (Crutcher, Ericsson, & Wichura, 1994). Using MPAS, a coder was presented segments of the verbal reports, drawn randomly from the transcription text file. The code applied to verbal reports indicated the type of loudness feature mentioned (decrease or increase) and the notes on which the feature began and ended.

The results of the study strongly indicated that expressive music performance is mediated by explicit mental planning or goal imaging. On most models, there was a high correlation between the number of loudness features explicitly identified by subjects and the performance scores for imitation accuracy. Analyses of individual features revealed that subjects who explicitly identified features consistently performed the features differently than the subjects who did not identify them. With nonidiomatic features, the musicians who identified features performed them significantly more accurately. With idiomatic features, subjects who identified features performed the features at more pronounced overall levels (i.e., louder crescendos, softer decrescendos). These differences between the groups of subjects—those who identified features in their vrbal reports versus those who did not—were determined to be statistically significant using repeated measures analyses of variance.*2

At the most basic level, the results of this study suggest that music teachers should pay greater attention to the thought processes that underlie expressive music performance. Most teachers want their students’ to share their thoughts about long-term performance goals and practice strategies. However, it is equally important to know what students are thinking when carrying out specific music performance tasks. Music teachers should consider using student verbalization in expressive performance training activities.*3

Also in light of this research, I encourage music teachers to direct their instruction specifically to the sound properties of their students’ expressive performances. The MIDI technology allowed precise measurement and analysis of the expressive property of loudness. These MIDI data reflect what was actually performed, not what the performing musicians’ intended to do. Of course, this is exactly what a real listening audience must rely on to perceive expressivity. Because listeners cannot look directly into the inner motivations or intentions of a performer, it would seem that music teachers should not focus their efforts on such elusive ideas, but instead attend more to the physical sound properties of performance.

Finally, teachers are urged to use additional verbalization to supplement the instructional approaches of aural modeling and nonmusical metaphors or imagery. For example, consider a music teacher who performs a musical excerpt in order for a student to imitate the style or expression of the model. The results of my research suggest that it would be unreasonable for that teacher to expect the student to accurately imitate the model unless the student can explicitly identify expressive features contained within the model. With regard to expressive instruction based on nonmusical metaphors or imagery, teachers must realize that there is a translation process involved in this approach. Many student musicians will need help translating a metaphor (e.g., "make the phrase soar like an eagle") into a plan for the perceivable sound properties of an expressive performance.

The information presented in this paper demonstrates the inter-applicability of the areas of technology, research in music cognition, and music performance training. Better establishing the relationships between these areas must be tactfully pursued. Just as some musicians traditionally have been threatened by increasing uses of technology in fields of music performance, there exists in some musical circles a rejection of efforts to study music in a systematic, empirical manner. This is especially true with regard to attempts to scientifically explain expressive performance.

Relatedly, it seems that uses of technology in music cognition research are most effective when the technology is combined with well-crafted research methods and a strong theoretical framework. In other words, technology must be considered a tool for accomplishing existing research objectives. Among music teachers who follow developments in technology, there is sometimes the temptation of choosing an exciting new technology and then looking for a way to use it in teaching activities. This is similarly tempting for music researchers. But just as teachers should start with existing student learning objectives and use technology to achieve them, researchers must begin with informed research questions and established theoretical frameworks.

Notes

*1 Both sets of features were validated in a pilot study.

*2 For details, see Woody, 1999.

*3 For more detailed suggestions for doing this, see Woody, 1999-2000.