INTERACTIVE MIDI INSTRUCTIONAL APPLICATIONS ON THE WEB USING JAVASCRIPT AND THE BEATNIK PLAYER

Interactive MIDI Instructional Applications on the Web Using JavaScript and the Beatnik Player

Keith Kothman
University of Miami School of Music

kkothman@miami.edu
http://www.music.miami.edu/courses/tdml7/

Interactive computer-based ear training drills require large amounts of data for non-repetitious drills and must respond quickly to user interaction. The World Wide Web (WWW) generally fails in both areas. The WWW operates on a whole-file transfer paradigm, meaning any musical information to be played must be sent as part of a pre-made file. In terms of ear training drills any item that you wish to isolate–a specific interval, a chord type, a scale–and each transposition of that item must be saved as a separate file to be delivered on demand to the user. This situation is both an inefficient means of data creation and delivery. Thousands of files would need to be created and, each time the user requested a new interval, chord, or other musical element, the file would have to be downloaded from a server, introducing a multi-second delay (at least) between user action and computer response.

JavaScript and the Beatnik Player in combination provide an effective solution to these problems. JavaScript allows for client-side interaction by interpreting text scripts within the user’s web browser. Unlike CGI (Common Gateway Interface) scripting, no additional calls are made to a remote server. The Beatnik Player accepts JavaScript commands for interacting with a variety of audio file types, including the ability to play individual MIDI notes that are not part of a pre-made file. These capabilities allow for the efficient algorithmic generation of drill material on the user’s own machine, which also creates a satisfying interactive experience for the user.

JavaScript

Although its name makes reference to Java (the object-oriented programming language developed by Sun Microsystems), Java and JavaScript are entirely different programming languages. JavaScript was developed by Netscape as a means for adding limited enhancements to HTML pages. Sensing the growing popularity of Java, Netscape licensed the name from Sun. The biggest difference in the two languages is that JavaScript is a scripting language, while Java is a compiler language. Scripting languages depend on a host environment to interpret text-based commands without any prior conversion of the text into machine-based byte code. The same text commands can be directly interpreted on any computer platform, provided the host application is present and running. Compiler languages require that the text commands be first converted, or compiled, into machine-specific code. It is this machine-specific code that is executed by the computer–not the original text. Programming languages such as C, C++, Fortran, and Basic are compiler languages. Examples of scripting languages include Lingo (part of Macromedia Director) and AppleScript (part of the Mac operating system).

In general, scripting languages share the advantage of being easier to learn, using commands that often resemble spoken language, making use of simplified programming structures that are accessible to programmers with limited experience. Their biggest disadvantage usually is speed of execution. Machine code executes faster than text that needs to be read, interpreted, and then executed.

JavaScript shares these advantages and disadvantages. Compared to Java, it is much simpler to learn, offering a great deal of flexibility in programming style. Its programming syntax resembles C, but–unlike C–it does not require the programmer to specify variables by type (integer number, text, floating point number). JavaScript’s disadvantage in the speed of execution is offset in large degree by the speed boost it provides to the standard browsing experience … especially as it applies to user-interaction. By not having to issue calls to a remote server and wait for new data to be downloaded–the main speed barrier on the WWW–JavaScript greatly speeds the interaction between user and web page. Therefore, even though a compiler language like Java is in fact faster at executing instructions that JavaScript, users are comparing JavaScript’s speed to that of their normal, non-JavaScript browsing experience. JavaScript easily wins that race.

Besides providing for client-side interaction in the web browser, JavaScript extends the functionality of the web browser by adding an object-oriented environment to the browser and HTML. Object-oriented programming environments allow the programmer to define objects as having properties that can be changed without having to recreate a new object. To make an analogy to the physical world, consider the differences between a golf ball and a basketball. An object-oriented environment considers both to be the same object–a ball–each having different properties. In such an environment the golf ball can easily be changed into the basketball, which literally would make storage more efficient if a user only needed one ball for all of the different sports they played. In WWW terms, the background color of an HTML page, set with the BGCOLOR= argument within the <BODY> tag, is a property of the HTML document accessible and changeable via JavaScript commands. Example 1 in the appendix shows how this property can be changed without having to reload the document. It uses HTML form elements such as pull down menus, buttons, and text boxes to trigger JavaScript functions that change the background color of the HTML page when viewed in a web browser.

Example 1. Using JavaScript to Change Document Properties

<HTML>

<HEAD>

<TITLE>JavaScript Color Changing</TITLE>

</HEAD>

<H1 ALIGN=LEFT>Changing Background Colors with JavaScript</H1>

<H4 ALIGN=LEFT>This page only works as intended with Netscape Navigator 3 and above.</H4>

(Explorer doesn't recognize Netscape color names.)<BR><BR><BR>

<HR>

<BR>

<FORM>

<OPTION>YELLOWGREEN

<OPTION>GREENYELLOW

<OPTION>CHARTREUSE

</SELECT>

<H3 ALIGN=LEFT>Or By Typing in Text...</H3>

Type in a Netscape Color Name or Hexadecimal color value,<BR>

then click on the "Change Color" button for either the background or text.<BR><BR>

</FONT>

</FORM>

<HR>

<H4 ALIGN=LEFT>Explanation:</H4>

<P>

The HTML Form (from the popup menu or field) changes the background color

property (BGCOLOR) of the page. BGCOLOR is usually set as an argument in the opening BODY tag. The values being sent are either Netscape Color Names or hexadecimal color values. The usefulness of color names is limited to Navigator. Explorer does not recognize these names.

</P>

<P>

<A HREF="index.html">Multimedia Resource Page</A>

</P>

</BODY>

</HTML>

Example 2 shows how to program button "rollovers" … the most common way that JavaScript is used on the WWW. Button rollovers are images that change when the mouse pointer moves over the image, graphically mimicking the physical actions of a button and suggesting to the user that they could click on the image and the program will respond with some action. This technique subverts–but does not alter–the file transfer paradigm of the WWW. To perform the rollover effect two separate image files must be loaded, one onto the page and one into the browser’s cache memory. Using object-oriented properties of the Image variable in JavaScript allows the replacement of the original image file source with the second image file source. This type of interaction only works if both images have been loaded onto the client computer. Having to call to a server for the image replacement would cause an unacceptable delay in the response time to the user’s actions.

Example 2. Dynamic Image Replacement (Button Rollovers)

<HTML>

<HEAD>

<TITLE>Button Rollover with JavaScript</TITLE>

<SCRIPT LANGUAGE="JavaScript"><!--

var over = new Image()

over.src = "image2.jpg"

/* Replace "image2.jpg" with your image for a button down or highlighted. It will show up when the mouse is over the image. In the body of the document, "image1.jpg" will be your raised, or unhighlighted, button image. */

//--></SCRIPT>

</HEAD>

<BODY TEXT="#000000" BGCOLOR="#FFFFFF" LINK="#FFA500" ALINK="#F37704"

VLINK="#FFA500">

<H1>Dynamic image and Status Text replacement</H1>

</A></P>

<P><FONT SIZE="+1">Image and window status text (bottom of window)

change with mouse rollover.</FONT></P>

</BODY>

</HTML>

The ability to program client-side interactions is only one part of the online ear training programming problem. The standard level of interaction with media files on the WWW–such as digital audio or MIDI–is play, pause, and stop. The Beatnik Player extends this capability to include a variety of interactions one has come to expect when working with MIDI sequencers. Beatnik was developed by a company originally called Headspace, which was founded by the electronic pop musician Thomas Dolby Robertson. Based on the success of the Player the company was recently renamed Beatnik (www.beatnik.com). The Beatnik Player is a plugin for Netscape Navigator and an ActiveX control for Internet Explorer (Windows only). It is also available as an Xtra (essentially a plugin) for Macromedia Director, and can be downloaded as part of Shockwave movies using Director 7. The Beatnik Player is designed to playback multiple music formats, including most digital audio types and MIDI. It also introduces it’s own file format–the Rich Music Format (.rmf)–to allow for customized sample playback and the inclusion of encrypted copyright information with the media file. Used as a sample player, Beatnik accepts MP3 encoded music samples that can be triggered from MIDI commands. Like the Yamaha MIDPlug and Quicktime, the Player includes a software General MIDI synthesizer with additional banks of high quality sample-based sounds.

The full power of Beatnik for software developers comes through the use of Beatnik’s Music Object Application Programming Interface (Music Object API). The Music Object is a JavaScript file that offers methods (functions attached to an object) for controlling the Beatnik Player through JavaScript commands. Recognized commands include MIDI volume control, individual track muting and unmuting in MIDI files, panning control, tempo control, and–most important for my needs–the ability to issue individual MIDI "NoteOn" and "NoteOff" commands. The ability to generate MIDI commands from the client’s browser that control an embedded plug-in clearly breaks the whole-file transfer paradigm of the WWW.

Example 3 shows how the Beatnik Player, the Music Object, and JavaScript interact. As stated above, the Music Object is a JavaScript file and can be included in an HTML page with the <SCRIPT SRC="music-object.js"> tag. Referencing a file in this manner allows for upgrades to the Music Object without requiring the programmer to change multiple pages. All pages simply reference a file that can be changed and improved at any time. The important aspects of this example are the inclusion of the music-object.js file, the JavaScript command that names the embedded Beatnik Player allowing it to be accessed via JavaScript, and the embedding of the Beatnik Player on the page using the previously-declared name and a method called "magicEmbed" that writes browser- and platform-specific HTML code when the page is loaded onto the user’s machine. In this example the embedded instance of the plugin is hidden from the user’s view. The combination of these three factors allows for MIDI commands to be issued and interpreted by the Beatnik Player.

Example 3. Playing Notes with Beatnik and JavaScript

<HTML>

<HEAD>

<TITLE>Playing Notes with Beatnik</TITLE>

<SCRIPT SRC="music-object.js">

<SCRIPT LANGUAGE=JavaScript><!-- //

//* Define function name to access embedded plugin

new music ('notePlay');

// -->

</SCRIPT>

</HEAD>

notePlay.magicEmbed ('<EMBED SRC="stub.rmf" HEIGHT=2 WIDTH=2 HIDDEN LOOP=TRUE>');

<FORM>

<H1 ALIGN=LEFT>Playing Notes With Beatnik</H1>

<H3 ALIGN=LEFT>With Individual noteOn() and noteOff() Commands...</H3>

<H3 ALIGN=LEFT>With a Single playNote() command...</H3>

</FORM>

</BODY>

</HTML>

The Ear Training Pages (http://www.music.miami.edu/courses/et/) are a set of online ear training drills developed by the present author using JavaScript and the Beatnik Player. The pages include interval, scale, and chord drills customized to complement our curriculum and teaching methods at the University of Miami School of Music. The drills are broken up into units that correspond with units in our first-year ear training classes. Through the use of HTML frames and standard HTML form elements the program pages are small (12-17K) and load quickly over telephone Internet connections. Using object-oriented properties of elements like variable arrays, the HTML code for each individual level in a type of drill can be exactly the same. Example 4 shows the <BODY> portion of one of the interval pages and how JavaScript is used to write the HTML code for the page at load time–"on the fly," in programming terms. Since JavaScript does not allow for the movement of individual graphic objects necessary for activities such as dictation, the emphasis is on recognition drills. Graphical manipulation of elements is possible using Director and this is a potential use of the Beatnik Xtra. Student score reporting is not a feature of the program, as this would require the use of CGI scripting. As a department, we are also committed to having students take their ear training tests in person … in class. Scorekeeping is provided to the students locally only as a means for allowing them to accurately evaluate their progress. The most unique feature of the pages is the ability for students to place chords into a consonant or dissonant context, allowing them to hear the voice leading properties of dissonant intervals and their resolutions or how consonant intervals occur as part of larger consonant constructs.

Example 4. Using JavaScript to Write HTML

<BODY>

<FORM>

<!--

document.write('<H1 ALIGN=LEFT>' + docTitle + '</H1>');

document.write('<H2 ALIGN=LEFT>Choices:');

document.write(' '+intervalNameShort[0]);

for (var i=1; i<intervalNameShort.length; i++) {

document.write(', ' + intervalNameShort[i]);

}

document.write('</H5><BR>');

document.write('Change Tempo ');

document.write('<SELECT NAME="speed" onChange="changeTempo(this)">');

for (var i=0;i < metroMark.length;i++) {

document.write("<OPTION>"+metroMark[i]);

}

document.write('</SELECT>');

document.write('<BR><BR>');

//-->

</SCRIPT>

<BR>

<!--

document.write('<SELECT NAME="selection" onChange="answer(this)">');

document.write('<OPTION defaultSelected>Select Interval');

for (var i=0;i<intervalName.length;i++) {

document.write("<OPTION>"+intervalName[i]);

}

document.write('</SELECT>');

//-->

</SCRIPT>

Your answer is ...<INPUT TYPE="text" SIZE="15" NAME="compared" onFocus="blur()">

Percentage correct is ...<INPUT TYPE="text" SIZE="3" NAME="currentScore" onFocus="blur()"><B>%</B> on your first try

<INPUT TYPE="text" SIZE="2" NAME="correct" onFocus="blur()"> correct out of

<INPUT TYPE="text" SIZE="2" NAME="intnumber" onFocus="blur()"> intervals

<!--

document.write('<SELECT NAME="userSelect" onChange="userIntervalMenu(this)">');

document.write('<OPTION defaultSelected>');

document.write('<OPTION>Your choice');

for (var i=0;i<intervalName.length;i++) {

document.write("<OPTION>"+intervalName[i]);

}

document.write('</SELECT>');

//-->

</SCRIPT>

<OPTION>Harmonic

<OPTION>Melodic

</SELECT>

</FONT>

</FORM>

</BODY>

Examining the Ear Training Pages reveals how my programming skill has developed over time … especially through the use of objects. As an example, the creation of a chord object greatly simplified the process of creating chord drills. In programming terms, chords are complex entities. Chords may appear in different voicings or inversions, contain different numbers of chord tones, yet all be the same chord type. Example 5 includes the makeChord object definition and several instances of the object, showing how this problem was overcome through the use of objects and properties. Each makeChord object includes the following properties: chord name, position name, and chord size. The last property is used for determining at what point arguments to the object become part of a new chord … the resolution of the chord. Each chord object also contains several methods that allow for a chord to be played back either harmonically or melodically, to compare the user’s answer to the correct answer, and to resolve the chord, if possible.

Example 5. The Chord Object

function makeChord (chordName,position,size) {

var args = makeChord.arguments; // num of args-1 eq num chord tones

this.chordName = chordName; // property

this.position = position; // property (inversion of chord)

var chordSize = size;

this.canResolve = true;

this.playChordHarmonic = playChordHarmonic; // method

this.playChordMelodic = playChordMelodic; // method

this.resolveChordHarmonic = resolveChordHarmonic // method

this.isCorrect = isCorrect; // method

this.chordTones = new Array(); // property-array filled with args

this.resolveTones = new Array(); // property-array filled with args

for (var i = 0; i < chordSize; i++) {

this.chordTones[i] = args[i+3];

}

if (chordSize != args.length - 3) {

for (var j=0; j < args.length - 3 - chordSize; j++) {

this.resolveTones[j] = args[j+3+chordSize];

}

else {

this.canResolve = false;

}

chords[0] = new makeChord("Major","root",4,0,16,19,24);

chords[1] = new makeChord("Major","root",4,0,7,12,16);

chords[2] = new makeChord("Minor","root",4,0,15,19,24);

chords[3] = new makeChord("Minor","root",4,0,7,12,15);

chords[4] = new makeChord("Diminished","root",3,0,3,6,1,5);

chords[5] = new makeChord("Augmented","root",4,0,16,20,24,5,17,21,24);

chords[6] = new makeChord("Augmented","root",4,0,12,16,20,5,12,17,21);

chords[7] = new makeChord("Major","6",3,0,8,15,-4,12,15);

chords[8] = new makeChord("Major","6",4,0,8,15,20,-4,12,15,20);

chords[9] = new makeChord("Minor","6",3,0,9,16,-3,12,16);

chords[10] = new makeChord("Minor","6",4,0,9,16,21,-3,12,16,21);

chords[11] = new makeChord("Major","6/4",4,0,21,24,29,0,19,24,28);

chords[12] = new makeChord("Major","6/4",4,0,12,17,21,0,12,16,19);

chords[13] = new makeChord("Minor","6/4",4,0,20,24,29,0,19,24,28);

chords[14] = new makeChord("Minor","6/4",4,0,12,17,20,0,12,16,28);

chords[15] = new makeChord("Dominant 7th","root",4,0,16,19,22,5,17,21);

chords[16] = new makeChord("Dominant 7th","root",4,0,7,16,22,5,17,21);

chords[17] = new makeChord("Diminished","6",4,0,12,15,21,-2,14,22);

chords[18] = new makeChord("Diminished","6",4,0,12,15,21,-2,14,22);

chords[19] = new makeChord("Major 7th","root",4,0,7,16,23);

chords[20] = new makeChord("Major 7th","root",4,0,16,19,23);

chords[21] = new makeChord("Dominant 7th","6/5",4,0,8,15,18,1,8,13,17);

chords[22] = new makeChord("Dominant 7th","6/5",4,0,8,15,18,1,8,13,17);

chords[23] = new makeChord("Dominant 7th","6/5",4,0,8,15,18,1,8,13,17);

chords[24] = new makeChord("Minor 7th","root",4,0,7,15,22);

chords[25] = new makeChord("Minor 7th","root",4,0,7,15,22);

chords[26] = new makeChord("Minor 7th","root",4,0,3,7,10);

Conclusion

Like scripting languages in general, there are several advantages and disadvantages to creating online music drills using these tools. On the plus side, and perhaps most important, it works! I didn’t have to learn Java, write or link to software synthesis code, or create hundreds of media files. The pages load quickly, respond to users’ actions instantaneously, and are easy to modify. There are still many untapped possibilities, including student customization of drills and the creation Shockwave movies to allow for on-screen dictation. In theory–and sometimes in practice–the pages work across platforms. The biggest problems encountered involve users needing to install the Beatnik Player and configure their browser properly. In one respect these problems relate to users not being as sophisticated as we often give them credit for being. Beatnik has made installation of the Player as painless and simple as possible … as easy as installing the Shockwave plug-in. Regardless, many students have never changed their web browser settings or configured new plug-in software, so they may experience difficulty if they need to adjust the Player’s settings manually. Also, many students access the WWW through AOL, using AOL branded versions of Navigator or Explorer. JavaScript fails to recognize these browsers as actual Netscape or Microsoft browsers, causing Beatnik to fail to load since the Music Object API includes functions that check to make sure that the user has Beatnik and that they are using an approved browser. It is also important to note–also as a disadvantage–that JavaScript is still a programming language with its own learning curve. With the development of web programming tools such as Dreamweaver, developers can avoid some of the time spent learning JavaScript to perform simple functions. The development of actual applications still requires a full knowledge of the programming language, however, and writing long and specialized functions is most reliable using a text editor.

Another aspect of JavaScript programming to consider is that all of your code is available to the public … just like HTML in a web page. There is absolutely no way that you can give a user access to web pages without everything being sent as part of that web page being public. The only possible way to shield your code from view is to incorporate it into a Shockwave movie, but there will be significant disadvantages in doing so, such as greatly increased download times. Rather than fight the public nature of JavaScript, I have decided to embrace it. Drawing inspiration from the growing Open Source movement I have decided to encourage people to take my code and modify it for their own needs. This will help me by allowing other programmers with either more time or motivation to follow through and make necessary improvements to my application and perhaps help to spread its use to more institutions. My only requirements–mostly unenforceable–are that individuals send me email if they or their classes are using the program and to notify me of any changes they make to the software so that I can evaluate them and decide if they should be included in the "official" release.

JavaScript, the Beatnik Player, and the Music Object API make possible interactive MIDI instructional applications on the WWW. While not without drawbacks, this combination of tools provides the developer with a relatively easy programming language and an extensive set of built-in functions for the control and generation of MIDI data embedded in an HTML document. I have used these features to create the Ear Training Pages, an online set of ear training drills tailored to the teaching needs of my department and school.

Resources

Beatnik. Company Web Site. http://www.beatnik.com/

Heinle. Designing with JavaScript: Creating Dynamic Web Pages. O’Reilly Press.

Kothman. The Ear Training Pages. http://www.music.miami.edu/courses/et/

Kothman. Multimedia Resource Guide. http://www.music.miami.edu/courses/multimedia/

Lemay and Moncur. Laura Lemay’s Web Workshop: JavaScript. Sams.net Press.

Netscape JavaScript Guide. http://developer.netscape.com/docs/manuals/communicator/jsguide4/

Netscape JavaScript Reference. http://developer.netscape.com/docs/manuals/communicator/jsref