The Trials and Tribulations of Developing Cross-Platform Multimedia Applications in Music Education

Dr. Scott D. Lipscomb

Institute for Music Research, The University of Texas at San Antonio

lipscomb@utsa.edu
http://music.utsa.edu/~lipscomb

In recent years, like a large number of other music instructors around the world, I have become convinced of the efficacy of using technology to assist students in the process of musical learning. It seems, however, that there remains a great divide between two sets of users: Mac-users and those using the PC platform.1 One of the major decisions facing every developer is the platform or platforms for which software is to be developed … a decision that, by necessity, can eliminate a large number of potential users.

I was extremely excited when I learned of the potential for cross-platform development; i.e., the idea that code could–in theory, at least–be written one time and compiled for use across several platforms. I set out immediately to learn how to create cross-platform versions of a variety of programs I had previously created using Visual Basic for Windows-based computers. Though more complex than scripting languages like Lingo,2 Visual Basic provides a relatively easy method for programmers to create software incorporating a user-friendly graphical user interface (GUI). The creation of these interfaces is easily accomplished by selecting an item from a Tool Palette (e.g., a button, label, textbox, scollbar, etc.) and simply using the mouse to point-and-click at the location where the object is to be located. Once placed, objects can be relocated or resized by clicking and dragging. One can accomplish the creation of similar interfaces using Hypercard (Mac), Authorware (Mac), Toolbook (PC), or Director (Mac or PC). Limitations of these approaches will be discussed below.

For the purpose of making applications available to the widest possible user-base, creation of web-enabled software provides perhaps the most viable alternative. Unfortunately, however, Visual Basic does not at present provide an easy method for creating such internet-compatible versions of software. According to pre-release promotional material, this weakness is at least partially addressed in version 6.0, to be released later this year. As a result of these short-comings, I began searching for another development environment.

Object-based programming vs. "spaghetti code"

OOP (Object-Oriented Programming) has kept the programming world buzzing since the release of C++ and the its importance was confirmed with the introduction of Java. How many potential programmers have been discouraged by subjection to spaghetti code excerpts like that in Listing 1? This code is taken from a program written in BASIC by the author (MELCOUNT.EXE) that prints a series of asterisks to the computer screen representing the number of times a given pitch class occurs in the musical theme under analysis. Though the output is not particularly beautiful by present-day standards, a comparison of the output for the initial 100 thematic notes of the first movement of J.S. Bach’s Brandenburg Concerto #2 (Figure 1a) and the first 80 thematic notes of Schoenberg’s Piano Concerto, op. 42 (Figure 1b) provides convincing visual confirmation of the difference between the diatonicism of the Baroque era and Schoenberg’s atonal (or pantonal) technique to accomplish the "emancipation of dissonance." Note that not one of the first 100 melodic pitches in the Bach concerto is outside of the diatonic scale of F major, the key of the first movement. Compare this selective pitch class collection with the relative equanimity of pitch classes used in the excerpt from Schoenberg’s concerto.

Listing 1. Code excerpt from MELCOUNT.

SUB Graph (type$, Labels$(), Frequency(), MelodyName$, MelodyLen, DurCount())

CLS

PRINT TAB(10); "Frequency of occurrence "; type$; " in"

PRINT TAB(12); MelodyName$; " (sample size ="; MelodyLen; "notes):"

FOR x = 0 TO UBOUND(Labels$)

LOCATE x + 4, 15

PRINT Labels$(x);

LOCATE x + 4, 20

PRINT STRING$(Frequency(x), "*")

NEXT x

LOCATE 24, 30

PRINT "Hardcopy? (Y/N) ";

DO UNTIL x$ = "Y" OR x$ = "N"

x$ = UCASE$(INKEY$)

LOOP

SELECT CASE x$

CASE "Y"

LPRINT

LPRINT TAB(10); "Frequency of occurrence " + type$ + " in"

LPRINT TAB(12); MelodyName$; " (sample size ="; MelodyLen; "notes):"

LPRINT

FOR x = 0 TO UBOUND(Labels$)

LPRINT " "; Labels$(x); CHR$(9);

LPRINT STRING$(Frequency(x), "*")

NEXT x

LPRINT

LPRINT

EXIT SUB

CASE "N"

EXIT SUB

END SELECT

END SUB

Figure 1. Output of MELCOUNT.EXE showing the number of occurrences for each pitch class in a) the first movement of Bach’s Brandenburg Concerto #2 and b) the 1st movement of Schoenberg’s Piano Concerto, op. 42.

Though not considered a truly object-oriented language, Visual Basic does incorporate an object-based syntax3 for accessing the properties of GUI items or other user-defined variable types. Using this syntax–"dot syntax," as it is called–objects and properties are represented in the following manner: Object.property. Listing 2 illustrates the use of this notation to represent a common, everyday object and some of its properties.

Listing 2. An example of Object.Property dot notation applied to a real-world object: a table.

Table.Material = "MARBLE"

Table.NumberOfLegs = 4

Table.Length = 48

Table.Width = 36

Table.Height = 30

For a more typical programmatic example–i.e., to change the background color (a property) of a textbox (an object) to blue and the color of the text itself (another property) to white–one would simply insert the code shown in Listing 3.

Listing 3. A more typical programmatic example of Object.Property assignment. 4

Textbox.BackColor = BLUE

Textbox.ForeColor = WHITE

 

Using this object-based approach assists the programmer to write code that is more readable and easier to maintain. Therefore, in comparison to the coding process of previous decades, Visual Basic and other object-based programming languages provide an easy method for creating the user interface and a much-improved means of writing the actual code that is triggered by user-interaction via the various objects of the GUI.

Interactivity is absolutely essential to programs created with Visual Basic, since it is an event-driven language. In contrast to "structured programming" of the past in which lines of code are processed in the same order every time a program is run, the order of processing in event-driven programming languages is determined to a large degree by the user. Programmers write code to respond to mouse clicks, text input, and other types of interaction performed by the user.

Therefore, Visual Basic provides programmers with a relatively easy method of creating aesthetically pleasing and user-friendly GUIs, an object-based means of referring to these graphic elements, and an event-driven architecture to respond to interactions of the user. However, Visual Basic presently provides no means for developing software for non-Windows platforms. In addition, the creation of web-compatible versions of applications has not proven particularly effective. For anything beyond the most basic functionality, the programmer must choose to compile the program as an ActiveX control. Currently, to access pages utilizing ActiveX technology, users must access the web site using Microsoft’s Internet Explorer. Undoubtedly, this will be addressed in future versions of other major browsers, but it remains a stumbling block and source of frustration at present.

Cross-platform development: Promises and disappointments

Traditional programming languages (e.g., Fortran, Basic, Pascal, C, Unix, and others) have served well in the creation of software for a specific operating system. The object-oriented approach of more recent programming languages (e.g., C++ and Java) assists in code maintenance and reusability. Finally, the cross-platform promises of Java and Microsoft’s cross-platform version of Visual C++ are certainly attractive. In reality, however, I have found that the further one ventures from the basic GUI objects and their built-in properties, the more likely the program will not, in fact, function on an alternative platform. In the documentation for the cross-platform version of Visual C++, Microsoft claims approximately a 90% code compatibility when moving from PC to Mac platforms. This still leaves a great deal of work to be done by the programmer before the Windows program can successfully be compiled and integrated onto the Mac platform. When the program incorporates multimedia in the form of sound, graphics, and/or animation, the percentage of reusable code drops significantly, resulting in even more work for the programmer in the process of converting software to the Mac platform. Because the sound-producing hardware and graphic processing on the Mac and PC platforms remain incompatible, the manner in which the various forms of multimedia are accessed varies greatly between the Mac and PC.

As an early attempt at cross-platform development, I decided to modify a program I created (SIGNAL.EXE) for my Introduction to Audio Technology class. The purpose of this application is to illustrate how the amplitude of simple sine components combines to form a complex signal shape and how these changes in signal shape are perceived as changes in timbre. The program consists of four modules. The first (Figure 2a), allows the user to determine the amplitudes (0.00 to 1.00) and phase relationships (0 to 359) for up to 10 sine components. When the user clicks the button labeled OK, the "Fourier Analysis" interface (2b) appears, showing a graph of each sine component (top), the resulting complex signal shape (middle), and a spectrogram of sine component amplitudes (bottom). By selecting the OPTIONS menu from this window, a sound file (WAVE format) can be made incorporating the complex signal shape just created, at a fundamental frequency and note duration determined by the user. Finally, by returning to the Main Form, the user can view an animation of an air molecule acted upon by the complex signal s/he created.

Figure 2. Screen captures showing the "Build a Complex Signal" module (2a) and the "Fourier Analysis" module (2b) of SIGNAL.EXE.

My initial attempts at a simple transformation of this Windows-based software to a Mac-compatible version using Visual C++ have proven unsuccessful to date. Though point-and-click creation of the user interface allowed relatively quick recreation of these components, a number of low-level calls in the original code–specific to the PC platform–were necessary in order to create the sound file, save it to disk, and play it back upon demand. A cross-platform version of this software would require knowledge not only of the Windows API (Application Programming Interface), but a similarly comprehensive understanding of the Macintosh Toolbox functions. It is at this point that I began to question whether such low-level coding is really the way to proceed. I am, after all, a musician and not a professional computer programmer.

In order to complete this project in the manner I originally intended, I am now convinced that it will be necessary to write a function to determine the user’s platform (a rather trivial function, in itself) and handle most, it not all, multimedia operations with a series of IF/ELSE statements, providing one set of instructions for the Windows version and a separate set for the Mac version. In my opinion, labeling such a development environment as "cross-platform" is, at best, questionable … at worst, deceptive.

Conclusions

My experience has shown that Visual Basic provides a quick, reliable method for creating functional software in a surprisingly brief amount of time. It is relatively easy to learn, but is available only for the Windows platform at present. In order to allow applications to be accessed via the Internet, the current version requires that the project be saved as an ActiveX control that is then placed within a Web page’s HTML code using the "Object" tag. At present, ActiveX controls can be accessed only from Internet Explorer, unless the browser has appropriate plug-ins installed.

Visual C++ allows much greater access to the operating system and low-level programming operations. In exchange, the program has a very steep learning curve. The cross-platform edition adds another level of complexity and, in my experience, has not proven to be worth the effort expended. Similar to Visual Basic, there is not yet an easy Web access solution since Visual C++ also requires the programmer to utilize ActiveX.

As a result of frustrations experienced in the process outlined above, I have begun to consider the possibility of reverting to Macromedia’s Director. The limitations of functionality inherent in such an authoring package can be alleviated by using the Lingo scripting language and/or third-party plug-ins. Director files can be played on either the Mac or PC platform. Using Shockwave, an internet-compatible version is only a few mouse-clicks away.

Java and DHTML provide two other potential solutions for the creation of cross-platform multimedia applications. Time and experimentation will reveal whether either of these relatively new development environments will allow programmers adequate cross-platform multimedia functionality to create the kinds of software necessary for music instruction.

As evident from the preceding paragraphs, my cross-platform development experience has been a frustrating one. As a scientist and researcher, I believe it is important to communicate one’s successes and failures. Unfortunately, to date, my experience in the realm of cross-platform multimedia development has been one of the latter. I have not yet given up and, in the not-too-distant future, I hope to report a successful solution to this cross-platform dilemma.

_______________________________

1. The focus on only these two platforms is not in any way meant to downplay the significance or validity of those who are developing software for use on UNIX or other alternative platforms. The present paper deals specifically with Macs and PCs because those are the two platforms used most widely in Music Departments.

2. Lingo is the scripting language developed for use with Macromedia¹s Director.

3. The term "object-based programming" is used instead of "object-oriented programming," because Visual Basic does not meet the strict criteria for OOP. Though encapsulation can be accomplished, inheritance and polymorphism are not yet fully integrated.

4. This code assumes that the variables "BLUE" and "WHITE" are defined elsewhere in the application and that numeric values are assigned to represent the appropriate colors.