mbn3@columbia.edu

Increasing interest in and implementation of WWW based distance learning resources presents the need for suitable design models. A thorough design strategy accounts for the many dimensions of a project. These include goals of the designers, study of intended users, resource development, authoring, assessment, and ongoing implementation review. Furthermore, design strategies require inherent flexibility, accommodating both the unexpected and unimagined.

The Music In The Classroom (MITC) project developed, refined, and implemented a design strategy for the creation of a WWW-based professional development resource site. The site aims at empowering teachers to develop original strategies for the interdisciplinary integration of music into their own classroom curricula. Focus was particularly directed toward educators at the elementary level.

This paper shares my experience with the MITC project. I will offer an overview of the MITC design process as well as some observations intended to serve as baseline ideas for future developers.

Impetus toward creation of this type of resource came from the confluence of three circumstances. First, I wanted to fulfill my dissertation requirements in such a way that the results might provide practical application to classroom practitioners, i.e., it needed to be usable, useful, and effective in the "real-world". Second there is a growing interest in arts-integrated interdisciplinary pedagogy … not just coming from academic circles. Messages posted to relevant newsgroups indicated that, beyond self-motivated interest, some teachers were being told by their administrators that this type of teaching was now expected of them. They didn't know where to start. Third, I am participating as a teaching artist and faculty member of the Creative Arts Laboratory. Based at Columbia University's Teachers College, the Creative Arts Laboratory (CAL) is a professional development project with the purpose of integrating the arts into core curricula.

My own experience, along with CAL evaluations, indicated that the pedagogical approach to integrating music into core curricula as practiced at CAL was viewed by participating teachers as both highly beneficial to students and very challenging to implement. Teachers related that a combination of workshops and the presence of Teaching Artists in their classrooms provided the models that enabled them to develop their own similar music-integrated practice. These ideas formed the core challenge of the MITC project. Could one develop the means to deliver teachers a facsimile of CAL training, given that most of them who were interested or required to begin implementing similar pedagogy would have no opportunity to participate in such a professional development project?

It should be noted immediately that I view a distance learning approach in this instance to be driven by circumstances rather than by an answer to the question, "What is best?". Most teachers don't have the option of participating in CAL or other similar projects. Thus, given the choice between "best" and nothing it seemed valuable to explore creation of an intermediate resource. The goal of this resource would be to enable classroom teachers to develop their own music-integrated practice. This task would be accomplished by providing a comprehensive self-training environment with particular regard to a pedagogical approach practiced in CAL. The interdisciplinary pedagogy advocated is characterized as "process-oriented" and "constructivist," … developing musical skills, concepts, and aesthetic perception through active music exploration. Throughout, the MITC project was guided by the assumption that it is not merely a presence of music in the classroom, but rather, the process by which children engage with music wherein lies the greatest potential for benefit.

Baecker et al. (1995) observe that "effective design involves a variety of skills-conceptual and technical, behavioral and aesthetic" (p.80). Content expertise must be added to the mix (Bednar et al., 1992; Norman, 1986). The designer of a WWW site aimed at self-training must create a scaffold for the learner that is usable, acceptable, and effective in maximizing the potential learning of a specific but diverse user population. Ultimately, design is a series of tradeoffs accounting for the goals of the designers, the users (and their situations), and the technology (Gould, 1988/95; Norman,1986).

At the time the project began in 1996, there was precious little academic literature to guide WWW based design or training. There was, however, a considerable body of literature available addressing both interface and hypertext design. While not web-specific, this material offered web designers a number of informing methods, guidelines, protocols, and assessment strategies. While specifics in approach varied, several threads emerged from my review of interface and hypertext design literature: (a) Users should be accounted for in/throughout the design process (e.g., Bødker, Grønbeck, and Kying, 1995; Gomol, 1990; Norman,1986; Redmond-Pyle & Moore 1995). (b) Iterative design processes should be employed (e.g., research review and papers in Baecker et al., 1995; Flagg, 1990; Neilsen, 1995; Rheingold, 1990). (c) The design process is an ongoing, evolving process (e.g., Gould, 1988/95; Schneiderman, 1992) (d) The design should be based on a theory of learning, not driven by technology (e.g. Bednar et al., 1992; Dick, 1992).

The above threads (generated in part during Phase 1 below) guided creation of the MITC site. A multi-phase design chronology was adopted as follows:

1. Specification of goals, and objectives. Clarification of assumptions and applicable learning theory.
2. Survey of current resources, models, and literature. Development of reference/links library
3. Development of assessment strategies
4. Design outline map (i.e. organizers, nodes, navigation keys)
5. Design/Production Chronology

Reflecting thread "a," ("Users should be accounted for in/throughout the design process"), a user-centered design approach was chosen. There are a range of user-centered design models. These run from taking user input in the form of designers acknowledging validity of users purposes and beliefs (e.g. Redmond-Pyle and Moore,1995) without giving them decision making power, to the "participatory design" models in which users have direct decision making power regarding the design (e.g. Bødker, Grønbeck, and Kying,1995). Given its context, the MITC project took a middle ground. While design decisions were mine alone to make, most of the user ideas suggested (and consistent with the projects goals) during Phases 2 and 4 ultimately appeared in the site. The positive benefits of this approach were revealed in the Phase 4 iterative testing cycles.

An iterative design approach (thread "b") was chosen. The iterative model considers design and evaluation together. Iterative design employs cycles of assessment and revision intended to move the design toward increasingly effective prototypes. It is an ongoing contextual review/revise process that addresses the unpredictability and diversity of interaction between user and design. Gould (1988/95) notes three key requirements for iterative design: (a) "Identification of required changes," (b) "An ability to make the changes," (c) "A willingness to make changes" (p.108)

Thread "c" ("The design process is an ongoing, evolving process") is an acknowledgment that Gould's requirements necessitate ongoing action. Design is dynamic and longitudinal. If the design is to progress in terms of effectiveness it must continuously evolve as do the users, their contexts, technologies, and the designers' ideas. It must be flexible in its ability to react to its own realization. In concrete terms, I changed specifics a number of times in creating the site, either at the insight of a new idea, a technical concern, or a colleague's observation. I followed my intuition when something just didn't seem to work, and was generally rewarded with a host of new material that had not been imagined earlier.

Thread "d" ("The design should be based on a theory of learning, not driven by technology") is consistent with a core assumption stated earlier. Namely, that it is not merely a presence of music in the classroom, but rather, it is the process by which children engage with music wherein lies the greatest potential for benefit. This speaks to a practice-refined pedagogy (and attendant theoretical underpinning). Fundamental to the success of the MITC project was the site's ability to model the approach to teaching that it advocated. Design decisions were constantly triangulated and adjusted for consistency between what the site did and what it asked teachers to do. This notion was viewed as critical not only in terms of communicating ideas, but in promoting acceptability.

User-centered design incorporates the ideas and needs of potential users. In Phase 1 of the MITC project, the purpose of data collection and analysis were to improve site creation and evaluation through exploration of perceptions and ideas of classroom teachers who had participated in CAL. While the specific individuals involved might not be the actual site users, it was assumed that their input, as classroom teachers, would increase the relevancy of the site to other classroom teachers. Interviews were also conducted with the CAL project director and project evaluator. The purpose of the CAL project director and project evaluator interviews was to account for those items that they believed were critical to the practice and effective transmission of the pedagogy. In sum, analyzed data provided information to build, and/or expand design criteria with regard to site content, organization, and materials. Those criteria included specific site resources and capability, along with notions of the pedagogy and how to communicate it.

Data collection (i.e. the interviews) was flexible, using a prepared guide and a semi-structured interview protocol. The guide was developed to both account for project goals and elicit unanticipated ideas. A grounded theory approach to data analysis was employed (Strauss & Corbin, 1990). Initial analysis suggested organizing findings in two formats. "View 1—Connections" was essentially conceptual in nature. "View 2—Nuts and Bolts" was pragmatic, presenting a number of principles in a structure that might parallel a teacher's working mode; namely a progression of background/planning, execution, and evaluation headings. Both views yielded specific materials (quotes, taxonomies, evaluation criteria, etc.) to be included in the site as well as sets of action reccomendations for site creation. The findings contributed directly to the site's organization and resources.(*1)

The importance of this research phase cannot be overstated. It may well be the single most potent guarantor of site efficacy. The shape and tone of the site, as well as those materials that resonated most with users in Phase 4 were directly traceable to the Phase 2 research.

Phase 3 of the MITC project focused on the production of a working prototype site. This prototype represented a combination of elements developed in Phase 1, my immersion in the Phase 2 analysis process, and through insight developed during my experience as a Teaching Artist and Faculty Member with CAL.

Olson and Moran (1996) describe design as "an art that mixes creativity and discipline" (p.272). Schön (1983) notes:

Consistent with these ideas, actual site authoring proceeded with a flexible and reflective attitude. The design outline, map, and ideas generated in Phase1 and Phase 2 were followed, but also yielded to modification and/or the possibility of new ideas occurring in process. Given the relative technical ease (*2) of authoring web pages, "storyboarding" was done as a facet of construction itself. This facilitated "conversation with the situation." User comments in Phase 4 subsequently indicated that the project benefited significantly from materials generated using this approach. As a practical concern, taking well-organized notes, and keeping multiple versions of pages (for direct comparison) proved useful.

Assumptions clarified in the Phase 1 (e.g. user environment and circumstances, available technology, technological sophistication of users) were referenced throughout construction. It was pointed out earlier that design is a series of tradeoffs, balancing the interests and objectives of the creator. Navigation issues—a critical concern—provide an example in this regard. Since conveying a pedagogy (rather than simply providing lesson plans) was the site's primary challenge, the decision was made to create a "path of least resistance" that lead users to and through pedagogy-related areas in the site. "New users" were invited to proceed to certain pages. This attempt to guide (by both overt and subtle means), rather than compel, made concrete the notion of balance with reference to the open navigation goals of hypertext systems.

Iterative design employs cycles of assessment and revision intended to move the design toward increasingly effective prototypes. Baecker et al.(1995) notes that an iterative model considers design and evaluation together. Flagg (1990) refers to formative evaluation. This approach seeks "to inform the decision-making process during the design, production, and implementation stages of an educational program with the purpose of improving the program" (p. 241). As mentioned earlier, Gould (1988/95) notes three key requirements for iterative design: (1) "Identification of required changes," (2) "An ability to make the changes," and (3) "A willingness to make changes" (p.108). Flagg (1990) outlines similar requirements.

Two iterative cycles were conducted in Phase 3 to determine site content and revision. Each cycle yielded a set of action recomendations. Simply put, these addressed what to keep, what to discard, and what to modify in subsequent prototypes. Action recomendations were generated through comparative analysis and triangulation with design goals. These resulted in 2 additional prototypes, the final being judged sufficient for public use.

In Phase 4, the MITC project again relied on an inductive research model. Two broad approaches to data gathering, Connoisseur-based (i.e. expert review) and Decision-oriented (i.e. user testing) studies were employed. As in Phase 2, data analysis employed grounded theory technique. The Connoisseur- reviewers were provided a copy of the site I interviewed them subsequently. Decision-oriented evaluations were conducted in a quiet studio and in my presence. Participants were selected on the basis of their matching the profile of MITC's target audience. Suggesting that participants "think aloud" as they navigated the site, I took notes and recorded user interactions by keeping a video camera trained on the computer screen and recording their comments. A prepared guide, along with questions that arose in observing their interaction with the site, formed the basis of the semi-structured interview (taped) that took place immediately after viewing the site.

Primary elements for organizing the evaluation included acceptability, usability, and efficacy of the MITC site. "Acceptability" concerns whether and/or how evaluation participants foresee themselves using MITC, and whether they might recommend it to others. "Usability" concerns the user-friendliness of the site, i.e., ease of navigation, flexibility, clarity/consistency of organization, learnability, help, and overall cognitive overhead. "Efficacy" refers to two sub-categories. Do participants perceive the MITC site as providing necessary resources for integration of music into core classroom curricula as advocated? Is the MITC site communicating the process-oriented pedagogy techniques at the heart of the design goals?

The MITC project resulted in the creation of a "web-ready" resource site. "Web-ready" refers to a working-model website that has been methodically user-tested and—as indicated by the results of that testing—revised so that it possesses sufficient degrees of acceptability, efficacy and usability to merit public posting on the internet. Iterative testing indicated that the content was successful in terms of MITC project goals. It was able to communicate the essential ideas and techniques informing the process-oriented constructivist pedagogy used in CAL.

Consistent with the notion of user-centered design, the ideas, suggestions, and reactions of participating elementary classroom teachers were central to both creation and revision of the prototypes.

The design process applied to MITC site development was effective. I advocate it as a model. My endorsement of the design process is not, however, without qualification. First, my way of working—reflection in action, informed intuition, playing—may not be for everyone. Things took more time and revision than they might have had I made an exact plan and approached the authoring in production fashion. It would, in other circumstances, require like-minded and patient collaborators. I believe that approaching production in an informed, exploratory way, did lead to ideas that wouldn't otherwise have occurred.

With regard to the actual construction of the site, prior experience with the range of necessary software would be advantageous. I would suggest the designer/author working solo have a substantial library of multimedia resources prepared and categorized by concept, size, and content. Having not always done this, I found myself distracted at points in the process where my attention would have been best applied elsewhere. One can always go back and address specific needs if the resources at hand prove insufficient, but it's more efficient not to have to.

Relating to the challenges confronting solo designer/authors, I have been convinced by my experience that design and creation of a WWW resource of similarly complex goals is best accomplished by a collaborative team with distinct but overlapping areas of expertise. These areas might include graphic design, programming, pedagogy, and content expertise. The user-centered design literature—along with my experience both in this project and as a technology teacher—suggest that the potential for success is greatest when there is a grounded empathy between all parties in a design. While the literature focuses on the relationship between users and design(ers), my feeling is that design team members themselves need to understand each other's professional needs and perspectives.

Flexibility is key and should be inherent to the design model. The relative ease of authoring web pages and saving multiple versions allows designers the luxury of following an intuition. One may easily revert to prior choices. In my experience, however, the products of disciplined exploration will contribute significantly to project results.

Finally, bringing users toward a realization of a project's goals is the designer's task. Technology is the "means," not the "end." Stay clear about both your objectives and assumptions. Consistent with project goals, it is the users whose ideas, perceptions, and learning styles must guide design decisions. Find out what they need, what they believe, and how they interact with your creation.

(*1)Amplification of the MITC project's data collection and analysis procedures and general findings can be found in Nord (1998). With regard to qualitative research methods in general, Strauss & Corbin (1990), Glesne and Peshkin (1992), and Maykut & Morehouse (1994) may serve as useful resources.