Nada Dabbagh, George Mason University, United States
Linda Burton, George Mason University, United States
Abstract
This paper will discuss the design, development, implementation, and evaluation of a graduate level course aimed at teaching the concepts and principles underlying Web-Based Instruction (WBI). The overall course goal was to provide an introduction to the principles of WBI and its role in creating learning environments that utilize the attributes and resources of the Internet and the World Wide Web. Particular emphasis was placed on pedagogical, technological, organizational, institutional, and ethical issues related to the design, development, and delivery of WBI.
The course was evaluated on two criteria: the ability of the students in achieving the course objectives and the effectiveness of the course structure in supporting the students towards that purpose. Assessment of students on the instructional activities outlined above served as evaluation instruments for learner achievement. The refection statement, a standard end-of-semester course evaluation survey, student comments, and anecdotes recorded by a peer observer who attended every class and participated in all of the instructional activities served as instruments for course structure evaluation. The evaluation revealed several important results leading to the identification of successful and unsuccessful instructional activities and the level of technological support required to aid students in implementing these activities. Key features in designing courses aimed at teaching about WBI and how to effectively design a WBI prototype were also revealed. These findings and their implications on the design of similar courses will be discussed in the full paper and during the presentation if this proposal is accepted.
The Design, Development, Implementation, and Evaluation of a Graduate Level Course for Teaching Web-Based Instruction
Introduction
This paper will discuss the design, development, implementation, and evaluation of a graduate level course aimed at teaching the concepts and principles underlying Web-Based Instruction (WBI). The overall course goal was to provide an introduction to the principles of WBI and its role in creating learning environments that utilize the attributes and resources of the Internet and the World Wide Web. Particular emphasis was placed on pedagogical, technological, organizational, institutional, and ethical issues related to the design, development, and delivery of WBI. The process of designing, developing and implementing this course was evaluated using a variety of qualitative and quantitative measures. These measures revealed important findings resulting in suggested guidelines for teaching WBI in post secondary education curricula.
Theoretical Framework
The move to online technologies is an evolving and dynamic process of investigation particularly in the area of examining the ability of such technologies to improve the teaching and learning process. In the case of WBI, several frameworks have been suggested to guide the design, delivery, implementation, management, and evaluation of courses that utilize online technologies. One such framework, which can be used to analyze and describe educational Web-Based materials is a three-dimensional framework proposed by Bannan and Milheim (1997). The first dimension of this framework describes the overall design characteristics of a Web-based course in terms of its instructional model (e.g. self-contained course or adjunct to classroom instruction) and its pedagogical philosophy (e.g. instructivist or constructivist). The second dimension describes the instructional methods and strategies used to facilitate learning and to deliver the course content via the Web. The third dimension describes the instructional activities that are mediated through the components (technological tools) of the medium such as electronic mail, Web page creation, posting projects to the Web, and computer conferencing.
Another framework that can be used to guide the design, development, and evaluation of any instructional program that utilizes the attributes and resources of the Internet and the World Wide Web is the Web-Based Learning (WBL) framework proposed by Khan (1998). It consists of eight dimensions: pedagogical, technological, institutional, ethical, interface design, resource support, online support, and course management. Each dimension has various items addressing its constituents in relation to Web-based learning environments. For example, the pedagogical dimension addresses issues such as instructional goals and objectives, the overall design approach of the course, content sequencing and organization, instructional methods and strategies, the instructional medium (the Web in this case) and the extent to which the course uses its attributes, accuracy of subject matter, and learner assessment. The interface design dimension addresses issues pertaining to the "creation of a seamless integration of content and control" through the physical layout of the information and the conceptualization and implementation of the navigation of the instructional program (Jones & Farquhar, 1997). The management dimension refers to the administration, maintenance, and operation of Web-based courses which includes security issues, updating of course information, provision of course evaluation, tracking of student progress, and feedback on student progress. The ethical dimension addresses issues such as course sensitivity to diverse populations, copyright issues, netiquette, and provision for representation of concepts for a worldwide audience. The resource support dimension addresses issues related to various online and offline learning resources which include archives, bibliographies of related books/journals, searchable glossaries, class distribution lists, and other resources related to WBI. The technological dimension addresses issues related to hardware, software, and course infrastructure. The institutional dimension addresses issues related to institutional policy in delivering WBI. And, the online support dimension addresses issues related to the provision of online resources that aid the learner in instruction or serve as information facilitators if students encounter unanticipated technical problems or questions on the course content. For more details about this framework and its dimensions see: http://www.gwu.edu/~etlkhan/keynote/dimensions/framework1.html.
According to Khan (1997) ‘WBI is a hypermedia-based instructional program which utilizes the attributes and resources of the World Wide Web to create a meaningful learning environment where learning is fostered and supported’ (p. 6). Depending on how those attributes and resources are used in a course by the instructor and the learners, the "pedagogical philosophy" underlying the teaching and learning process can range from a strict instructivist approach to a radical constructivist approach (Reeves & Reeves, 1997). A strict instructivist approach typically results in a Web-based course that has a tutorial structure in which the content is organized by the instructor and delivered or imparted to the students; and a radical constructivist approach typically results in a more learner-centered pedagogy where students use Web features as tools to construct their own knowledge representations by restructuring content information and creating and contributing their own resources to the course structure (Bannan & Milheim, 1997; Reeves & Reeves, 1997).
Conceptualizing this course was based on the above theoretical underpinnings in an effort to expose students to current frameworks utilized in guiding the design, development, implementation, and evaluation of WBI and to instructional methods, strategies, and activities that could influence the pedagogical philosophy of the resulting learning environment in the process of applying such frameworks.
Course Design
The course addressed in this paper was designed for students enrolled in a graduate degree program in Instructional Technology (IT) at Towson University. Typically these students are employed full time and attend the IT program on a part time basis, taking anywhere from 2 to 4 years to complete their degree. The students represent a variety of teachers, multimedia developers, instructional designers, and technology coordinators. The course was intended as an elective to be taken towards the end of their coursework to insure entry skills in ‘Instructional Design and Development’ (IDD) or a working knowledge of ‘Integrating Technology into Instruction’. Knowledge in basic desktop computing applications and Internet applications was also assumed.
To begin developing this course, a search for similar efforts at other institutions was essential. Two colleagues, one at George Mason University (GMU) and another at George Washington University (GW) had begun such efforts and a three-way communication paved the way to the sharing of work in progress. Selecting the text was almost predetermined as our colleague at George Washington University had edited a book on Web-Based Instruction which emphasized pedagogical, technological, and institutional issues related to the design and development of WBI. The text was therefore a very appropriate choice for this course given the theoretical framework outlined above.
The next step was to develop the course objectives. Having taught an undergraduate course entitled ‘Using Information Effectively’ which conveyed to students how to effectively use the Internet and the World Wide Web as a research tool, a communication tool, and a collaborative learning tool; the first objectives’ draft was formulated using the syllabus of this course as a guideline. The draft was then shared with our colleagues at GMU and GW and the process was reciprocated resulting in several iterations of preliminary course goals and objectives.
Furthermore, an informal task analysis on the emerging content of WBI and associated research revealed factors that should be included in the course content as well as additional information on specific learning outcomes that students enrolled in such a course are expected to be able to demonstrate. The analysis was facilitated through a comprehensive Internet search, an ERIC search, a review of relevant resources the authors had gathered from previous distance education classes and materials collected at related conferences. Selected relevant materials from this analysis served as the bibliography for the course (see research bibliography).
A final draft emerged with the following objectives:
Upon the completion of this course, students will be able to:
In conclusion, although this course was about WBI, the use of the Web and the Internet in delivering the content and implementing learning activities was supplementary to in-class instruction. The class met on a weekly basis for the duration of the semester and all Web-related and online activities were adjunct to this traditional classroom format. This is known as Web-supported or Web-enhanced instruction. The classroom however was equipped with state of the art computers with Internet connections and Web development software to support the nature of the course objectives. Each class session usually began with a brief discussion of the readings for the week followed by an online quiz and a presentation by the instructor on a concept in WBI. The second half of the class was devoted to teamwork on a group project and other related assignments whose details are discussed below. A total of nine students enrolled in the class.
Course Development and Implementation
Several instructional activities were developed to support the objectives outlined above. Weekly online discussions aimed at facilitating students’ articulation of their understanding of issues discussed in the readings were conducted through a class listserv. Each student was required to post two questions to the listserv related to the topic of the week, respond to another student’s questions, and provide feedback on responses to their questions by others. The questions and validated responses were used to construct weekly online quizzes administered during class time. A brief discussion guided by the instructor preceded the administration of the online quiz each week allowing students to further elaborate on issues that were in need of clarification. Presentations by the instructor and guest speakers on related topics also supported this process. At the end of the semester, a final exam assembled using questions from the quizzes was administered. Students were also required to submit a two page reflective statement discussing their viewpoint on the effectiveness of the online discussions, online quizzes, and online resources in enhancing their learning of the concepts articulated in the readings.
The main percentage of the course grade however was assessed through the design and development of a WBI prototype on an instructional topic of their choice. This was a group effort. Prior to commencing this project, students were asked to evaluate existing online courses using the Web-Based Learning (WBL) framework mentioned above. Having completed this activity, the students were a step closer to understanding the issues involved in designing an online course and proceeded to discuss these issues within their groups. The group assignments were based on mutual interests in content domains which students had communicated to the instructor via a class survey administered at the beginning of the course. One of the questions on the survey specifically asked each student to select an area of interest for which they would like to develop a WBI prototype. After the groups were identified, subject matter experts in the designated content areas were assigned to each of the groups. The purpose of the subject matter expert was to support students in content design and to provide an expert review of the prototype as formative evaluation.
The project was implemented in three phases. The first phase consisted of submitting a project proposal based on specific guidelines supplied by the instructor. The guidelines were based on Ritchie and Hoffman’s (1997) interpretation of how instructional design principles can be incorporated when designing instruction for WWW delivery. Further elaboration by the authors on these rudimentary guidelines yielded the following seven instructional design elements that guided students in developing their project proposal:
The second phase was a design document, which would serve as a blueprint for the actual prototype. The design document was a detailed elaboration of their project proposal with the added task of arranging the instructional content and activities based on the Instructional Framework for Web-Based Instruction (Milheim, Bannan, and Harvey, 1997). This framework identifies a continuum of WWW instructional activities in which lower levels are subsumed in higher levels resulting in increasing levels of interactivity. The levels range from information delivery (level 1) to information synthesis and creation of resources by students (level 5), to the design of immersive collaborative environments (level 6). Students in this course were only required to develop a prototype that incorporated levels 1, 2, and 3 of the framework. Level 1 included dissemination of basic information to learners such as syllabus, class notes, assignments, etc., allowing for such material to be accessible locally or at a distance. For level 2, students were required to provide pre-defined links to outside sites that related to the topic of instruction, and to specify relevant instructional activities to be carried out at these sites. Finally, for level 3, students were required to provide online interaction for learners utilizing one or more forms of synchronous and asynchronous computer mediated communication.
The final phase of the project involved developing the prototype using a Web development tool (FrontPage was used in this course), adding the necessary Web features to make it functional, and evaluating it using formative evaluation techniques. The instructor suggested the following Web-based components that would enhance interactivity as required by level 3 of the Instructional Framework:
The prototype was evaluated on its interface design, usability, and instructional design and delivery components. In terms of interface design and usability, issues such as navigation structure, link structure, visual appeal, and accuracy were assessed. For the instructional design and delivery component, the prototype was evaluated on the strength and cohesiveness of the relationship between learning outcomes, instructional/learning activities, and learner assessment techniques. This relationship was based on the overall pedagogy or instructional strategy adopted for the design of instruction which was addressed by the students in the project proposal phase.
Course Evaluation
The course was evaluated on two criteria: the perceived ability of the students in achieving the course objectives and the effectiveness of the course organization and structure in supporting the students towards that purpose. Assessment of students on the instructional activities outlined above served as evaluation instruments for learner achievement. The refection statement (which was also an instructional activity), a standard end-of-semester course evaluation survey, and student comments and anecdotes recorded by a peer observer served as instruments for course structure evaluation. The participant observer attended every class and participated in all of the instructional activities.
The evaluation revealed several important results leading to the identification of successful and unsuccessful instructional activities and the level of technological support required to aid students in implementing these activities. Key features in designing courses aimed at teaching about WBI and how to effectively design a WBI prototype were also revealed. These findings and their implications on the design of similar courses are discussed below.
Methodology and Data Sources
As mentioned above, the course was evaluated on two criteria: perceived learner achievement and course organization and structure. For the purpose of this case study, perceived learner achievement will be assessed primarily through students’ reflection statements. The reflection statement required each student to describe his/her experiences with the online components used in the course, mainly the online quizzes, online listserv, and online resources. Students were asked to discuss their experiences with these course components in terms of their effectiveness in enhancing their learning. Students were also required to reflect on the WBI models and frameworks that they learned about in the readings and to select a preferred model, provide adequate reasoning for selecting such a model, or suggest their own model if they found fault with other models. Therefore we felt that the content of the reflection statement was sufficient in providing information that demonstrates perceived learner achievement. In the following sections we will discuss the methods used in evaluating the course organization and structure which include the reflection statement. We will then summarize our findings and present our conclusions.
Course Structure Evaluation Methods
The methods used in evaluating the course organization and structure were comprised of observation and participation of a faculty member who helped conceptualize this course, student reflection statements, and an end-semester course assessment tool.
Participant observation
The faculty member who attended every class and was a full participant in all class activities recorded her observations on specific instructional strategies as outlined below.
Presentations on Special Topics
‘Instructor presentations frequently included examples to illustrate the concepts under study. On several occasions outside experts were brought in to present and to answer students questions. One session that was particularly interesting and helpful was when the instructor arranged for the author of the course text to come and meet with the class. The author’s explanations made his framework more meaningful. We had used his framework to evaluate existing online courses prior to starting our own WBI prototype.’
‘Another session that was particularly helpful was when a professor who was an expert on WebCT (a Web-development tool for putting courses online) was invited to the class to explain how he had used this tool in an introductory Computers and Creativity class to conduct online activities and present the course content. He shared some insightful findings about the features of this tool that were very helpful in the development of our prototype.’
Evaluating existing online courses
‘An excellent learning experience was looking at various online courses on the Web to see examples of the types of courses that were available online and actually being used. Students were given lists of several existing online classes and were asked to review these courses based on a Web-Based Learning (WBL) framework developed by Khan (1998). Aspects of this framework were presented and discussed and students were required to compare and contrast the features of the online courses in relation to the dimensions of this framework. This was an interesting and informative assignment, which took several weeks to complete. It allowed students to look more critically at specific elements of the online courses such as interface design, perceived interactivity, instructional strategy, online support, institutional support, management support and technological issues.’
Teamwork
‘Teamwork was a major part of the course. The group experience was interesting and challenging. The instructor surveyed students in the class about their background and interests and suggested the group makeup. My group was composed of three students. Like many adult learners taking college courses, times available for group meetings outside the class was limited since all members had the equivalent of full time jobs in addition to taking the course. The only time that our schedules permitted out of class meetings was a half-hour before the scheduled class time.’
‘At our first meeting we briefly brainstormed ideas to determine an instructional topic for the WBI prototype. One member was interested in developing a Web site to support people who are in grief due to a crisis situation since she had reviewed a similar Web site and had some experience in that area. The other, an elementary school teacher was interested in a site that teaches young children how to discriminate between animal sounds, and I suggested something related to computer or Internet basics since there was a definite need for online education in that area as well as it being a topic in which we all had a basic knowledge. After two more meetings that included much discussion of these as well as other ideas and investigation of other instructional Web sites, we chose computer basics as the topic for our instructional Web site.’
‘In subsequent meetings we determined how to go about organizing this Web site. Each group member had ideas that were usually compatible with what others were thinking. These ideas were largely based on what we gleaned from the online courses that were evaluated earlier in the semester. When our ideas differed, we discussed and negotiated until we came to a consensus or agreeable solution.’
‘At times, group members disagreed on interface design issues but were able to resolve these issues based on suggestions from our subject matter expert and the readings. As group work evolved, email became more frequent and was important in determining each group member’s tasks. At times group members were frustrated by the slow response of other members that was usually due to limited access, technical problems, lack of time, or members just not regularly checking email.’
‘Throughout the group experience, the instructor met periodically with groups. The instructor gave very detailed feedback in the later development stages of the prototype which was extremely helpful.’
Online Listserv
‘The instructor had set up the listserv before the class began and during the first class session the online expectations and protocols were explained. Overall, the established listserv protocols worked well even though it did take a few weeks for all to become accustomed to them. The instructor assisted by reminding students either via email or in class of the protocols when she observed that students were not following them.’
‘This experience required going online three or four times a week. This would not be a problem for full time graduate students who are often on campus, but since most students in the class had full time jobs or worked part time and took several classes; finding the time to get online at the appropriate times took some planning on the student’s part. Calling in from remote locations to the campus computer often took longer than necessary. However, even with these circumstances, most students met the requirements. If requirements were not met, the instructor checked with students either before the class sessions or in the next class session as to what had occurred. On several occasions, students thought they had posted to the listserv, but for some reason, questions had not gone through.’
‘Overall posting to the listserv was very helpful as a learning tool as it forced me to look at assigned material early in the week even though at times I was less enthused about participating than at others. However, it just became one of the things that I knew I was to do early in the week. Although this activity did encourage students to at least look at the assigned chapters early in the week, I frequently found myself just perusing the chapters in search for good question material as opposed to studying the concepts. Some chapters seemed not to have material that was worthy of asking intelligent or higher level questions, but most contained significant concepts.’
Answering Others’ Questions
‘I found that trying to find the answers did help me learn the material for in looking for the answers I frequently reviewed material I had looked at before when I was attempting to compose a question. Additionally, I often encountered material that I found interesting and would get sidetracked reading that material. Sometimes, I would have to go online several times to find an unanswered question. In answering questions, when several were available, I often found myself choosing easier questions, or ones I already knew the answer to rather than ones I had to spend more time seeking the answer. However, on several occasions I enjoyed engaging in discussions centering around the question and posting my opinions in reaction to or about questions.’
‘For the most part, students composed questions that were significant in learning major frameworks or concepts in WBI. I was impressed that most questions were well designed and indicated that the student had reflected on the content and put thought into the development of the question.’
Validating Answers to Question Asked
‘Validating others questions was usually fairly simple. It was just a matter of reading the answer and usually sending a comment to the person answering the question such as "This is exactly what wanted!" or "Your answers are correct. Great job!" Occasionally a question would be answered either partially or totally incorrect and the correct answer with explanation needed to be given.’
Reviewing Others’ Questions
‘Reviewing and studying other students questions gave me a perspective of what others students in the class perceived to be significant content. I found reading the questions to be helpful in learning the material and sometimes they would motivate me to go to the chapters to find other related material. For example, if a student asked a question that requested naming and describing four of the six factors of a concept or framework, after reading the answer given, I would go to the chapter to find the factors that were not named in the answers.’
Quizzes From Online Questions
‘During the first several weeks of class I felt that the online quiz was anxiety producing and did not create a positive learning experience especially the first 20 minutes of class time. In other words, instead of going to the class looking forward to an enjoyable learning experience, I went in just hoping that I could remember all the information I had reviewed from the listserv questions.’
‘At times I felt that the quizzes took more class time than the learning that resulted from the overall experience. Even though the quizzes were short, they seemed to take about twenty minutes of class time each week. A possible alternative would be a short discussion on the material each week and quizzes given less often such as every two or three weeks.’
‘Sometimes I felt that the rewards of the online learning experience were not worthy of the time required for the activity. However, in my above analysis I probably am not taking into consideration the rewards that the experience provided in learning about the value of online communication in the learning process.’
Student Reflection Statements
As stated earlier, students were asked to write a two-page reflection statement addressing the following questions:
Student responses to question two of the reflection
statement will be discussed only to the extent that they bear relevance
to the evaluation of the course organization and structure. To this effect
these responses will be used to demonstrate ‘how much students learned’
which is an item in the end-semester course evaluation that the students
were required to address. It is also important to mention here that after
a couple of class sessions a new activity emerged to complement the listserv
discussions. Students expressed the need to engage in an in-class discussion
prior to taking the online quiz. They felt that posting, responding, and
validating questions using the listserv was not sufficient in preparing
them for the online quiz. As a result, an in-class discussion about these
questions and their answers was facilitated by the instructor at the beginning
of every class session. Students also reflected on this activity in their
statements. Below are excerpts from these statements on each of the online
activities as well the in-class discussion.
Listserv Discussion
‘I liked the idea of generating our own questions about the topics we read and having our classmates answer them. In this way we could bring ideas that we felt were interesting or important about the subject to the group and we could see what others had to say through their answers.’
‘I did not feel that these question and answer sessions could be considered a true discussion. I don’t recall there being any third party involvement in any of the questions/answers posted.’
‘The idea of selecting and posting two questions which typify the content of the article(s) was also helpful. Selecting important questions is a type of synthesis-thinking. A reader who doesn’t think synthetically is not reading actively and will not retain much of what is read.’
‘One thing that I find unusual is its very nature of being asynchronous. Often students misunderstand each other because they have long forgotten what other students’ arguments were before they hit on the next topic. I would rather see this discussion carried out synchronously on a chat line as the class size is quite small. This will certainly bring about other problems in practice as most students are studying part-time while engaged in their professional jobs full time however it might be a more effective learning experience.’
‘I found the listserv discussion to be the most enriching … It was very interesting to see the different questions that each of us developed. I also enjoyed the variety of topics each of us covered. It was an interesting way to get a more in-depth look into a point of the chapter that I may not have been drawn to.’
‘Early in the semester it was very demanding to keep up with the reading and post questions, answers, and verifications. It was a constant pressure on the mind that you needed to check the listserv to see who posted what so you can respond and who answered your question so you can verify it.’
‘A lot of the questions were rather simple and did not require higher-order thinking skills. People were intentionally looking for easy questions because they knew the questions would be on the quizzes and then on the final exam. It is also very easy to compose and answer a question without actually reading the whole chapter.’
‘I think I would like to have more discussion on the listserv. Just posting questions did not seem to really use the best part of the listserv to its best ability.’
In-class Discussions
‘The in-class discussions which evolved after several weeks of readings was a positive thing. Giving the class an opportunity to voice opinions regarding aspects of the readings promotes active learning. Voicing one’s thoughts helps to codify and structure them. Also, hearing others’ viewpoints causes re-examination of one’s own ideas–always a healthy activity.’
‘Eventually we began discussing the questions and ideas before each quiz in class. I felt these discussions enhanced what we did throughout the week, while the quizzes did not offer the same enrichment.’
‘I found the in-class discussion to be very helpful and enlightening. It should be moved partially to an online discussion form.’
Online Quizzes
‘Rather than on-line quizzes, I would have preferred to have been graded on the quality of my contributions to the in-class discussion which happened prior to taking the online quiz. I think that such an approach would have caused all the students to examine and review the articles more thoroughly–and learn more from them.’
‘I found the multiple-choice questions easier to answer online than the essay type. There is so much typing to do and as a student, I would rather write on paper when under the pressure of taking a quiz.’
‘The quizzes, although certainly an evaluation tool, I do not believe enhanced my learning. Since they were the exact questions we developed during the week through our listserv, I did not feel that the quizzes tested our learning and comprehension of the information.’
‘It became difficult after a while to keep the facts straight with all the different frameworks and models that we were reading about therefore the quizzes were not very effective in helping us discriminate between the many concepts underlying these frameworks.’
Online Resources
‘I made very little use of these resources, accessing them only two or three times during the term. Whenever I needed the syllabus, I was able to reference the paper copy to obtain the information that I needed. The instructional resources available from the course home page were of minimal use. I saw no compelling reason to view, for example, the PowerPoint presentation on the Instructional Framework.’
‘The online resources page had a few items for us to investigate. One of the resources offered our WBL dimension reviews. I thought this was a great idea if others posted as well.’
‘Other than at the very beginning, I seldom accessed the pages except to link to Khan’s framework or the Levels of Interactivity.’
‘An idea for this page is to include a bulletin board posting section where we as students could post site findings and other resources. Going back to our diverse experiences, I know we have discovered a great variety of resources and this would be a wonderful common area to share them.’
‘Overall, I learned a lot by browsing through the instructional resource pages to utilize and locate educational resources and discriminate between educational and non-educational sites.’
‘It is very good to have both options, online information and hardcopy by printing online information.’
End-semester Course Evaluation
This was a standard quantitative assessment based on a scale of 1-5 that rates the instructor as well as the course on several criteria. Since this case study is about evaluating the course organization and structure, only items related to the course will be discussed.
Students were required to rate the course in terms of:
‘The Event-Oriented Design (EOD) model proposed by Thomas M. Welsh in chapter 19 of Khan’s WBI book (1997) attracted me more than the other formal models and frameworks mentioned in the readings. The EOD model was down-to-earth and proscriptive, rather than an abstract statement of certain theoretical notions. There were actual steps described for implementing the model. By far, though, the most attractive aspect of Welsh’s ideas was that his model made it "a relatively simple matter to enable the same events in a more traditional campus environment." (Page 164) In my current community college situation, the development of on-line courses is proceeding in fits and starts. I think that Welsh’s model will fit that environment well, allowing even for a partial on-line implementation in the context of a regular classroom-based offering. Also, as Welsh points out in his conclusion, I may be required to teach a classroom-based offering of a given course (such as statistics) while developing and making available an on-line version of the same course. The EOD model allows for the flexibility demanded in that situation.’
‘Schrum’s framework is cohesive and easy to comprehend. It includes four major areas for consideration in the design and development of WBI: pedagogical, technological, organizational, and institutional. However aspects like online support, interface design, and resource support should be incorporated or addressed under the technological dimension and aspects concerning management of online instruction should be incorporated under the organizational dimension.’
‘The two frameworks that made the biggest impressions on me were Khan’s WBL framework and Reeves and Reeves interactive learning dimensions. I think Khan’s framework is useful as a checklist for helping authors determine what components and features they may need in their particular WBI program, and reminding them about the necessary, but less important details of information that aid the learners who are taking the course online such as navigational tools. Reeves and Reeves model however came across as being the most down to earth, comprehensible explanation of ways to approach the design and evaluation of WBL. The model does not offer cut and dry solutions as to how to design the best course or module, rather how to determine what type of instructional strategy, what components of the Web, etc., an author could use to design the most appropriate instruction for his/her purposes.’
‘In designing my own model I would include three categories: content or pedagogical, technical, and Web components. In the content or pedagogical category I would include accurate content, course requirements, objectives, and assignments. I would also include ethical issues associated with creating an online course making sure that the information is presented in a clear and unbiased way so as not to offend anyone. Instructors should make sure that proper netiquette is used and that their design approach is clear. This includes varying the material to meet the needs of different learning styles. Students should also be allowed to give anonymous feedback about the site. The technical category would include student support and instructor/facilitator support. The Web components category would include usability issues, interface design, navigation structure, and hyperlink structure and accuracy.’
As demonstrated in the above excerpts, students were able to synthesize their knowledge on WBI and go beyond the information given to provide their own rational for selecting their most-preferred framework or design their own framework for guiding the development of WBI. Although transfer of learning was evident, the overall rating of the course (item 6 on this evaluation form) was unsatisfactory. A positive correlation of this rating with the ratings on items 2 (relationship to the syllabus), 3 (consistency of assignments with course objectives), and 4 (text and/or class materials) was apparent. The rational for this correlation is evident in our discussion of our findings and conclusions below.
Findings & Conclusions
Courses in WBI are at the forefront of every higher education institution and the need for guidelines on how to design, develop and implement such courses is eminent. This analysis revealed several important findings that formed the basis of a set of guidelines for designing, developing, and implementing a course on WBI. According to the Australian National Training Authority (ANTA), ‘an emerging pattern for the development of online courses is that of a staged and incremental process where the focus for developers is more on understanding the potential of the technology and exploring its features (March, 98).’ This course was designed to communicate to students that WBI involves several dimensions other than the technological. In particular, course activities were designed to address a variety of Web-based frameworks and instructional strategies that are integral to the design, development, and delivery of online courses. Evaluating these activities resulted in guidelines that could be used to improve the delivery of courses designed to teach about WBI. The framework used to present these guidelines will be the instructional activities themselves. The evaluation results and the ensuing recommendations for each activity will be discussed below.
Presentation on Special Topics
This was described as a positive learning experience from the standpoint of the participant observer. Bringing in experts to present on special topics helps contextualize learning and allows students to interact with the experts meaningfully. It is therefore strongly suggested that expert modeling and interaction with experts on relevant topics be included in the design of such courses. Expert modeling and interaction with experts may be conducted in face-to-face settings if feasible or may be incorporated as an online activity through some form of Computer Mediated Communication.
Evaluating Existing Online Courses
This activity was described as an excellent learning experience by the participant observer and was also discussed by students in their reflection statements as being instrumental in guiding the design and development of their WBI prototypes. The activity required that students use the WBL framework to evaluate existing online courses and synthesize their findings in a report which was posted on the Web. In doing so, students were engaged in critical thinking skills. Posting the reports on the Web also provided examples that can be shared amongst students. These examples can be used for modeling, discussion, or review. Based on these findings it is suggested that engaging students in higher order thinking skills through the evaluation and synthesis of existing online courses and modeling these efforts by posting their work to the Web is critical to enhancing their learning of WBI.
Teamwork
Teamwork was a fundamental component of this course and was evaluated by the participant observer. It was noted that graduate students frequently have many time consuming obligations such as work and family which makes scheduling outside of class group meeting times for assigned group projects very difficult. Students therefore should be encouraged to use electronic communications for group meetings as much as possible. It was also suggested that the instructor should be copied on group communications and should take a larger role in facilitating and monitoring group activities. It was also noted that graduate students often have differing backgrounds and areas of interests and that getting group members to agree on project topics is difficult. Offering a list of pre-selected topics for students to choose from may expedite this process.
Online listserv
Posting, responding and validating questions through an online listserv was also a fundamental component of this course, the objective being to provide students with a means of interacting with the reading material and enhancing group communication. This activity was evaluated by the participant observer as well as the students through their reflection statements. Although most agreed that generating questions was a reflective activity which engaged students in higher level processing of the material an provided multiple perspectives of the content, the results were a mix of positive and negative comments. The following suggestions were noted to improve this activity:
Although this was not a planned but rather an emerging activity, the consensus was that it was a very positive learning experience which further enhanced the knowledge of concepts articulated in the readings and helped reduce the anxiety that preceded taking the online quiz. Students were able to ask questions about the readings and explain to others how they generated their quiz questions and what they were looking for as responses. This provided the comfort zone that was lacking on the online listserv. The absence of a ‘third party’ intervention on the online listserv as one student put it created an anxiety level that negatively impacted the learning process. It is strongly suggested therefore that an open discussion forum be used to complement a prescribed activity and that the discussion be facilitated by the instructor to provide the necessary learning guidance for students.
Online Quizzes
Generally the comments on the online quizzes were negative. This activity was perceived as anxiety producing and cumbersome in terms of online requirements. The participant observer noted that quizzes should be given less frequently and at the end of the class after students had been given the opportunity to discuss the material. Students echoed these comments in their reflection statements and added that the online format was limiting due to time constraints and the types of questions asked. Most of the students resorted to multiple-choice type questions which were easier to generate for the online listserv but difficult to answer due to their inflexible nature specially when the material became increasingly complex with the many frameworks and concepts presented. The few open-ended questions that were included in the quizzes also presented some difficulties for the students in terms of typing and space limitations. Students did not think that the online quizzes enhanced their learning experience. It is therefore not recommended that this activity be used in a course on WBI. Instead, student generated questions on the readings followed by a discussion (online or in-class) that is facilitated by the instructor was perceived as a more effective learning experience.
Web-Supported Resources
Mixed results were expressed regarding the usefulness of online resources to support the instructional process. It is important to keep in mind in this context that this course was conducted in a traditional format where students met face-to-face on a weekly basis. Had the course been conducted partially or fully online, the authors feel that the usefulness of the online resources would have been more prevalent. Some students did not feel the need to visit these resources unless they were instructed to do so as part of a specific activity. Others enjoyed browsing the many sites provided and most liked the convenience of having access to the syllabus and other course material as needed. One suggestion was that students be responsible for adding to these resources and that a designated area be made available for this purpose. We feel that this is a very important suggestion which enhances generative learning activities by allowing students to contribute to the course content. As discussed above, posting students work on the Web provides a shared resource for the group and an opportunity for students to learn from other students. Finally, we feel that since the instructor also provided print-based copies for most of the class assignments and activities some of the Web resources may have been regarded by the students as redundant.
The World Wide Web does not have the intrinsic ability to teach however it can provide features that are conducive to learning and instruction (Dabbagh, 1998). Educators and instructional designers are able to take advantage of these unique features to create meaningful instructional environments where learning is fostered and supported. Although some Web-based features such as synchronous and asynchronous communication, unfiltered searchability, global accessibility, and hypermedia interactivity are inherent to the medium, the learning activities that they enable are critical to the design of effective instruction and must be grounded in learning theory irrespective of the medium used. Courses in WBI should demonstrate that in order for WBI to be effective, it must be pedagogically driven, dynamically designed, action-oriented, and content specific.
References
Australian National Training Authority (ANTA) (February, 1998). Teaching and Learning Styles that Facilitate Online Learning. [WWW document]. URL http://www.tafe.sa.edu.au/lsrsc/one/natproj/tal/
Bannan, B., & Milheim, W.D. (1997). Existing Web-based instruction courses and their design. In B.H. Khan (Ed.), Web-Based Instruction (pp. 381-387). Englewood Cliffs, NJ: Educational Technology Publications.
Dabbagh, N.H. & Schmitt, J. (1998). Redesigning Instruction through Web-Based Course Authoring Tools. Educational Media International, v35, n2, pp. 106-110, June 98.
Khan, B.H. (1998). A Framework for Web-Based Learning. [WWW document]. URL http://www.gwu.edu/~etlkhan/keynote/dimensions/framework1.html
Jones, M.G. & Farquhar, J.D. (1997). User Interface Design for Web-Based Instruction. In B.H. Khan (Ed.), Web-Based Instruction (pp. 239-244). Englewood Cliffs, NJ: Educational Technology Publications.
Khan, B.H. (1997). Web-based instruction (WBI): What is it and why is it? In B.H. Khan (Ed.), Web-Based Instruction (pp. 5-18). Englewood Cliffs, NJ: Educational Technology Publications.
Reeves, T.C., & Reeves, P.M. (1997). Effective dimensions of interactive learning on the World Wide Web. In B.H. Khan (Ed.), Web-Based Instruction (pp. 59-66). Englewood Cliffs, NJ: Educational Technology Publications.
Ritchie, D.C., & Hoffman, B. (1997). Incorporating Instructional Design Principles with the World Wide Web. In B.H. Khan (Ed.), Web-Based Instruction (pp. 59-66). Englewood Cliffs, NJ: Educational Technology Publications.
Welsh, T.M. (1997). An Event-Oriented Design Model for Web-Based Instruction. In B.H. Khan (Ed.), Web-Based Instruction (pp. 59-66). Englewood Cliffs, NJ: Educational Technology Publications.
Research Bibliography
Berge, Z., L., & Collins, M. P. (Eds.), (1995). Computer mediated communication and the online classroom (Vols. 1-3). Cresskill, NJ:Hampton Press.
Dede, C. (1990). The evolution of distance learning: Technology-mediated interactive learning. Journal of Research on Computing in Education. 22(3), 247-264.
Harasim. L., Hiltz, S. R., Teles, L., & Turoff, M. (l995). Learning networks: A field guide to teaching and learning online. Cambridge, MA:MIT Press.
Institute for Distance Education (1997). University of Maryland System [WWW document]. URL (http://www.umc.edu/ide/ide.html)
Kilby, T. (1996). What is Web-based training? [WWW document]. URL (http://www.clark.net/pub/nractive/alt1.html).
McMannus, T. F.(1996). Delivering instruction on the World Wide Web [WWW document]. URL (http://www.edb.utexas.edu/coe/depts/ci/it/projects/wbi/wbi.htm.).
Moore, M. G., & Kearsley, G. (l996). Distance education: A systems view. Belmont, CA: Wadsworth Publishing Co.
O’Donnell, J.(l997). New Tools for Teaching. URL (http://ccat.sas.upenn.edu/teachdemo)
Reader, W., & Hamond. N. (1994). Computer based tools to support learning from hypertext: Concept mapping tools and beyond. Computers and education, 22(1-2), 99-106.
Shotsberger, P.G.(1996). Instructional issues of the World Wide Web: Exemplars and precautions. Educational Technology, 36(2), 47-51.
Willis, B. (1994). Distance Education: Strategies and Tools. Englewood Cliffs. NJ. Educational Technology Publications.
Yahoo (1997). Online teaching and learning conferences [WWW document]. URL
(http://www.yahoo.com/Education/Instructional_Technology/Online_Teaching_and_Learning/Conferences/)
Yale Center for Advanced Instructional Media. (1997). Web Style Manual. URL (http://info.med.yale.edu/caim/StyleManual_Top.HTML)
Author Information
Nada Dabbagh is an Assistant professor in the Instructional Technology program at George Mason University. She teaches graduate courses in Instructional Design, Web-Based Instruction, Applied Psychology, and Technology Integration. Her main research interests are: the design and evaluation of Web-Based Instruction (WBI), problem generation and representation in Problem-Based Learning (PBL), and the contextualizing of instruction through Constructivist Learning Environments (CLE).
Nada received her doctorate in Instructional Systems Design (INSYS) from The Pennsylvania State University in August of 1996. During her doctoral program she worked for the Applied Research Laboratory at Penn State in the capacity of a senior training specialist, developing interactive, multimedia-based training modules (CBT) for the Warner Robins air force base. Prior to enrolling in the INSYS doctorate program, she was an instructor in Computer Science at The Pennsylvania State University for six years. She taught courses in fundamentals of computing and algorithm development to liberal arts, business, and engineering students.
Nada was recently awarded a technology fellowship in the Center of Instructional Advancement and Technology (CIAT) at Towson University in which she designed and developed a problem-based learning environment that exposes students to the contextual and problem-solving nature of the process of Instructional Design. This research effort is on-going with the goal of building a Web-supported database of problem-based case scenarios to support the teaching and learning of Instructional Design through authentic contexts. Currently, she is working on evaluating the pedagogical effect of traditional courses that have been redesigned for online delivery using Web-based Course Authoring tools. She recently published an article and a book chapter that articulate her findings on this issue.
© Copyright 1999. The authors, Nada Dabbagh and
Linda Burton, assigns to the University of New Brunswick and other educational
and non-profit institutions a non-exclusive license to use this document
for personal use and in courses of instruction provided that the article
is used in full and this copyright statement is reproduced. The authors
also grants a non-exclusive license to the University of New Brunswick
to publish this document in full on the World Wide Web and on CD-ROM and
in printed form with the conference papers, and for the document to be
published on mirrors on the World Wide Web. Any other usage is prohibited
without the express permission of the authors.