WebCT in the Classroom: A Student View

Creighton University is a private, independent, Jesuit, Catholic coeducational university of three colleges, five professional schools, and a graduate school. It is Nebraska's largest private university, and enrolls more than 40 percent of all students attending independent higher education in the state. As the work and personal commitments placed upon our students created diverse demands on their time, the need for a flexible approach to study and course participation, tailored to the unique needs of all students, was evident. WebCT (developed by the University of British Columbia to provide a Web-based set of Course Tools) was chosen to provide online learning capabilities to meet this new demand within Creighton's educational environment.

WebCT has been used as a supporting agent to traditional teaching techniques for the 1997-98 academic year, during which it was used within the structure of nearly 20 courses (enrolling a total of more than 400 students) in the English, Theology, Modern Languages, Nursing, Occupational Therapy, and Business programs. Concerned with the impact WebCT might have within a traditionally oriented liberal arts teaching environment, the authors explored the views of the students (rather than the faculty and system administrators) who were required to use the WebCT tools in their classes. The responses of the students in these classes, as collected using a survey approach, are being assessed to determine student reaction to the use of each applicable tool, and Web CT's contribution to the overall learning experience

Preliminary conclusions as to the effectiveness of the tools, based only upon the results of the first semester of use, show the following:

• 84% of the students surveyed recommend continued use of WebCT,
• 75% recommend WebCT for other classes,
• 92% feel it is important for them to have the experience of using the latest computer techniques applied to their discipline.

These values, as well as other survey results will be updated based upon inclusion of the survey results from the second semester of WebCT use and will be reported in the final paper. Student recommendations will also be discussed at that time.

Creighton University is a Jesuit, Catholic coeducational university of three colleges, five professional schools, and a graduate school. It is Nebraska's largest private university, and enrolls more than 40 percent of all students attending independent higher education in the state. U.S. News & World Report has ranked Creighton among the top colleges and universities the past nine years, #1 in its class of mid-western academic institutions in l997and 1998. It is committed to excellence in its selected undergraduate, graduate and professional programs. In its stated mission, Creighton exists for students and learning. Creighton faculty members conduct research to enhance teaching, to contribute to the betterment of society, and to discover knowledge. Faculty and staff stimulate critical and creative thinking and provide ethical perspectives for dealing with an increasingly complex world.

It is with these values in mind that Creighton faculty embark on technology-assisted teaching. Work and personal commitments often create many demands on our students' time. Students increasingly desire flexibility in modes of education. Virtual classrooms, as adjuncts to the traditional classroom, can provide such flexibility; they allow teachers and students to interact from any place, at any time, and with additional resources as support.

This paper presents the findings of our first-year study of the incorporation of a program thought to provide flexibility into courses at Creighton University. "What are the educational benefits and burdens to the student of such a virtual classroom adjunct?" is the question motivating our study. Hence, the study examines only student reactions. Faculty experiences with the use of an integrated program of computer tools, while an increasingly important concern, are not included here.

Part One: Preliminary Matters

The Tool: WebCT

WebCT ( http://homebrew.cs.ubc.ca/webct/ ) is one of several emerging integrated learning programs (e.g. First-Class, Lotus Notes/Domino, WCB/Web Course in a Box, Top Class, Virtual-U, and Learning Space). WebCT was developed in the Department of Computer Science at the University of British Columbia. Dr. Murray Goldberg (goldberg@webct.com ) is its primary designer.

WebCT provides a secured location where faculty can place course materials, including syllabi, assignments, lectures, and presentations. It integrates communication tools, including a bulletin board, chat room, private e-mail, and calendar in one place (a course website), to facilitate interaction between faculty and students. It provides instructional tools to support course content such as a glossary, references, self-test, and quiz module. Students, too, can place assignments and other materials in WebCT in courses for which they are registered. WebCT gives faculty the course management tools for grading, tracking student interaction and monitoring class progress.

WebCT is platform independent. Students can access their WebCT course materials using a web browser (Netscape and Internet Explorer are favored) from any computer connected to the campus intranet or the Internet. WebCT is available for UNIX and Windows NT.

The best source for information about the limitations of WebCT is the WebCT-sponsored user list-serv. On a regular basis, faculty, information systems staff, and administrators who are considering its adoption ask the users about its implementation. In the last year, evaluations of the product have occurred on several campuses; the results of these evaluations are made freely available to such inquirers. One of the most criticized aspects of WebCT is that it does not yet provide whiteboarding or teleconferencing within its system. Additionally, it encourages faculty to create course materials that are used primarily on-line. A compile tool allows students to compress and down-load files but emphasis is not placed, in the design of a course website, on this feature. Hence, some students who must access their WebCT course via costly ISP charges may be considered as somewhat dis-enabled. Currently, there appears to be a desire to have the WebCT development team emphasize the fine-tuning of existing tools and features over introducing new versions with additional tools.

WebCT at Creighton

Until Fall, l997 most faculty created home-grown environments to meet their technological classroom needs. A faculty member (co-author of this paper) discovered WebCT during an intensive academic technology training program (US West Fellowship Program). After a brief survey of like-designed tools and preliminary use of the beta version of the WebCT program in an Environmental Ethics course, the co-author convinced the Vice-President of Information Systems to purchase and install WebCT on one of Creighton University's Unix servers. This installation is updated as new versions are released. He also requested that on-going evaluations of WebCT be conducted.

Use of WebCT by faculty at Creighton is voluntary. WebCT (Version 1.0 and 1.1) was incorporated into courses offered in the Departments of English, History, Modern Languages, Philosophy, Theology, and Occupational Therapy during the 1997-1998 academic year.

Two Distinct Surveys

A WebCT student evaluation survey was constructed by the co-authors of this paper to provide insight into the student's reactions to the introduction of WebCT within their classes; specifically, data was sought in five areas:

• baseline student demographics
• available resources for WebCT
• use of specific WebCT tools
• general impact of WebCT
• continued use of WebCT

Data gathering to support this study was planned to coincide with the end of each of the initial semesters (Fall,1997 and Spring, l998) in which WebCT was used to support classes at Creighton. The survey is a paper-pencil form administered with other evaluation instruments regularly used to evaluate courses. Faculty were highly encouraged but not required to administer the survey to their students. In the Fall of 97, all faculty using WebCT distributed the survey. Of the 199 students enrolled in classes using WebCT, 147 students responded (73.9%). In the Spring of 98, one faculty member chose not to administer the survey; of the 149 remaining students enrolled in classes using WebCT, 84 responded (56%).

Two distinct sets of data resulted. At the end of the Fall 97 term, the survey form and computer-scored response sheets were circulated with a format that supported 7 possible responses to the questions posed to the students, including "Undecided," "Don't know, " and "Does not apply." Seeking similar data for the Spring 98 term, the survey was again distributed, but inadvertent use by the administering unit of a different computer-scored response form limited the students to only 5 possible responses to the questions posed on the survey. Of the three possible responses mentioned above, only the "Undecided" category remained on the Spring computer-scored response form. Consequently, direct comparisons of these response categories between semesters will not be attempted in this study.

Despite this inconsistency the authors found valuable information in the 5 areas mentioned above. This information is reported in Part Two.

Part Two: The Survey Results

Baseline Student Demographics

The two groups studied differ widely in education level (year in college) and computer experience. The Fall 97 student group is characterized by a large percentage of first and second year students (61.2%), while the Spring 98 students in the survey group were predominantly upperclassmen (nearly 80%).

The students present an array of computer experience levels, with the younger students of the Fall group being the more experienced group of users.

Table 1

In both semesters, we found that 60% of the students had no "formal" computer course work. Of those in the Fall group that did have coursework, 62% had taken the course(s) in high school;19% reported that they had taken classes at Creighton. The Spring group's response was more widely-distributed. Of those reporting formal course work, 40% of them had taken the courses in high school, 25% at Creighton, and nearly 19% had taken courses at a community college.

As a means of gaining insight into their experience levels, we asked students if they had taken other courses requiring use of computer technology. We found that in the Fall group, 53% were taking their first course, while 24% reported this was their third or more offering requiring use of computer resources. In the Spring group, 30% reported this as their first computer-related course, while 36% were taking their third or more offering using various computer resources.

Additionally the Fall group of students reported they have computers in their dorms and/or at home; the principal computer they used was in these locations as well. The Spring semester reports the presence and use of computers at home and in Labs as their principal resource. In both groups, the majority of computer expertise comes principally through "non-formal" education avenues.

In summary, we appear to have surveyed two very distinct groups of students. The Fall group was comprised of younger students with more computer training and personally-owned computers. The Spring group primarily consisted of older students, with less formal training, and split in their use of personally owned and institutionally-provided computers.

Available Resources for WebCT

One objective of the study was to determine if our students had sufficient resources available to conduct the required interfaces with WebCT. The survey asked:

• if the students had receive sufficient instruction in the use of WebCT;
• whether the individual computer skills the student brought to the course were adequate to use WebCT; and
• whether there were sufficient computers available to the students for them to complete their assignments and participate in the course structure.

In assessing the responses, we decided to use a threshold of 20% in a response as a "trigger" to alert us to potential areas of significance.The data obtained for this portion of the survey may best be presented as a simple table, below.

Table 2

In summary, the students strongly believe (over 80%) that they received adequate instruction and that their individual computer skills were adequate. At a slightly lower level of agreement (over 70%) they believe their computer resources were adequate. We may conclude from these responses that, for the majority of students, the available instruction and computer resources were adequate, and they believe their existing computer skills were adequate to meet the tasks demanded of them.

Effectiveness of WebCT Tools

We next assessed the effective use of the WebCT embedded tools. To adequately judge whether or not the students felt the tools were "used effectively" we found it judicious to combine responses from some categories, i.e., for purposes of this analysis responses of "Agree" and "Strongly Agree" are grouped, as were "Disagree" and "Strongly Disagree". One immediate finding was that courses were not using every tool. This became evident through the relatively large numbers of responses in the "Does Not Apply" category in the Fall Group, to the point that it became prudent to recalculate the percentages in this area after excluding this category. Recalculated, the remaining responses more accurately reflected the opinions of those students who were indeed required to use the tool in question. The details of the results in this area are included Table 3 at the end of the paper.

Several WebCT tools are considered effective by more than 60% of the students survey. The tools that garner this rating from the students in the Fall group are the Home Page, Bulletin Board, Path Editor, and Quiz. The rankings from the Spring group show only two tools with as high a rating, the Bulletin Board, and Path Editor. Some tools rank lower in the students' eyes (less than 40% agreed the tool in question was used effectively). In the Fall group those tools were the Private Mail, Self Test, and My Notes. The Spring group give the My Notes, Self Test, Glossary, and My Progress tools a lower rating.

General Impact of WebCT

In addition to the specific assessment of WebCT tools, we sought feedback on the general impact of adding WebCT to courses. We constructed questions to assess student opinions about how WebCT affected their time and their learning experience.

The survey data concerning time are fairly straightforward. Over 80% of each group both semesters reported that the time connected to their WebCT course site was four hours or less per week. A very small percent (7%) in the Spring reported that WebCT connection time was 5 hours or more per week.

When asked if the student found the time it took to use WebCT excessive, we find that the data closely correlates with data regarding time spent, although we don't get an exact match. Data from the Fall group, our younger students, shows that 73% believe the time required was not excessive (recall 80% report using WebCT 3 hours or less a week). The Spring group, our older group, presents a similar response profile, but now we drop to 2 out of 3 reporting the time was not excessive.

"Using WebCT in this course increased my course workload" was a statement to which we asked students to react. Over one third of the Fall group and nearly half of the Spring sample felt that the inclusion of WebCT had indeed increased their workload over that of a similar course without WebCT. And while a full 68% of the Fall group say "no", WebCT does not increase their workload, this number drops to just over one third of the Spring group with a similar opinion.

In sum, using WebCT does not seem to place any unwarranted time burden on most students surveyed, nor do students perceive the time spent to be excessive. A significant number of older students do not deny that it places an additional burden of work on them.

Learning

Our survey is designed to elicit responses to a number of general learning issues. We want to know:

• if the inclusion of WebCT played a role in retaining student interest in the course subject-matter
• whether WebCT helped the student learn his/her subject more quickly.
• whether conventional methods of learning are easier than methods used by WebCT (e.g. reading text on-line).

Over half of our Fall group (57%) believe WebCT helps maintain their interest in the subject, but nearly 20% do not agree with this and 20% are undecided. The Spring group reports 44% agreement that WebCT helps maintain their interest, but one in four either disagree or are undecided. In the area of learning pace, nearly half of the Fall group believe WebCT helps them learn the material more quickly, while 22% disagreed and 25 % were undecided. In the Spring group, 41% contend that WebCT helps them learn more quickly; in this case just over 3 in 10 disagreed and roughly 2 in 10 are undecided.

The next area we surveyed addressed the relative ease with which the students profess to learn by reading from a text versus the newer approach of reading the equivalent amount of material as posted on webpages inside their WebCT course. In this case slightly over one third of the younger students felt the text was the easier method, nearly another third disagreed that the text was easier and 1 in 5 could not decide. In the Spring group just over half agreed that the text was easier, while not quite 20% disagreed; 20% could not decide.

In a closely related question, we asked the students if it was easier to learn course material with the conventional method of having all material in a text compared to WebCT-based material. The responses here echoed somewhat those of the previous question. Just over 40% of the Fall group disagreed in this case, 20% agreed and the same percentage could not decide. In the Spring group, 30% disagreed while 2 in 5 agreed that the text was easier; we see 25% who can not decide.

In sum, the responses to questions designed to test student perceptions of the learning efficacy of WebCT are underwhelming. The range of positive response is 30-50% (unlike the clear indicators of 80-95% in earlier questions). And, there appears to be more ambivalence, indecisiveness, or outright disagreement in each group.

Continued Use of WebCT

Our study next addressed the views of the students in terms of continued use of WebCT in the course they were just completing, and their assessment of the use of WebCT for future courses. The Fall group of students give a strong endorsement of WebCT. Over 80% of the Fall group recommended that the course they were in should continue to incorporate WebCT as a means of course enhancement; the support level dips to just over 60% for the Spring students. When it comes to the use of WebCT for future classes, 75% of the Fall group supported the recommendation, and once again the level of support drops, to just over 60%, in the opinion of the students in the Spring group.

Finally, we wanted to know if the students felt they should have experience in using the latest computer techniques applied to their discipline. The Spring group felt this experience was valuable as indicated by the 94% who responded "Yes" to this question. The Fall group gave a 92% endorsement to this final question on the survey.

In this section of our report we succinctly repeat and analyze our most important findings.

Student Demographics

Two distinct and unique samples were contacted with our survey of users of WebCT. The first group surveyed (Fall 97) were comprised of primarily first and second year, computer-literate students (60%); we clearly drew primarily third and fourth year students, with less computer-literacy in the second group (Spring 98). This pattern of differences, displayed in Table 1, above, was unplanned. Although students surveyed were not assumed to be a random sample, nevertheless, we did not set out to compare the younger, more computer literate student with the older, less-computer experienced student. Because the response forms were not identical for the two groups, direct comparisons regarding indecisiveness, ignorance and inapplicability of question content are not possible. Nevertheless, some important comparative results (albeit unsurprisingly) between our younger and older students retain validity.

While the students surveyed are generally familiar with computers, in both groups the majority of them do not have extensive formal computer training. Those that sought such training have done so from sources primarily external to Creighton University. While this is understandable in the case of the Fall group, that the Spring group's formal training traces back to high school or community college resources bears closer examination.

Available Resources for WebCT

Both groups felt the training they were given to use WebCT was sufficient to the task. Since we did not inquire of the faculty what the nature of that training might be we cannot comment further about that training. However, we do know that WebCT requires a number of skills that are not achieved without some instruction. We believe a follow-up survey of our faculty to determine the successful elements of their instruction of WebCT is warranted. The fact that students believed their general computer skills were adequate to completion of the tasks required of them is good news. However, since some of the more sophisticated tools in WebCT apparently were not used by our faculty, we should be cautious in our enthusiasm. At a minimum, we may be finding in this data some truth to the claim that WebCT is a relatively easy program for student use.

The difference among the two groups over the presence of computers in their dorm room also bears closer examination. While both groups report the use of computers at home, only the younger students have predominance of dorm computer use. This finding may bear upon current trends to provide laptop computers to all students. So, too, may the finding that 30-34% of the students surveyed found it irritating to find a computer on campus.

Effectiveness of WebCT Tools

The tools available to the student within the WebCT shell structure are varied. It became obvious to us that we were encountering cases where not all tools were being used in every class. We were able to identify those tools universally deemed "effective" as well as "favorite." Those tools include ones that are central to the WebCT program, i.e. the Path Editor and the Bulletin Board. The ineffectiveness of the Private Mail tool is corroborated by the ambivalency about this tool we read on the WebCT list-serv. Otherwise, we are cautious in drawing conclusions about the efficacy of the WebCT tools, for lack of unambiguous evidence. Our survey instrument and computer scoring forms need to be redesigned to insure that we are gathering data about WebCT tools that are actually used by the student, rather than tools that are being judge ineffective from lack of use by course instructor. Additionally, we want to inquire why so many of the tools available in WebCT were not used. We are careful not draw conclusions based on uncorroborated assumptions at this point.

General Impact of WebCT

Using WebCT in a course is not time consuming, both in fact and in perception. In fact, most students connect to WebCT for 4 hours or less a week. However, time connected to a WebCT course site must not be confused with time examining course webpages found at the WebCT course site. For example, some students will spend very little time on-line if they are prone to print course Webpages and read them at their desks rather then "desktops." A compile tool (which we failed to inquire about) allows the student to download webpages onto their own computers. Because of these possible variables, we must be cautious to not draw unfounded conclusions about the total time used to include the WebCT contents into the overall student study patterns. A revision of the survey will detect these differences.

When it comes to the workload imposed by using WebCT, we find the student opinions somewhat split, seeming to follow a demographic line. Nearly half the older students felt the workload was increased, yet this same proportion of the younger group did not believe this to be the case. This difference may be attributed to varying experience bases in both computer technology as well as course workloads, and, therefore, differing concomitant expectations. This is a subject for more detailed examination in future studies of this area.

The results of the questions about retaining interest and learning pace are underwhelming and, additionally, one might make the case that indecisiveness on the part of our students to a degree not observed previously in this survey may be observed in the data at this point. Although small, it records very near or above our threshold of 20%.

But, in comparing modes of reading (text-based or on-line), we were surprised to find that a full third of our students in both groups said reading the traditional text was NOT easier. We would have thought this percentage would be lower. These data are further enhanced by the finding that only a quarter of the younger students and less than half the older students reported a preference for all course material to come from conventional text-based sources. We surmise that while it is easier for the students surveyed to read a small amount of text from a book rather than the screens, when it comes to the total course, WebCT is indeed a valuable contributor to the overall course success.

Continued Use of WebCT and Information Technology

Students who have experienced WebCT are prone to recommend its continued use either in the course in which they are currently experiencing it, or in future courses. The data varies in strength here between groups and between current and additional courses, but in all cases it stays above 50%.

More dramatic are our students' opinions about the use of the latest computer techniques applied to their discipline. Whether WebCT or some other information technology resource, there is no doubt the students are overwhelmingly in support of the incorporation of this technology by their professors into their academic experience while at Creighton University.

Conclusions

In conclusion, the results of this survey have given us some very preliminary initial impressions regarding the decision to include WebCT as part of the learning environment in Creighton University courses. We believe the results show that WebCT computer aided instruction is:

• Not time-consuming
• Matches students' computer abilities at its level of use
• Contains central tools (Path Editor and Bulletin Board) which are relatively easy for our students to use
• Is perceived by the younger student to help the learning process.
• Supports the perception of both younger and older students that computer-aided instruction is important for success in their future careers.

Implications: Looking Ahead

We can draw some specific implications for Creighton University from the analysis above. It would behoove us to:

• Examine why some of our students are turning to external resources for formal training in computer technology.
• Explore why students experience irritation in finding a computer to use on campus.
• Determine if we have increasing numbers of students bringing computers with them for placement in their dorm rooms. If these numbers are increasing we should compare this data with recent nationwide trends to provide students with laptops upon entry to the University.
• Discover the content of the training in WebCT our faculty give to our students, as it appears to be sufficient for the level of use of WebCT they make.
• Encourage, through faculty development, the more robust use of WebCT by our faculty if it fits their needs.
• Expect our students in first and second year courses to be more open to the use of WebCT in their courses.
• Watch the trend toward thinking that traditional text-based reading is NOT easier than web-based reading. In so doing, we should ask what this implies for our mission to provide excellence in education.
• Revise the survey form to more accurately report on aspects of WebCT use.
• Consider surveying the faculty who include WebCT in their course in the same five areas of WebCT use.
• Remain aware of our students' expectations for learning computer technology skills that prepare them for their future careers.

© 1998. The author(s), Dean A. Morss and Patricia A. Fleming, assign(s) to the University of New Brunswick and other educational and non-profit institutions a non-exclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant(s) a non-exclusive license to the University of New Brunswick to publish this document in full on the World Wide Web and on CD ROM and in printed form with the conference papers, and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the author(s).