Keywords: Internet-enhanced Education; Student Feedback; Evaluation; Online Components in Traditional Course Settings
This paper examines the use of IntraKal (http://www.anlon.com), a class management and communication software program, by faculty in the Williamson College of Business Administration at Youngstown State University, during a three-month trial period, January - March, 1998. At the conclusion of the Winter academic quarter (March, 1998), students evaluated the frequency of access and usefulness of the software as well as their perceptions of its impact on their own learning.
The software was accessed by students on and off campus via the Internet, using standard web browsers. Although all four faculty members had previous Internet experience, this was the first intensive exposure to the Web for many students. Students in the business school do not live on campus (98% are commuter students), are older than traditional students, and many lack previous Internet experience. A college-wide survey conducted in September 1997 indicated that 39.1% of students report that they do not have the ability to access the Internet at home, work or school. (3)
No formal assessment of faculty motives was done prior to this trial, but increasing and/or facilitating communication, providing useful online experience for students, and managing the exam and grading process were mentioned as goals for the software. The software was installed in December, 1997 and training for faculty was provided just prior to Winter Quarter, 1998.
All courses had some course information available online. At a minimum, this information included a course syllabus and information concerning the various ways in which the instructors could be contacted. Several of the course instructors also used other features of the software extensively. This included providing students with lecture notes, case materials, sample exams, and assignments. In addition to these static sources of information, faculty also used interactive forms of communication -- e-mail, electronic bulletin boards and chat-rooms including online office hours.
E-mail was an integral part of this trial. Students were provided with University e-mail accounts, although some students preferred to use e-mail accounts from commercial vendors. When students registered online for these courses, they were required to specify a personal e-mail address. This address was entered into faculty rosters and gradebooks. When assignments or exams were completed, confirmation e-mails were sent to students' personal accounts. Faculty could also send bulletin board postings or other notices quickly to a class section or several sections of the same course using this e-mail roster. In addition, students could contact faculty members through e-mail links at several locations within student menus.
The electronic bulletin board and chatrooms were used only slightly. Each course bulletin board allowed faculty and students to post messages for all class members to read. A "response" feature allowed students comments to be organized by topic. Through the bulletin board, students could also send anonymous messages to the faculty. These messages were not posted on the bulletin board, but sent directly to faculty e-mail with a generic system return address that was not traceable to individuals. Chatrooms were created for each course, for departments, for the college as a whole, for each level of students (i.e., freshmen, sophomores, etc.). Some instructors used the course chatroom to hold online office hours at pre-specified times.
Two faculty members used online testing with immediate feedback and all used sample exams. All students had online access to exam and course grading information. One of the more attractive features of the software is the capability it provides for administering online exams. The capacity exists for students to take the exams from any computer with an Internet connection. Without the means for monitoring students at remote locations and with no expectation of students adhering to a self-monitored honor code, the taking of exams was generally restricted to computer labs under the supervision of the instructors. However, several instructors used the "remote exam-taking feature" to give students "sample" exams that they could complete at their leisure.
In most cases, students met as a class and took the actual exams as they normally would have, except that the exam questions were accessed via the Internet. Upon completing the exams, students were able to indicate that they were ready to have their exams graded and would then receive immediate feedback as to how well they had done on the multiple choice portion of the exam. After all class members had taken a particular exam, e-mails with "missed" questions and other exam information could be sent to an entire course section by the faculty.
Some faculty members had essay questions on exams or used this feature for other assignments. When essay type responses were required, students' answers were placed in a file which instructors could access at their convenience for grading. Student responses were coded by student numbers preserving anonymity. A"corrected" response could be e-mailed back to students. Although the immediate feedback was not available for essay responses, there were still a number of advantages to be realized with such a system. Among these advantages were: students' ability to easily edit their responses before submitting them, evaluators being able to avoid the biases created by students' handwriting, and relatively quick feedback - in that students could have their scores and the evaluator's comments e-mailed to them before the next scheduled class meeting.
Once exams or essays were completed, students' grades were placed in the instructors' "online grade books." Students were able to access their grades at any time during the term using their personal user names and passwords. This kept the students' progress in the course salient in their minds, allowed the instructors to answer any questions about grades (generally via e-mail), and make any necessary changes/corrections more quickly. A particularly appealing feature of this type of feedback was that it empowered students with direct access to information typically under the strict control of the instructor in a "traditional" classroom.
It should be noted that the differences in individual faculty usage of the software may skew data to underestimate student satisfaction. Students in courses where the faculty member did not use a particular feature or used it infrequently may be more likely to choose values on the low end of the scale, like "very strongly disagree," "very seldom," or "not at all useful." While these are valid opinions, it is also important to examine students with greater exposure to particular features of the software to determine "experienced user" opinions of usefulness and value.
The average student ranked sample exams, item 24e, (m=4.87) and exam feedback, item 24f, (m=5.05), somewhat lower than traditional learning methods, including studying the text, item 24a, (m=5.22) and attending class, item 24b, (m=6.11), "in terms of how useful they were to [the student] in working toward the grade expected in the course." This average does not fully describe students' assessment of the usefulness of the software since different aspects of the software were utilized by faculty. For example, sample exams, item 24e, were considered useful by 57.2% of all students, but those in courses with the opportunity to take sample exams found them more than "somewhat" useful (86.4%). Although the software was not intended as a replacement for traditional teaching methods, students seemed to find its features useful.
One instructor exclusively used the online software for exams, one instructor never used the exams, and the remaining two had limited online testing. Although the mean frequency reported for item 1g - "Taking exams" was 2.56, the value soars to 6.54 in the courses where online testing was the only method of evaluation, and ranges from 2.39 to 0 for the other courses. Similarly, the usefulness of taking exams in courses with online testing, item 2f, had a mean 6.54 with 100% of students rating it useful (values 5-7) and 79.2% of students choosing the "very useful" value (7).
The performance measure, item 15, is slightly less, with over one-third of students (38%) neither or agreeing nor disagreeing with the statement "Overall, I feel as if the material that was made available on the online classroom software improved my performance in the course" which contributed to the low mean value (4.59) for this item. Again, the results for the students in the courses with exclusive online testing are significantly different, with no students disagreeing.
Students, even those without prior Internet experience, reported that the software was easy to use and understand. Good menus and clear instructions contributed to this perception. The ability to get questions answered quickly and easily "in real time" was also a benefit. Expanding discussions "out of the classroom" through bulletin boards and chatrooms may also benefit students with more opportunities to interact, not only with faculty, but with other students in a commuting-student environment.
Some students reported difficulty accessing the software, ranging from inability to connect to the Internet from commercial sources at peak hours to inability to use web browsers effectively. Some of these problems may have been due to students' lack of prior exposure to the Internet, or lack of access at home (for commuting students). As student and faculty Internet exposure and access increases, most of these challenges will disappear.
Organizational computing infrastructure was also a challenge. A server and system administrator is required to store and organize information. The availability of hardware was also a factor that prevented all faculty from utilizing online testing.
2. Tom Diederich, "Tochnology may spark education revolution," Computerworld. Online edition. 21 May 1998. http://cnn.com/TECH/computing/9805/21/education.idg/index.html(return to text)
3. Internal Memoranda, Williamson College of Business Administration, Youngstown State University. Youngstown, OH. November, 1997. (return to text)
4. Wilson, ibid.(return to text)
5. Todd Woody, "Academics rebel against an online future," The Industry Standard. Online edition. 12 June 1998. http://www.cnn.com/TECH/computing/9806/15/academics.idg/index.html. (return to text)
Diane Ciba, MBA
Research Assistant/Marketing PhD Student
University of Connecticut
12 Homestead Avenue
Waterbury, CT 06705 USA
diciba@aol.com
Thomas Rakestraw, PhD
Associate Professor/Management
Williamson College of Business Administration
Youngstown State University
1 University Plaza
Youngstown, OH 44555 USA
rakestra@usaor.net
© 1998. The authors, Diane Ciba and Thomas Rakestraw, assign to the University of New Brunswick and other educational and non-profit institutions a non-exclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive license to the University of New Brunswick to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers, and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.