The development of this system started on 1994, and its first version was already developed with the latest Object Oriented technology for reusable components and it was presented at the Escuela Técnica Superior de Ingenieros Industriales (Universidad Politécnica de Madrid) on May 1995. On September 1995 it was first successfully used with new registered students and with a data base consisting of a set of physics, mathematics and chemistry questions, all of them available on a client/server architecture. Results of this application allow the students to see their knowledge level on basic engineering subjects, and also allow the teachers to know exactly the level of the students and to adapt their subjects to the actual group they had to deal with. The 2.0 version was developed during 1996 and 1997. This version may be used and distributed through World Wide Web: the students can use the system anytime from any WWW terminal, during the whole year.
Keywords: Self-Evaluation, Student Tracking, WWW, Internet, Intranets, Multimedia, Databases, Object Oriented
In order to provide a system for the assistance of teachers in adapting their courses contents and, also, first year students to know their starting knowlegde level, in an specific subject, at the ETSI Industriales in the Universidad Politécnica de Madrid, a WWW tool has been developed [1]. Besides the tool can be used in other two different ways: (a) by the teachers for students trucking in defined periods of time [2] and (b) by the students for self-monitoring during the course.
The whole system includes a World Wide Web server, a multimedia database management system, the database contents and a graphic user interface [3]. As the WWW must allow a large number of students to use a time and (theoretically) location independent system, a multimedia DBMS is able to manage all kinds of files giving flexibility to the database. The Object Oriented choice makes easy the design, migration and reusability of the system components. In the other hand, the quality of both, the database contents and the user interface, can become into the key features for the tool success.
The WebTutor system for self-evaluation and student tracking uses an Object Oriented Data Base that is structured in several blocks in which are stored all the data about educational centers, courses, tutors, students, pedagogical contents and exams. The most important block is the Contents database (see Fig. 1) that includes all the subjects in order to evaluate, such as Mathematics, Physics, etc.
Fig. 1. Multimedia Contents Data Base Scheme
These subjects are organized in Didactic Units associated to questions sets subsequently used for the exams generation. Wide sets of Aims are included in the Contents database. These Aims or goals are interpreted as the desired concepts learned by the student when finished the training time [4]. This particular item is one of the system highlights because provides a powerful tool to analyze the learning misconception of a particular student and the subsequently actions.
Fig. 2. Tests course configuration
Every student has assigned a periodical test sequence related to the registered subjects. The exercises have a variable number of questions from the data base with a different kind of answers: single choice selection, multiple choice selection, true/false, string type, integer number, real number or numerical array type. All the tests have to be configured by the tutor [5] assigned to a student (see Fig. 2). This configuration includes the following parameters: total number of questions, questions distribution according to their difficulty level (easy, medium, difficult), Didactic Units and Aims included in the selection and student or group of students target. When the test configuration has been defined and activated, the student can do one o several test of each model his tutors has been prepared for him. When the student selects one of the configurations, the system generates the test automatically taking the questions from the database by a random process. In this way, all those exams generated from the same configuration will be different making easy the not well-assimilated subjects or aims revision. When the student finishes the test, WebTutor gives the student this obtained mark in this moment and updates the values in the Aims and Results database once a results statistic process over the set of a particular student tests has been done. The student can also check the results with the correct answers and see an explanation of how to answer any of the test questions. These results can classify in: Global Results, Didactic Units Results and Items Results and all of them are available for the corresponding teachers (see Fig. 3).
Fig. 3. Accessing Course results
The results are given to both, the student and the tutor; in this case the tutor can get them individually or collectively. These data let the tutor analyze the assimilation level of one student or a group of students in a Didactic Unit or an Aim. The system result is a data set that shows the knowledge level of each student or group of students about one specific subject. With these data, a revision can be suggested when the student knowledge level is not adequate. Moreover, the tutor can aim the subject contents to the desired level taking into account the students assimilation capacities.
The system database can be divided in four functional groups: Documents Storage and Representation, Questions Storage, System Groups and Users Registration and Results Storage. All of these groups are connected with each other in the global system as showed in Fig. 4-7.
The documents included in the test questions are stored in the Documents Storage and Representation functional group. Each document is composed by a list of heterogeneous elements (text, images, audio, video,...) giving flexibility to the questions contents (Fig. 4). Each element is properly showed inside the question general context.
Fig. 4. Documents Storage and Representation Classes Scheme
The Questions Storage functional group stores the contents of the subjects to be evaluated. Each subject is divided in chapters or units with an associated set of questions/answers. Each question has a wording and an answer with an specific kind. All the subjects are detached in concepts. Each question evaluates a set of concepts by means of a relative weight that indicates the proportion the question resolution takes part (Fig. 5).
Fig. 5. Questions Storage Classes Scheme
The System Groups and Users Registration functional group is designed in order to store any kind of education center organisation. A group is defined as a student set registered in a subject with an specific tutor (Fig. 6). The three functional group components can be modified independently.
Fig. 6. System Groups and Users Registration Classes Scheme
The Results Storage functional group collects the students tests answers and results (Fig. 7). Each exam is stored including all the questions and the student answers. The results are calculated from the student answers and the question correct answers matching. The tests are generated by means of the configuration parameters made by the tutor. Each new configuration adds a test item in the students group test list. The test generation is produced when the students chooses the test item in this list.
Fig. 7. Results Storage Classes Scheme
The development of this system started on September 1994 supported by the Sociedad de Amigos de la ETSII-UPM, and its first version was already developed with the latest Object Oriented technology for reusable components and it was presented to the ETSI Industriales - Universidad Politécnica de Madrid on May 1995. On September 1995 it was first successfully used with exactly 251 new registered students and with a data base consisting of 1000 physics, 1000 mathematics and 1000 chemistry questions, all of them available on a client/server architecture. Results of this application permitted the students to show their knowledge level on basic engineering subjects, and also allowed the teachers to know exactly the level of the students and adapt their subjects to the actual group they had to deal with.
The 2.0 version was developed during 1996 and 1997. This version allows use and distribution through Internet (WWW, see Fig. 8). Currently, the students can use the system anytime from any terminal, during the whole year. The Internet version was design for ATE.net, an Spanish educational technology consortium created in order to apply this system in secondary schools and other educational projects. ATE.net consortium gathers Fomento de Centros de Enseñanza (with more than 30 educational centers, primary and secondary schools, throughout Spain), the Spanish telephone company Telefónica, the Banco de Santander and the Fundación para el Fomento de la Innovación Industrial (F2I2, belonged to the Ministerio de Industria and the Universidad Politécnica de Madrid).
Fig. 8. System implementation scheme
The system has been applied to the new registered students at the E.T.S.I. Industriales (UPM) in 1995 and 1996. In september 1997 the Webtutor Internet version was applied using HTML and Java programming technology. The students have been able to use the system anytime, from anywhere there were a WWW terminal, during the whole year. Results of this application have allowed the students to determine their knowledge level on basic engineering subjects (Mathematics, Physics and Chemistry), and also the tutors to know exactly the level of the students and adapt their subjects to the actual group they had to deal with.
The same tool, with a different Contents database, has been successfully used during the 97/98 course en 33 Spanish secondary schools. In this case, the database contents embraced subjects such Language, Mathematics, Natural History and Social Education.
A WWW system for the assistance of students and teachers to determine students knowlegde levels in an specific subject has been developed. This software solution gives the possibility to each student to assess his knowledge, and to produce a complete aims analysis to prepare his or her particular learning strategy. Also the system results can assist teachers in adapting their courses contents before and during the teaching period. The system architecture and the graphic user interface design facilitate the user interaction with it. The interaction is based on hypertext and multimedia possibilities on a web based application.
Self-evaluation and student tracking systems help in the educational process because: (a) they help the student to identify their misconceptions and understand their real deficiencies, (b) the teachers and professors can analyze and design a strategy for each student in order to reduce his or her misconceptions and (c) the global information generated by the system helps the organization to design the appropriate curricular context.
The authors express their gratefulness to the Sociedad de Amigos de la ETSII-UPM and ATE.net for their support and to S. Coronado and Departments of Physics, Mathematics, Chemistry and Graphic Engineering (ETSII-UPM) for their collaboration on the implementation work.
[1] WebTutor: Manual de Administración. Laboratorio de Informática - Fundación para el Fomento de la Innovación Industrial, ETSII-UPM. Sept. 1997.
[2] Hämäläinen, M., Whinston, A.B. and Vishik, S., Electronic Markets for Learning: Educational Brokerages on the Internet, Communications of the ACM, 39, 6 (Jun. 1996), 51-58
[3] Coronado, S., García-Beltrán, A., Jaén, J.A. and Martínez, R. WebTutor, a knowlegde based system for evaluation and tutorship in Tasks and Methods in Applied Artificial Intelligence Vol. II, Proceedings of IEA-98-AIE, Jun. 1998.
[4] Eskenasi, A., Vladimirova, T. and Vassileva, J. Incorporating Student Models in Adaptative Testing Systems, ETTI 30,2, (1992)
[5] WebTutor: Manual del Profesor. Laboratorio de Informática - Fundación para el Fomento de la Innovación Industrial, ETSII-UPM. Sept. 1997.
Angel García-Beltrán, Assistant Professor,
agarcia@etsii.upm.es
José A. Jaén, Full Profesor and Vicedean of ETSI Industriales,
jjaen@etsii.upm.es
Raquel Martínez, Assistant Professor,
rmartinez@etsii.upm.es
José A. Criado, Research Fellow,
jacriado@dii.etsii.upm.es
División de Informática Industrial - Universidad Politécnica de Madrid
ETSI Industriales, C/ José Gutiérrez Abascal, 2. E-28006, Madrid (SPAIN)
Tel: +34-91-336-3200
Fax: +34-91-561-8618
http://www.dii.etsii.upm.es
© 1998. Angel García-Beltrán, José A. Jaén, Raquel Martínez and José A. Criado assign to the University of New Brunswick and other educational and non-profit institutions a non-exclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grants a non-exclusive license to the University of New Brunswick to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers, and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.