North American Web Developers Conference, October 1998:

Providing Intelligence in Web-based Tutors

Jane M. Fritz
University of New Brunswick
Fredericton, NB, Canada

ABSTRACT

The TeleLearning Group in the Faculty of Computer Science at the University of New Brunswick is engaged in developing a suite of on-line tools and applications to support the teaching and learning of discrete mathematics. These vary from Java applets that can be used independently to explore a specific concept in discrete mathematics to a comprehensive Web-based Intelligent Tutoring System, DiscMath, currently under development. One component of this project, a Web-based Proof Tutor, provides exercises and guidance for students learning a variety of proof techniques: logical equivalences, logical implication, direct proof, indirect proof, and mathematical induction. This facility has been used by several classes in universities in the United States and Canada over the past 18 months. A version of the Tutor soon to be available through Netscape 4.04 uses Java applets to offer explanation by animation.

The newest extension to the Proof Tutor is the addition of intelligence. There are many challenges to providing intelligent guidance to a subject which is abstract in nature and inexact in the responses which are possible as a proof solution. This paper discusses the process of meeting these challenges. We address mechanisms by which to offer learners flexibility in attempting new examples and exercises when more than one solution may be possible. We discuss a solution to providing intelligence to unknown users who are struggling to understand how to construct proofs. We also look at the tradeoffs involved in choosing this solution, including development time and ease of adding new learner error patterns as they are identified.

Keywords: Web-based tutors, proofs, intelligent tutoring systems, user interface design

1. Introduction

The art of constructing a mathematical proof is a challenge to learn and a challenge to teach [Solow 1990]. It is not possible to memorize a technique and apply it successfully without deep understanding. The exact nature of two proofs using the same proof technique will vary depending on the specific problem statement. The algebraic stumbling blocks associated with formulating a proof will vary from problem statement to problem statement, regardless of the fact that the proof technique itself may be the same. There may be more than one way to prove the same problem statement. There may be more than one stumbling block associated with completing a proof. It is difficult for students to communicate their particular dilemmas in understanding the process, even when they are one-on-one with an instructor. Therefore, an attempt to deliver tutorial assistance to students through a Web tutor presents many challenges. The original Proof Tutor was devised as a pragmatic way to provide explanations of why we construct proofs certain ways for specific examples. This paper discusses the structure and justification for the original Proof Tutor and then proceeds to discuss the design of the enhanced version, which is learner driven and provides intelligent feedback.

2. Existing On-line Tutor

Proof Tutor (http://www2.cs.unb.ca/profs/fritz/CS1303/tutor/mainttr.htm) has been available to students studying discrete mathematics for three academic terms [Fritz 1997]. In order to overcome the non-trivial implementation issues of accepting mathematical notation as input and trying to solve an unknown problem, this implementation was restricted to providing solutions to selected problems. The value to the students over seeing solutions in a text or through class notes taken from problems worked on the board is threefold: the explanation for why we are taking each step is provided, the reason (mathematical justification) for a step can be animated, and the students find it reassuring to have an additional avenue of support.

Proof Tutor provides a collection of solved problems, with step-by-step explanation of strategy, for several methods of proof: logical equivalence, logical implication, direct proof, proof by contradiction, proof by contraposition, and mathematical induction. The main menu is shown in Figure 1.


Figure 1. Main menu of Proof Tutor.

Figure 2 illustrates the interface for each method of proof, in this case looking at proof by logical equivalence. The interface presents a collection of examples, the overview of the procedure involved, and a detailed solution for each example as it is selected. The use of a "Why?" button to follow the strategy for each step is shown in Figure 3, using an example of mathematical induction. The second "Why?" button has been invoked, thus activating its corresponding explanation window.


Figure 2. Interface for an example of Logical Equivalence.


Figure 3. An example of Mathematical Induction with a "Why?" box in action.

The explanation windows are generated by JavaScript code behind the "Reason" and "Why?" buttons. This separation of task allows the help provided to be changed simply by changing the HTML file contents of the help windows, which are identified through the JavaScript. We have used static text, images, and also animation to explain the mathematical reason for a proof step. For example, the animated version of the distribution law shows the relevant term physically move from the expression of the previous step to take its new position in the expression of the new step. This animation is provided through a Java applet that is embedded in the window's HTML code. Voice could be added as another form of learner support without changing the main part of the application.

This application has been used for three academic terms by students at UNB and at two other participating universities in the United States. The objective effectiveness of this support tool has not been evaluated yet, but anecdotal evidence suggests that it is very popular with students, who frequently ask for more examples to be included.

3. Limitations

Although the application seems to meet a perceived need, the Proof Tutor is limited in several ways. Its lack of interactivity translates into a passive role on the part of the learner, so that deep learning is not promoted. Because the problems are solved one way only and are static, there is no way for a student to appreciate that there is more than one right way to solve several of the presented examples. In this passive application there is no way for a learner to learn from his or her mistakes, because the user is not attempting a solution. In order to provide a Proof Tutor that promotes deep learning, the application must:

Such an application translates into learner-directed step-by-step proof construction with feedback at each step. The implementation questions that arise are:

The answer to these questions provides the constraints that drive the design of the new version of the Proof Tutor. Not surprisingly, these were the same questions whose answers resulted in the original version to be created. First of all, the problem of inputting mathematical notation remains non-trivial, so the short term solution is to provide learners with prewritten expressions from which to choose. And the challenges identified by these questions expose the reality of the content domain of proof construction: each problem statement to be proved has a unique set of stumbling blocks leading up to a solution. In other words, a generic problem-solving structure will not work. Also, in providing for step-by-step solutions that are learner-driven, there is no timely way for a program to be able to recognize what the learner is trying to do so as to provide feedback. The following sections discuss the methods arrived at to permit learners to attempt their own solution to a diverse set of proof problems, during which they are provided with expert feedback and guidance.

4. Adding Intelligence

By intelligence we mean that the feedback provided to learners as they attempt each step in a proof must recognize what they have tried to do and be able to respond accordingly. The recognition has to allow for the fact that there may be more than one way to construct the proof. The response must offer an explanation for what is wrong, allow a student to make another attempt if wrong, and possibly provide a hint. As a tutor, the application must provide two levels of guidance: guidance at a particular step, and also guidance to encourage the learner to appreciate how to develop an overall strategy to construct a problem solution. Both of these levels should be reflected in the feedback given to the learner.

We can distinguish two areas of expert knowledge that needs to be brought to bear in providing intelligent feedback. Because each problem statement has its own set of issues, potential errors related to each particular problem need to be identified and reflected. Because there are strategies and procedures that are common to each method of proof, these need to be identified and reinforced in feedback at appropriate times. Common errors that are made by novice proof constructors should be included as choices in a tutor so that these common mistakes can be addressed. When only correct solutions are presented, the learner does not get the opportunity to understand why his or her way does not work.

5. Our Solution

The compromise that allows us to handle the inputting of mathematical notation is to provide multiple choice selections for each step and also for the reason for each step. Although it is not ideal, it is clean and doable, two positive traits. The compromise that allows us to provide appropriate feedback and strategic guidance is to choose the example problems to be presented and identify a set of right, wrong, and misguided solution steps for each example problem that are representative of student solutions. This allows common errors to be addressed and explained. This section describes the knowledge base that is established to store common errors so that realistic options are provided to the learner. It also outlines the problem solution tree that is constructed for each example problem. The problem solution tree uses material captured in the knowledge base to promote consistency among feedback messages and reuse of code.

5.1 The Knowledge Base

The knowledge base contains material gleaned from experts (experienced instructors) that reflects several common approaches to solving both specific problems and categories of problems. The knowledge base has three parts; it contains correct solutions (sometimes more than one for a given problem) for every example problem presented, it contains common mistakes encountered in formulating proof solutions, and it also contains hints and other helpful remarks. Mistakes may be common to a particular proof method or common to a particular rule of algebra or logic. The method used to ensure completeness in acquiring expert knowledge for a given problem is: The knowledge base is built up during the process of detailing individual example problem solutions. Once a few example problems have been developed, the knowledge base contributes to the consistency of feedback to the learner while it continues to grow. Each example problem will have its own solution tree.

5.2 A Problem Solution Tree

In an expert system, the problem domain can be thought of as being defined as a decision tree. The system prunes the tree as questions are asked and the remaining branches of the tree are followed and pruned until we arrive at the expert result. In our domain of problem statements requiring proof solutions, it has been established that each problem is unique, therefore each problem will have its own decision tree. The decision tree effectively maps a correct solution, optional alternate correct solutions, and incorrect steps. We call this structure a problem solution tree. Each branch on the tree can be thought of as a step in the proof, and the branches are followed according to the chosen steps and their accompanying reasons, as shown in Figure 4. Each correct step choice in a Step node will have a corresponding Reason node. Once a Step/Reason node has been traversed satisfactorily, navigation continues with the next step. In the construction of a problem solution tree, we encapsulate common stumbling blocks for the problem steps by reviewing material in the knowledge base. The knowledge base is effectively a repository of feedback messages. Some examples of common stumbling blocks that can be reflected in a decision tree are:
  1. Trying to prove a hypothesis by using the conclusion,
  2. Improper use of the Distribution Law by distributing over the same operator,
    e.g., ( p /\ q ) /\ ( p /\ r ) <=> p /\ ( q /\ r ),
  3. Using the wrong definition of p -> q,
  4. Failing to see common subexpressions.


Figure 4. Schematic of a Problem Solution Tree.

Figure 5 illustrates an implementation strategy for presenting an example problem for a learner to work on step by step. The main window looks exactly the same as the passive version, except that the window is refreshed by subsequent steps as the learner completes them. The "Next" button invokes a "Step" window, which displays a drop down box with a choice of steps. A response to the learner's choice of step is displayed in the Step Response box. When the learner has identified a satisfactory step, the "Reason" section of the window is activated, in which the learner identifies the corresponding mathematical reason for that step. When the step and reason are both identified, the learner clicks o the "Update" button, at which time the main window is updated to include the new step and reason and a new "Next" button is inserted on the line below the new step. The Step/Reason window is a realization of a Step/Reason node in a problem solution tree.


Figure 5. Implementation of a Problem Solution Tree.

5.3 The Tradeoffs

As with any solution design, there are tradeoffs involved. Desirable features have a cost that must be weighed against need. Some features are unfeasible due to technical constraints, some due to unsatisfactory implementation time or cost. We provide a pragmatic design that can be implemented in several ways. The following tradeoffs are being made to allow for a timely implementation:

6. Evaluating the Value Added

It is our intention to undertake a study of the relative achievement of students who choose not to use any form of on-line Proof Tutor, of those students who are provided with the passive version of Proof Tutor, and of those students who are provided with the intelligent version of Proof Tutor. Those results will be compiled during the fall and winter terms of the 1998-99 academic year.

7. Conclusions

Our experience in using Web-based support materials for traditional classroom learning is that students are very trusting of the relevance of these assigned materials to success in the course. For that reason it behooves us to provide on-line materials that promote deep understanding rather than superficial understanding. We have been providing students with an on-line tutor that supports their understanding of why a correct proof solution is indeed correct. Our enhanced on-line tutor supports students' understanding of why common incorrect approaches are indeed incorrect and how to overcome misunderstandings. A comparison of performance outcomes will be forthcoming.

References

Jane Fritz, "Learning Proof Techniques Through Visualization", Proceedings of Ed-Media 97, AACE, Calgary, Canada, June 1997, pp. 1460-1470.

Daniel Solow, How to Read and Do Proofs: an introduction to mathematical thought processes, 2nd ed., John Wiley and Sons, New York, 1990.

Dr. Jane M. Fritz
Professor
University of New Brunswick
P.O. Box 4400
Fredericton, NB, Canada E3B 5A3
jane@unb.ca
http://www2.cs.unb.ca/profs/fritz

© 1998. The author, Jane Fritz, assigns to the University of New Brunswick and other educational and non-profit institutions a non-exclusive license to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive license to the University of New Brunswick to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers, and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the author.