In this study, the principal author used John Biggs' (1999) theory of 'constructive alignment' and a problem-based approach to guide the design and delivery of an undergraduate web-based unit. The distinguishing pedagogical feature of the instructional design was the use of asynchronous text-based threaded discussion as the main medium for communication, instruction, collaboration, and assessment. The challenge in applying Biggs' theory was to ensure that the curriculum objectives were appropriately 'aligned' with all teaching and learning activities, as well as all assessment tasks. The unit presented an excellent case study for an in-depth examination of the effect that asynchronous text-based discussion, structured according to constructivist pedagogical principles, has on student learning. The textual records of students' discussions and other assessment products provided a rich source of data that were analysed using the SOLO taxonomy. This research supports the view that embedding student collaboration and problem-based learning in asynchronous text-based discussion provides teachers with a powerful strategy to support students' active engagement with content and facilitates high levels of conceptual understanding.

Keywords: assessment, asynchronous discussion, collaborative learning, constructivist pedagogy, learning processes, problem-based learning, SOLO taxonomy.

Constructivist theories of student learning are derived from cognitive psychology and, while varying in specifics, are grounded in three key assumptions about learning. Constructivism assumes that effective learning requires active engagement with subject content; collaborative learning environments; and opportunities for problem solving (Marton & Saljo, 1975; Bruner, 1985; Brown & Atkins, 1988; Ramsden, 1992). Vygotsky's (1962) research on the relationship between cognition and language has made a significant contribution to our understanding of the central role played by the social dimension of learning contexts. The use of technology in education has introduced new ways for social interaction to take place in online contexts.

Asynchronous (time-independent) text-based discussion is commonly incorporated in the delivery of online education. As this online tool supports collaborative activities as well as dialogue, it has great potential for enhancing student learning. However, it is not always used effectively. While students readily participate in social 'chat' forums, their level of participation and cognitive engagement in online discussion for academic purposes tends to be low (Ragan, 1998). There are several reasons why online discussions may not facilitate learning. These include, inadequate discussion management strategies (Collins-Brown, 1999), poorly constructed discussion topics (Beaudin, 1999), non-existent or infrequent teacher feedback (Rossman, 1999), irrelevant or negative feedback from teachers and peers (Rossman, 1999), and difficulty in sustaining the momentum of discussion (Beaudin, 1999; Winiecki, 1999). It is not surprising then that research findings on the effectiveness of online dialogue in enhancing student learning are equivocal. Nevertheless, there is support in the literature for its potential for facilitating greater participation among learners, improved decision-making and higher-level reasoning (Miller & Miller, 1999:108).

Theories within the constructivist pedagogical paradigm are most relevant to collaborative learning contexts (Biggs, 1999; Miller & Miller, 1999; Tam, 2000) and offer insights into ways in which asynchronous online discussion may be effectively employed. A basic tenet of constructivist theory is that learning is a process of conceptual change whereby individuals construct new understandings of reality. While learning can be categorised and defined in numerous ways, this paper is concerned with academic learning, which "…involves complex cognitive activities engaged by students in a personal interaction with learning tasks embedded in particular learning contexts" (Tang, 1998:102).

An underlying assumption used in the development of ME330 was that learning is an active process in which students construct their own understandings (personal meanings) of multimedia design issues. Understanding is facilitated by engaging in appropriate learning activities such as collaboration and dialogue with teachers and peers, as "good dialogue elicits those activities that shape, elaborate and deepen understanding" (Biggs, 1999:13). A central argument of this paper is that, provided asynchronous text-based discussion is thoughtfully structured, it is an excellent vehicle for engaging students in meaningful online dialogue. Biggs' theory of 'constructive alignment', which he defines as "… a marriage between a constructivist understanding of the nature of learning, and an aligned design for teaching" (1999:26), provides a useful framework for structuring online discussions.

Effective instructional design (ID) according to Biggs involves aligning three pedagogical elements–curriculum objectives, teaching methods and assessment tasks–and maintaining consistent learning-related activities across all stages of instruction. The aim of this approach is to maximise opportunities for students to participate in appropriate learning activities. Constructive alignment is effective because it "… makes the students themselves do the real work; the teacher simply arranges things so that it is more likely that they will" (Biggs, 1999:27). A problem-based instructional design supports the integration of multimedia design theory and its practice: students learn about multimedia design through the activity of designing a multimedia product. The experience of practising the skills of multimedia professionals provides students with learning processes that are relevant and motivating, and enable them to take responsibility for their knowledge construction. The application of problem-based principles to guide the instructional design of 'Multimedia Design Update' (ME330) and the analysis of its impact on student learning is discussed in the following sections of this paper.

Unit ME330 is a Year 3 unit in the Bachelor of Multimedia degree at Southern Cross University. The main teaching objective of this Unit was to enable students to identify and evaluate developments in the field of educational multimedia, with specific reference to design strategies applied to the production of multimedia applications. In particular, students were required to examine the links between design strategies and educational effectiveness of interactive resources. Traditionally, ME330 had been delivered in a one-hour lecture, two-hour tutorial format and students were assessed on individual performance in written essays or reports on relevant topics. Over the summer of 1999, the principal author revised ME330 to incorporate online technology. She used a constructivist approach in re-engineering the Unit, with a view to:

• providing students with opportunities to apply instructional design theory in a practical and relevant context;
• giving students experience of working in a multimedia team and engaging in collaborative tasks; and
• developing students' skills in communication.

While these objectives could have been undertaken by traditional, face-to-face (FTF) teaching, it was felt that Web technology offered a more relevant and dynamic environment for multimedia students. The distinguishing pedagogical feature of ME330 was the use of asynchronous text-based threaded discussion as the main medium for communication, instruction, collaboration, and assessment. The challenge in applying Biggs' theory was to ensure that the curriculum objectives were appropriately 'aligned' with all teaching and learning activities and all assessment tasks. This was achieved by using a problem-based learning (PBL) approach in structuring ME330.

Problem-based learning principles provide an excellent framework for achieving the alignment of teaching objectives, learning activities and assessment tasks. As Biggs (1999:207) states, 'PBL is alignment itself'. The 'problem' assigned to the ME330 group was presented in the form of a class project: Design an online study guide for an undergraduate course, "Issues in Multimedia Design". Students were initially required to work in small groups or pairs to develop a design topic (eg: aesthetics) and later collaborate as a class to integrate all design topics into a single study guide. Figure 1 illustrates how learning activities related to unit objectives and assessment tasks.

Figure 1: Constructive Alignment of ME330 Unit

It is generally recognised that students' energies are primarily focussed on the assessable components of course material. Thus, the design of assessment tasks was central to the educational effectiveness of ME330 and warrants further discussion.

There were four assessment tasks in the Unit. Learning responses to these tasks were graded against detailed criteria that were provided to students at the commencement of the semester, and were available on the ME330 web site for easy reference.

A1: The online discussion facility was the mechanism for student control of learning. Online discussions provided the main means of communication (with peers and the instructor), sharing resources and interacting with Unit content. Detailed criteria used for assessing postings were provided to students to guide their participation in tasks such as collaboration on project planning, resource sharing, peer review and feedback.

A2: A group assessment task was included to reinforce the importance of teamwork skills and to give students practice in developing a real multimedia product in a small team context.

A3: After providing constructive feedback online to other groups on their study guide 'prototypes' (a component of A1), each student was required to write a critical review paper that developed these formative evaluation comments. Students were expected to argue strongly for their position and justify their views by referring to literature, theories and current multimedia design practice.

A4: The final assessment task involved the summative evaluation of the integrated study guide product and the instructional design of the Unit in terms of educational effectiveness. This task required students to reflect on the entire teaching and learning process from the perspective of active participants and as instructional designers.

The application of a problem-based instructional design ensured the 'constructive alignment' of curriculum objectives, learning activities and assessment tasks in Unit ME330. Importantly, this constructivist approach provided a pedagogically sound structure for the online asynchronous discussions that played a central role in this Unit. As Figure 1 illustrates, the design of the assessment tasks was crucial in achieving this. The sequencing and nature of the assessment tasks were carefully structured to maximise students' participation, collaboration and active engagement with subject content. The online discussion facility gave students a means of developing their understandings of multimedia design issues by actively writing out their thoughts and sharing them with peers and the instructor. The textual record of the discussion threads provided a valuable source for student reflection during the semester and was available for research analysis. The instructor used the discussion facility to provide timely feedback to students and to intervene to keep discussions on track and sustain their momentum. Furthermore, by monitoring the discussions, she was able to track individual student's cognitive development and gain insight into students' learning processes. It was anticipated that the logically structured, problem-based pedagogical design of the Unit would assist students to develop high levels of conceptual understanding of subject content. In the next section, methods used to evaluate students' engagement with content and conceptual understanding will be discussed.

The Australian higher education system requires the quantification of students' academic achievement and so the quality of student learning is usually judged by assigning norm referenced or criterion referenced marks or grades on a range of assessment tasks. The grades students obtained in ME330 provide one measure of learning outcomes, and could be compared with the grades obtained by students who had undertaken the Unit in previous years when it was delivered in a traditional way. However, the inadequacies of relying on numerical measures of learning have been discussed widely in the literature (Biggs, 1999, 1992; Entwistle & Ramsden, 1983; Kolb, 1976; Marton & Saljo, 1988), and alternative, qualitative learning assessment strategies have been developed. For example, Bloom's taxonomy (Bloom et al, 1956); the Learning Style Inventory (Kolb, 1976); the Inventory of Learning Processes (Schmeck et al, 1977); the Structured Observed Learning Outcomes (SOLO) taxonomy (Biggs & Collis, 1982); the Approaches to Study Inventory (Entwistle & Ramsden, 1983); and the Learning Process Questionnaire (Biggs, 1987). While an examination of this important issue is beyond the scope of this paper, it should be noted that differences in approaches to assessment are underpinned by conflicting assumptions about the nature of student learning.

The focus of this study was to understand student learning processes and gain insight into how students used elements of the teaching-learning environment to meet the learning objectives of the Unit. Three aspects of student learning were of particular interest: the level and quality of students' engagement with content, changes in students' understandings of multimedia design concepts, and the level of conceptual understanding students finally achieved. It was expected that the problem-based instructional design of ME330 would ensure that students sustained high levels of interaction with each other and with subject content throughout the semester, and that this would be associated with the development of high levels of conceptual understanding. Unit ME330 presented an excellent case study for an in-depth examination of these issues. Extensive data were obtained from the asynchronous discussion records, students' responses to assessment tasks, and the online multimedia product developed by the class. Document analysis and the application of SOLO taxonomy (see below) were the main methods used to analyse the data.

The textual records of students' online discussions were sorted into four categories: social, technical, procedural and academic. The first two categories were not included in the data analysis for the following reasons. Postings of a social nature, while contributing to group cohesion and interaction, were not related to academic content; similarly, comments of a technical nature, which in the main related to the functioning of Zope, were outside the curriculum. Thus, the analysis of discussion records was based on postings classed as either 'procedural' or 'academic'.

Procedural postings: The problem-based, collaborative nature of ME330 required students to develop effective communication skills in order to manage small and large group work. Thus, procedural postings, which related to group management issues such as establishing group members roles and responsibilities, time management and posting protocols, provided an indicator of students' level of engagement with these tasks. In order to provide a meaningful evaluation of students' engagement levels, two factors of their contributions were considered: the participation level or number of postings made during a time interval, and the quality of the content of each post.

Academic postings: Contributions to discussions that related directly to the academic content of ME330–resources, theory, design practice and development, justification of study guide elements, reviews, critical analysis, and evaluation–were subjected to analysis using the SOLO taxonomy to evaluate students' levels of conceptual understanding.

Given its strength in delineating conceptual processes, the SOLO taxonomy seemed to offer the most relevant data analysis strategy for the purposes of this study. SOLO also has the advantage of being applicable across a range of disciplines (Hattie & Purdie, 1998).

In discussing the application of the SOLO taxonomy to establish criterion-referenced teaching and learning objectives, Biggs (1999:46-48) develops the concept of 'cognitive complexity'–how well the level of understanding is exemplified–within each level of understanding. This hierarchy of cognitive complexity was used by the principal author to develop a system of SOLO tags for the analysis of students' learning responses. That is, within the relational and extended abstract levels, SOLO tags were used to distinguish between learning responses that represented an adequate or basic understanding, and those that represented a very good, more complex level of understanding.

Relational (a) & (b): The most basic level of relational understanding–'Relational (a)'–is a 'compare and contrast' response. Biggs (1999:48) refers to this as declarative understanding, which is the ability to use a concept to integrate a collection of data. 'Relational (b)' is a more complex or higher level of relational response and is reflected by the ability to apply a concept to a familiar data set or problem. Biggs (1999) calls this functional understanding.

Extended Abstract (a) & (b): Here students’ responses are characterised by a high level of abstract thinking, originality, or generalisation, and going beyond what is given or expected; for example, the ability to theorise, generalise, hypothesise, and reflect (Biggs, 1999:48). The SOLO tag 'extended abstract (a)' represents learning responses that demonstrate the ability to relate content to existing principles; whereas the 'extended abstract (b)' SOLO tag represents responses involving questioning and going beyond existing principles. The SOLO tag system used in ME330 is shown in Figure 2.

Figure 2: SOLO Tag System

It should be recognised that higher levels of learning responses require mastery of less complex levels of cognitive functioning (for example, functional understanding requires mastery of declarative understanding). Thus SOLO is a very useful pedagogical tool for teachers: it provides a framework to logically sequence learning activities, provide meaningful feedback to learners, and identify areas of conceptual difficulty. In short, SOLO provides a 'framework for understanding understanding' (Biggs, 1999:51).

In ME330, students' participation in online asynchronous discussions formed a significant part of their assessment and the threaded textual records of their discussions provided a rich source of data on student learning processes. SOLO was used to analyse these records and other assessment products to provide an evaluation of students' learning throughout the semester (see Figures 3a & 3b).

Data concerning the effectiveness of the asynchronous discussion facility in ME330 was provided by students' responses to assessment item A4, the reflective paper. A4 contained students' responses to the task of evaluating the educational effectiveness of the instructional approach used in the Unit. These data were analysed to identify the key issues raised by students in relation to the asynchronous discussion and the constructivist pedagogy (see Figure 4).

Twelve, final year, undergraduate students were enrolled in the Unit: seven were male and five were female. All save two students were taking the Unit in the on-campus mode. One of the off-campus, male students was repeating the Unit. The students were all experienced computer users, although none had used asynchronous discussion for educational purposes. While all class members were familiar with constructivist theory and its implications for instructional design, their undergraduate experience had largely exposed them to traditional, lecture-tutorial/laboratory teaching and learning environments.

Space constraints preclude a detailed presentation of the data analysis, but Figures 3a and 3b present a summary of the findings relating to the relationship between students' SOLO levels, their engagement with content, and their final grades for Unit ME330. Figure 3 shows that one student (S5) achieved the extended abstract (b) level; six reached extended abstract (a) level; one reached relational (b) level; two reached relational (a) level; and two students (S10 & S12) failed to demonstrate that they had reached the relational (a) level of conceptual understanding. Students S10 and S12 did not pass the Unit and they had the lowest levels of engagement with content in the class. Of the seven students who developed extended abstract SOLO levels; three (S3, S4 & S5) demonstrated very high levels of engagement with content, and two (S3 & S5) were awarded high distinctions for their final grades. S7 was the only student at the relational (b) level and this student received the only credit final grade.

Figure 3a: Learning Evaluation: SOLO Level, Engagement Level & Final Grade

Figure 3b : ME330 Learning Evaluation Rankings (1=high to 5=low)

Table 3b ranks the 12 students (1=high to 5=low) in the three categories of SOLO level, engagement level and final grade, with ties. A Gamma test was undertaken to determine the degree of agreement between the rankings for SOLO and Engagement, and the rankings for SOLO and Final Grade. (Use of the Gamma statistic is preferable to Spearman R or Kendall tau, when the data contain many tied observations.) The analysis gives g = 0.950 (p = 0.0008) for the SOLO-Engagement rankings and g = 0.956 (p = 0.0002) for the SOLO-Final Grade rankings. This demonstrates that, for Unit ME330, there was a close, positive association between students' SOLO levels, their engagement with content, and their final grades. This held for all students, except one (S11), a bright student who chose not to participate in group-work or contribute to the online discussions.

Assessment task A4 part (b) ("Reflections on Unit ME330") provided the instructor with detailed and constructive feedback from students on the educational effectiveness of the approach taken in Unit ME330 and the use of online asynchronous discussion. The analysis of A4 assessment documents revealed that although several students expressed a preference for traditional teaching approaches, there was strong support for the ME330 problem-based teaching and learning environment. As one student (S6) noted:

Figure 4 provides a summary of the main issues raised by students in the reflective paper assessment.

Figure 4: Students' Reflections on ME330

Extracts from two A4 papers are presented below to provide a flavour of students' experience of the Unit. S9 had failed the unit in 1999 and was repeating it in off-campus mode; S5 was the most academically talented member of the class.

Continuous access to the textual record of discussions enabled the instructor to develop an accurate knowledge of each student's grasp of the subject matter. This understanding proved extremely beneficial as it allowed her to use the online discussion facility to provide timely and relevant feedback to individual students, small groups and the entire class as required. The discussion records provided a mirror to students' thinking processes and the online facility supported an open, two-way communication between the instructor and her class members. To what extent this would be feasible with a large class is unclear, as facilitating and monitoring structured online discussions proved time consuming.

From the instructor's perspective, the problem-based instructional design was a very effective way to deliver a dynamic and relevant program. The extensive use of online asynchronous discussion provided the instructor with insights about students' learning that improved her responsiveness in meeting students' needs for guidance and support, and resulted in a rewarding teaching experience.

This case study has examined the application of a constructivist theory to an online collaborative learning environment that employed a well-structured asynchronous text-based discussion strategy. The research findings support the view that embedding student collaboration and problem-based learning in asynchronous text-based discussion provides teachers with a powerful strategy to support students' active engagement with content and facilitates the development of high levels of conceptual understanding for students. Further research is required to test this approach to teaching large classes, and in a range of academic disciplines.

Barrows, HS (1986) A taxonomy of problem-based learning methods, Medical Education, 20(6), 481-486.

Biggs, JB (1999) Teaching for Quality Learning at University, Buckingham: SRHE & OUP.

Biggs, JB (1992) A qualitative approach to grading students, HERDSA News, 14(3), 3-6.

Biggs, JB & Collis, KF (1982) Evaluating the Quality of Learning: the SOLO Taxonomy, New York:Academic Press.

Bloom, BS, Engelhart, MD, Furst, EF, Hill, WH & Krathwohl, D (1956) Taxonomy of educational objectives: the cognitive domain, New York:McKay.

Boulton-Lewis, GM (1995) The SOLO Taxonomy as a Means of Shaping and Assessing Learning in Higher Education, Higher Education Research & Development, 14(2), 143-154.

Boulton-Lewis, GM (1998) Applying the SOLO Taxonomy to Learning in Higher Education, in Dart, B & Boulton-Lewis, G (Eds) Teaching and Learning in Higher Education, Melbourne:ACER.

Brown & Atkins, (1988) Effective Teaching in Higher Education, London:Routledge.

Bruner, JS (1985) Models of the Learner, Educational Researcher, 14, 5-8.

Collins-Brown, E (1999) Effective Pedagogies for Managing Collaborative Learning in OnLine Learning Environments, Educational Technology & Society, 2(2, 12-19.

Entwistle, N & Ramsden, P (1983) Understanding Student Learning, London:Croom Helm.

Kolb, D (1976) Learning Style Inventory, Boston:McBer & Co.

Marton, F & Saljo, R (1988) On qualitative differences in learning, Br. J. Educational Pyschology, 46, 4-11.

Marton, F & Saljo, R (1975)

Miller, SM & Miller, KL (1999) Using Instructional Theory to Facilitate Communication in Web-based Courses, Educational Technology & Society Vol.2 (3) 106-114.

Ramsden, P (1992) Learning to teach in higher education, London: Routledge.

Rossman, MH (1999) Successful Online Teaching Using An Asynchronous Learner Discussion Forum, JALN Vol.3 (2) 91- 97.

Schmeck, R, Ribich, F & Ramanaiah, N (1977) Development of a self-report inventory for using individual differences in learning processes, Applied Psychology Measurement, 1, 413-431.

Tam, M (2000) Constructivism, Instructional Design and Technology: implications for transforming distance learning, Educational Technology & Society Vol.3 (2) 50-60.

Whittle, JI, Dal Pozzo, E & Gregory, M (in press) Online Collaboration, ASCILITE 2000 Conference Proceedings, December, 2000.

Vygotsky, (1962) Language and Thought, Cambridge MA: MIT Press

Jan Whittle (BA, DipEd, MEd (Adelaide))

School of Multimedia & Information Technology

Southern Cross University, Coffs Harbour 2457 NSW Australia