Overview

Project Brief
Project Summary
Publications
Project Team and Contributors

In a climate where individual institutions are experiencing increased costs at the same time as they face increased demand for more flexible approaches to learning, the Australian Universities Teaching Committee (AUTC) considers there is benefit to be gained in developing shared resources and disseminating successful, generalisable templates between institutions. One product of this assessment is a project, now completed, which is captured here as 'Learning Designs'.

The project was commissioned in 2000 by the AUTC to explore the use of Information and Communication Technologies (ICTs) to facilitate flexible learning opportunities for students by identifying learning designs that have been demonstrated to contribute to high quality learning experiences and determining which learning designs may be redeveloped in a more generic form.

The project commenced in November 2000. Following are extracts from the Final Report of the project, produced in December 2002. These provide a concise picture of project aims, activities, deliverables and recommendations made for further research and development.

Project Aims

This Australian University Teaching Committee (AUTC) project aimed to produce generic/reusable learning design resources to assist academics to create high quality, flexible learning experiences for students. This was achieved by:

1. Identifying high quality learning designs used in higher education;
2. Selecting those suitable for redevelopment in the form of reusable software, templates and/or generic guidelines; and
3. Developing these reusable resources and making them accessible from a central web site.

The term "learning design" refers to a variety of ways of designing student learning experiences, that is, the sequence of activities and interactions. The scope of a learning design may be at the level of a subject/unit or subject/unit components. This project focuses on learning designs implemented with the use of ICT and how flexible learning opportunities for students can be afforded through the use of such technologies. The composition of a learning design, particularly when ICT mediated, has been informed by the work of Oliver (1999) and Oliver and Herrington (2001). Thus, for the scope of this project, a learning design comprises three key elements: the content or resources learners interact with, the tasks or activities learners are required to perform, and the support mechanisms provided to assist learners to engage with the tasks and resources.

"High quality learning experiences" refer to experiences resulting from an environment which encourages students to seek understanding rather than memorisation and which encourage the development of lifelong learning skills.

"Flexible learning" refers to an educational approach that meets the diverse needs of students. This project focused on how ICT can be used to design flexible opportunities for students.

Project Activities

Development of a set of principles for high quality learning in higher education

A crucial aspect of this project was the development of a mechanism to identify high quality learning designs. Two leaders in the field of learning in higher education (Professor David Boud and Associate Professor Michael Prosser) were commissioned to advise the project team about what constitutes high quality learning in higher education.

High quality student learning is purported to have potential to be achieved by ensuring four key principles (Boud & Prosser, 2001, 2002) are operationalised in a learning environment: engaging learners, acknowledging context, challenging learners, and providing opportunities for practice.

Development of an Evaluation Instrument - Evaluation and Redevelopment Framework (ERF)

The "Boud and Prosser principles" formed the basis of an evaluation instrument referred to as the Evaluation and Redevelopment Framework (ERF). Its purpose is to identify ICT-based learning design exemplars that foster high quality learning experiences, and to determine whether these exemplars have potential to be redeveloped in a more generic/reusable form.

Application of an evaluation instrument to Learning Design Exemplars

The project identified 52 ICT-based learning design exemplars and 28 were selected for evaluation. An ERF evaluation team (64) comprised international educational technology and/or pedagogy expertise. Two evaluators were allocated to conduct an evaluation for each learning design exemplar. The results of the evaluation phase assisted the project team to select a number of exemplars suitable for redevelopment in a more generic form.

Development of reusable Learning Design resources

Fifteen learning design exemplars were identified as suitable for redevelopment in a more reusable form. Redevelopment has taken two forms:

1. Guidelines: The generic learning design is abstracted from the exemplar and described in detail. Implementation tips and advice on how to develop a similar learning design are also included.
2. Software tools: Generic software tools (with accompanying documentation for use) have been developed to facilitate the reuse of particular learning designs.

Documentation of Learning Design Exemplars

Whilst the evaluation process served to identify a selection of learning designs suitable for redevelopment in the form of generic guidelines and/or software tools, the rich resource of learning design exemplars collected by the project was deemed valuable for inclusion in the form of "rich descriptions" of implemented learning designs. The project team felt that these descriptions could serve as implicit guidelines as academics may be able to abstract for themselves the generic design from these contextualised descriptions.

A Learning Design Exemplar Description template was devised which designers were requested to complete.

Development of Project Web Site to house the project deliverables

A project web site has been developed to facilitate dissemination of the project deliverables.

In order to facilitate easy and meaningful access to the reusable learning design resources and learning design exemplars, interviews with potential end users were conducted and a Learning Design Classification Framework was devised (based on a grounded approach). The Framework incorporates the following Learning Design types/foci:

PROBLEM-BASED LEARNING focus: The emphasis of the learning design is on the process of students solving a real world problem presented to them.

PROJECT/CASE STUDY focus: The emphasis of the learning design is to create a product or artefact. The "learning by doing" process may be supported by case materials from which the learner can distil/abstract lessons learnt to apply in a new project situation.

ROLE-PLAY focus: The emphasis of the learning design is subrogation: "walking in the shoes of others".

COLLABORATIVE focus: The emphasis of the learning design is interacting and collaborating with peers to facilitate construction of knowledge.

CONCEPT/PROCEDURE DEVELOPMENT focus: The emphasis of the learning design is to understand and/or consolidate student learning about concepts and/or procedures.

Project Deliverables

Key outputs from this project include:

• The Boud and Prosser principles for high quality student learning in Higher Education (Boud & Prosser, 2001, 2002).
• The Evaluation and Review Framework (ERF).
• The Learning Design construct (Oliver, 1999, 2001; Oliver & Herrington, 2001).
• The Learning Design Sequence construct.
• A Learning Design Classification Framework.
• Four generic software tools:
  • Investigation eShell.
  • Predict-Observe-Explain eShell.
  • Online Self and Peer Assessment Tool.
  • VisChem Molecular–Level Construction Tool.
• Generic guidelines to assist the design, development and implementation of the
  following learning designs:
  • Explore, Describe, Apply: A problem focussed learning design.
  • En-Role, Research, React, Resolve, Reflect: Developing and using online role-play learning designs.
  • Review, Select, Compile and Argue: A problem focussed learning design.
  • Review, Access, Question, Decide, Report, Reflect: Structured Problem Solving.
  • Observe, Represent, Refine: Developing scientifically-acceptable mental models of non-visible physical phenomena (based on VisChem).
• Learning Design Exemplar descriptions (32): Each description is accompanied with its own Learning Design Sequence. These descriptions can serve as implicit guidelines.
• Web Site to facilitate dissemination of project deliverables.

Recommendations for further research and development

A number of further issues flow from the project. Some of these recommendations relate to support for use of the resources, some related to evaluation of the effectiveness of the resources and others to research agendas. The project team recommend: -

1. A system be set up so that the design base can be added to following review and processing of learning designs in the format adopted by this project.
2. An extensive evaluation be carried out on the site and the guidelines to ensure they can be used and that they work when they are used.
3. A dissemination process be put in place to assist staff development programs, and to support a community of practice among users with mechanisms for users to report outcomes and refinements to the designs.
4. A process be established to explore metadata descriptors for learning designs in line with the current standards movement.
5. Mechanisms be established to determine the quality of learning designs developed from the project resources, possibly through peer review.
6. Effective linking of the project to other related international projects, such as the SOuRCE Project, be developed.
7. An electronic performance support system (EPSS) be developed to support users in the design process.
8. The formalism used in this project to represent learning sequences be further explored.

Among important project activities were developing various reports, communicating about progress and disseminating project outcomes. Reminders of these activities are captured below, where you may skip directly to either of the following:

• Conference Papers/Presentations
• Project reports and Publications

Conference Papers/Presentations

Conference presentations are listed in reverse chronological order below.

Australasian Society for Computers In Learning In Tertiary Education (ASCILITE) 2002

Shirley Agostinho presented a paper that illustrated the feedback received on the use of the ERF (the project's evaluation instrument) and explained how this evaluation instrument could be resued by the academic community for formative and summative evaluation purposes when designing and implementing learning designs.

Agostinho, S, Oliver, R., Harper, B., Hedberg, H., & Wills, S. (2002). A tool to evaluate the potential for an ICT-based learning design to foster "high-quality learning". In A. Williamson, C. Gunn, A. Young., & T. Clear (Eds.), Winds of change in the sea of learning. Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education. (pp. 29-38). Auckland, New Zealand: UNITEC Institute of Technology.

Reusable Learning Designs: opportunities and challenges
International Conference
sponsored by The Australian Universities Teaching Committee (AUTC),
Thursday, 5 December 2002

The Core Team: Barry Harper, Ron Oliver, John Hedberg, and Sandra Wills presented the outcomes of this project at this conference.

Powerpoint Presentation is available here. (Filename: AUTCICTProject.ppt)

Education Week 2002
- Flinders University, November 2002

Barry Harper gave a presentation about the project and outlined how the project team believes this project will influence learning in higher education in the future.

ICT-Ed Evaluation Workshop,
University of Canberra, September 30, 2002.
http://cerg.infotech.monash.edu.au/icted

Team member: Associate Professor Catherine McLoughlin, ACU presented the following paper at this conference:

McLoughlin, C. (2002). AUTC Project: ICT's and their role in flexible delivery.

The main question asked was:

• Have the projects listed at this URL that involve ICT been considered as part of the learning objects research?

Teaching Online: ACT Showcase- teaching with a range of educational technologies,
Conference held at Australian National University, September 2002

Team member: Associate Professor Catherine McLoughlin, ACU presented a paper titled: "AUTC Project: Development of generic learning activity designs" at this conference. There was great interest in this project with the following questions asked :

• How have/will students be involved in evaluating the design of the learning
  designs and will they be trialled with students before being put on the
  Website?
  Answer from Project Manager: This is outside the scope of this project, but it's an important issue that the proejct team needs to consider once the project is completed.
• Can we use the ERF and where can we find it?
  Answer from Project Manager: Certainly! It is a tool free available for use.
• Is this a finite list of online designs? If so, how do you know?
  Answer from Project Manager: Yes, this will be explicitly stated on the final project web site.

The Higher Education Research and Development Society of Australasia (HERDSA) 2002

Oliver, R., Harper, B., Hedberg, J., Wills, S., & Agostinho, S. (2002). Formalising the description of learning designs. In A. Goody, J. Herrington, & M. Northcote (Eds.), Quality conversations: Research and Development in Higher Education, Volume 25 (pp. 496-504). Jamison, ACT: HERDSA.

Oliver, R., Herrington, J., & McLoughlin, C. (2002, July). ICT-based learning designs. Poster session presented at the 2002 Annual International Conference of the Higher Education Research
and Development Society of Australasia (HERDSA), Perth, Australia.

(Photo of poster is available from "Project Pics".)

EDMEDIA 2002

John Hedberg and Ron Oliver presented the following paper at this conference.

Hedberg, J., Wills, S., Oliver, R., Harper, B. & Agostinho, S. (2002). Developing Evaluation Frameworks for Assessing Quality ICT-based Learning in Higher Education. In P. Barker & S. Rebelsky (Eds.), Proceedings of ED-MEDIA 2002, World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 729-735). Denver, Colorado, USA: Association for the Advancement of Computing in Education.

Australasian Society for Computers In Learning In Tertiary Education (ASCILITE) 2001

Ron Oliver and John Hedberg explained the project and presented a project update.

Harper, B., Oliver, R., & Agostinho, S. (2001). Developing generic tools for use in flexible learning: A preliminary progress report. In G. Kennedy, M. Keppell, C. McNaught & T. Petrovic (Eds.), Meeting at the Crossroads. Proceedings of the 18th Annual Conference of the Australian Society for Computers in Learning in Tertiary Education. (pp. 253-262). Melbourne: Biomedical Multimedia Unit, The University of Melbourne.

World Conference on Educational Multimedia, Hypermedia & Telecommunications (EDMEDIA) 2001

Barry Harper presented a brief overview of the project and how the project facilitates potential international collaboration amongst university institutions.

Harper, B., O'Donoghue, J., Oliver, R., & Lockyer, L. (2001). New designs for Web Based Learning environments. In C. Montgomerie, & J. Viteli (Eds.), Proceedings of ED-MEDIA 2001, World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 674-675). Tampere, Finland: Association for the Advancement of Computing in Education.

Open and Distance Learning Association of Australia (ODLAA 2001)

Barry Harper presented aspects of the project as part of a keynote presentation on the research and development issues of online learning. Many of the participants at this conference are part of the project’s Evaluation Team thus it was a good opportunity to discuss the ERF.

The 9^th Improving Student Learning Conference,
Heriot-Watt University, Edindurgh, Scotland

Ron Oliver presented a paper at this conference whereby he briefly described this project. He also had the opportunity to discuss this project with the SoURCE project team who attended this conference.

National Council on Open and Distance Education (NCODE) (now ACODE)
- Flexible Learning Australasia 26, 23 - 24 July 2001

Barry Harper and Sandra Wills presented a progress report to NCODE members at their meeting in Adelaide, at the University of South Australia on Tuesday, 24 July 2001. The presentation gave NCODE members an overview of the project objectives and current status and illustrations of the application of the project’s ERF.

Support from members was sought in identifying appropriate evaluators and learning designs which could yield generic designs.

The NCODE participants responded very positively to the project, supporting the pedagogical emphasis and commending the progress made. Members expressed the need for such a project and stated their strong support for offering names of possible evaluators and learning designs to evaluate. They also encouraged further reporting of the progress of the project at subsequent NCODE meetings.

The outcome from this presentation resulted in 18 ICT-based learning products and/or settings being nominated for the project to pursue plus 25 ERF Evaluator nominations.

Project Reports and Publications

Following are two key publications and three informal research reports (among several) prompted by the project:

Boud, D., & Prosser, M. (2001, April). Key principles for high quality student learning in Higher Education—from a learning perspective. Paper presented at a workshop held on April 27, 2001 for the AUTC funded project: Information and Communication Technologies and Their Role in Flexible Learning, Sydney, Australia.

This paper forms the basis of the project's evaluation instrument: Evaluation and Redevelopment Framework that was designed to facilitate the identification of high quality learning designs and to determine if such learning designs have the potential for redevelopment in a generic/reusable form.

Boud. D., & Prosser, M. (2002). Key Principles for High Quality Student Learning in Higher Education: A framework for evaluation. Educational Media International, 39(3), 237-245. [Publisher's website]

The paper builds on their previous paper about the principles for high quality learning in higher education from a learning perspective.

Herrington, J., & Oliver, Ron. (2002, March). Identification of generic ICT-based software tools. Unpublished manuscript. [.pdf copy]

This report summarises generic ICT-based tools that currently exist for use in flexible learning environments.

Herrington, J., McLoughlin, C., & Oliver, R. (2001). Pedagogical strategies employed in technology-based learning projects. Unpublished manuscript. [.pdf copy, Part 1 ; .pdf copy, Part 2]

This paper provides a summary of the literature review about the forms of technology-based learning being reported in the literature with the view to discovering patterns and themes in the forms of learning strategies being employed, the enhanced learning outcomes being reported and the nature of the evaluation methodologies employed.

Oliver, R., McLoughlin, C., & Herrington, J. (2001, April). Review of Evaluation Frameworks. Paper presented at a workshop held on April 27, 2001 for the AUTC funded project: Information and Communication Technologies and Their Role in Flexible Learning, Sydney, Australia. [.pdf copy]

This report summarises the findings of a literature review about the current evaluation tools employed in ICT-based learning environments.

Critical to the success of this project was the willingness and enthusiasm of the higher education academic community to participate. The project delved into issues concerning reusability of existing high quality learning designs across institutions, without the participation of the designers of such learning environments, the outcomes of this project would not have been possible.

The team experienced an overwhelming willingness by the academic community to participate in the project. We are indeed grateful for the significant contribution made by this community. We would like to particularly thank two groups of people. Firstly, to the evaluators who participated in evaluating the learning design exemplars collected by this project, - thank you for your time to conduct the evaluation and the valuable feedback submitted about the evaluation process. Secondly, to the designers who submitted their learning design exemplars to this project – thank you very much for your willingness to share your experience!

This project has involved the work of approximately 140 people who have contributed in various capacities. The following diagram indicates most of the contributing groups, highlighting some of the key people involved.

The full list of contributors includes:

Michael Adams, Shirley Agostinho, Anne A'Herran, Shirley Alexander, Frances Alter, Max Angus, Roger Atkinson, Chuda Basnet, Jim Beck, Maureen Bell, Sue Bennett, Chris Bigum, Curtis Bonk, David Boud, Gary Brierley, Sam Bucolo, Robert Cannon, Helen Carter, Beth Cavallari, Lynne Chapman, Penny Collings, Betty Collis, Bob Corderoy, Jenny Cox, Michael Crock, Barney Dalgarno, Wim de Boer, Michelle de Souza, Ann Deden, Liz Devonshire, Roger Dickinson, Carolyn Dowling, Andrew Dunbar, Erik Duval, Allan Ellis, Peter Evans, Fran Everingham, Michael Fardon, Niki Fardouly, George Fernandez, Mark Freeman, Paul Fritze, Georgina Fyfe, Tanja Golja, Maree Gosper, Halima Goss, Lewis Gratton, Jan Gray, Diane Hansford, Barry Harper, John Hedberg, Jan Herrington, Jo Hetland, Lindsay Hewson, Mick Hillman, Garry Hoban, Dale Holt, Chris Hughes, Albert Ip, Kel Jackson, Wanda Jackson, Marie Jasinski, Sue Johnston, Matthew Kearney, Peter Keeble, Gregor Kennedy, Mike Keppell, Denise Kirkpatrick, Tony Koppi Irene Kreis, Diana Laurillard, Roni Linser, Lisa Lobry de Bruyn, Lori Lockyer, Joe Luca, Mary Jane Mahony, Inge Matt, Iain McAlpine, Rob McLaughlan, Catherine McLoughlin, Sue McNamara, Kim McShane, David McSwan, Louise Manner, Jim Meek, Rohan Miller, Christopher Morgan, David Murphy, Karl Mutimer, Som Naidu, Ted Nunan, Linda O'Brien, John O'Donoghue, Simon O'Mallon, Roisin O'Reilly, Ron Oliver, Martin Olmos, Debra Panizzon, Elizabeth Patterson, Russ Pennell, Robyn Philip, Michael Prosser, John Reeves, Thomas C. Reeves, Peter Remfry, Mary Rice, David Rich, Lesley Richardson, Karl Rudd, Greg Ryan, Joan Salmon, Elizabeth Santhanam, Chuck Schneebeck, Sandy Schuck, Nathan Scott, John Shepherd, Rod Sims, Alan Smith, Robyn Smyth, Elizabeth Stacey, Brian Stone, Sue Stoney, Roy Tasker, Jim C. Taylor, Peter Taylor, Peter Twining, Andrew Vincent, Manjula Waniganayake, Thomas West, Barbara White, Gerry White, Martyn Wild, Marilynn Willis, Sandra Wills, Audrey Wilson, Janis Wilton, Jonathon Wright, Rob Wright and Samina Yasmeen.

We hope that the outcomes of this project serve as an impetus for uptake of ICT-based learning designs and that the concept of sharing amongst the academic community is further encouraged by future projects of this nature.

Your feedback is important in enabling us to review this innovative resource. Please submit any comments you have to dmc@uow.edu.au or contact the Digital Media Centre on +61 2 4221 3891.