ORIGIN OF THE LEARNING DESIGN
The original concept was conceived by Teng Liaw and Ruth McNair from the Department of General Practice at The University of Melbourne. Funding for the initial development was secured through the University's Teaching and Learning (Multimedia and Educational Technology) Committee in 1998. In association with Mike Keppell and Gregor Kennedy from the Biomedical Multimedia Unit the instructional and conceptual design of the program was mapped out based on the concept of a decision tree with branching pathways. After an initial implementation further funding was secured through the University's Teaching and Learning (Multimedia and Educational Technology) Committee in 1999 and the design and architecture of the program was greatly revamped and extended by Terry Judd and Gregor Kennedy of the Biomedical Multimedia Unit. Using this architecture, John Marty from the Department of General Practice was instrumental in the development of the second case and the introductory tutorial for the program.

TIMES THE LEARNING DESIGN HAS BEEN USED
Two versions of the program have been developed and implemented with first year medical students at the University of Melbourne. Version 3.0 is set to be implemented in April 2003.

MODIFICATIONS SINCE FIRST USE
The program was modified after its initial implementation which was the subject of an extensive formative evaluation (Liaw, T., Kennedy, G., Keppell, M., Marty, J., McNair, R., 2000). A number of concrete recommendations emerged from this evaluation which included:

• shortening the interview.
• "chunking" interview segments to address continuity and repetition.
• the inclusion of a site map and "go back" and "replay" buttons to address discrimination.
• revamping the interface to improve navigation and functionality.
• adding volume checks and controllers on interface.

Substantial changes were also made to the project's software architecture. Whereas Version 1 relied on a 'hard-coded' interview structure with minimal organisation of the external resources (QuickTime® audio and video), Version 2 adopted a database of resources and structured interview 'definition' files that could be readily modified using a developer version of the software. The ability to add additional interviews (scalability) was also included.

Reference:

Liaw, T., Kennedy, G., Keppell, M., Marty, J., McNair, R. (2000). Using multimedia to assist students with communication skills and biopsychosocial integration: An evaluation. Australian Journal of Educational Technology, 16 (2) 104-125.

DISSEMINATION
The learning design has not been reused or redeveloped for use in other contexts.

The University of Melbourne is planning to launch a commercial version of the Communicating with the Tired Patient in 2003.

RESEARCH CONDUCTED ON THE DESIGN
The program was modified after its initial implementation which was the subject of an extensive formative evaluation (Liaw, T., Kennedy, G., Keppell, M., Marty, J., McNair, R., 2000). The evaluation was based on the model developed by Gregor Kennedy (Kennedy, 2003 in press) and employed an iterative mixed method approach. The final phase of the evaluation involved qualitative and quantitative data collection from 110 first year medical students. A number of concrete recommendations emerged from this evaluation which included: shortening the interview; "chunking" interview segments to address continuity and repetition; the inclusion of a site map and "go back" and "replay" buttons to address discrimination; revamping the interface to improve navigation and functionality and adding volume checks and controllers on interface.

A second evaluation was completed with Version 2.0 which considered the integration of the program into the curriculum and an initial investigation of how students were completing their interviews. Data collection was through student tracking (audit trails) and an electronic questionnaire. The data from this evaluation are currently being analysed.

Preliminary findings show that the program was accessed 324 times in the academic year after its implementation in March, 2001. The majority of use was in the final weeks of March when the program was recommended as a resource in the medical curriculum. This period accounted for 53.7% of total program usage over the academic year and 87.5% of first year students accessed the program.

A more detailed analysis suggests that students were using the program in a variety of ways, some of which were unexpected by developers (see below). Initial findings show that, 72.2% of students who accessed the program completed a meaningful interaction (attempted or completed an interview) and 43.1% of students completed at least one interview.

Reference:

Kennedy, G.E. (2003, in press). An institutional approach to the evaluation of educational technology. Educational Media International.

DESIGN EFFECTIVENESS VERSUS INTENDED OUTCOMES
Communicating with the Tired Patient is one of many resources available to students to assist them develop their knowledge and skills in clinical interviewing. While no evaluation has directly assessed the learning outcomes of the program, this would be a difficult evaluation to design given the program's close integration with the broader curriculum.

However, in the first evaluation students were asked how the program helped them learn about communication and interview techniques. The five most common responses by students are reported below.

1. My understanding of the use of questions, language and communication in general was improved: 53
   
2. I didn't learn much new; the program reinforced material from the tutorials: 12
   
3. I had an improved idea of the general process of interviewing: 7
   
4. I learnt more about observing body language and non-verbal cues: 6
   
5. I had an improved understanding about dealing with Non English Speaking Background patients: 4

The results of the evaluations have indicated that many students are using the program and while some spend a limited time interacting with it, the majority complete a meaningful interaction. This is to be expected for this type of self-directed learning resource which is not mandatory for students.

It is our opinion, therefore, that the program contributes to students' understanding of clinical interviewing skills in an integrated problem-based medical curriculum.

UNEXPECTED LEARNING OUTCOMES
This is not really an unexpected learning outcome but an unexpected learning process. In the first module of the program students are required to play the role of a doctor and to listen to three or four potential ways of asking patients questions. After listening to each audio question students select the one they think is the most appropriate given their location in the interview. The program was designed in this way so that students would focus on the tone, structure and content of different questions in order to determine their appropriateness. The evaluation showed that a small cohort of students was not reviewing each audio option and subsequently making a selection; rather they were selecting each audio option in turn and making their judgments on the basis of both the doctor's question and the patient's response. This, to a certain extent, undermined the developers' vision but nonetheless must be regarded as a strategic approach to determining the appropriateness of different clinical questioning techniques.

HOW LEARNER ENGAGEMENT IS SUPPORTED
The learning design supports engagement predominantly through the use of a situated model and role-play.

We propose that by giving students a meaningful task (to carry out a clinical interview), by asking them to adopt a specific role (in this case that of a doctor) and by providing them with a degree of control (they direct the interview given a number of options) their engagement will increase. In addition, we propose that engagement is fostered by asking students to reflect on specific components of their own interview.

It is clear from our observational studies that some students become affectively involved in the program. From the first evaluation it was noted that students spent a lot of time reviewing the audio options and some students closed their eyes when listening to the audio in order to focus on the difference between the audio options. One student was heard to say "No, no, no. I don't want you to say that!" after making an audio choice that she was unhappy with. Students were also observed concentrating fiercely on the video responses of the patients in order to pick up both verbal and non-verbal cues. These observations suggest that the audio and video were useful in engaging students in the content of the package.

Comment from the Evaluation Team...

The evaluation of this exemplar concluded that learner engagement was supported reasonably well in this learning design. Sample feedback included:

"Excellent level of engagement which was achieved through:

• the situated professional context;
• the ability to choose scenarios and investigate different approaches;
• the provision of instant and informative expert feedback;
• the self-reflective questions and comments;
• relevance to the assessment tasks and accompanying coursework activities".

ACKNOWLEDGMENT OF LEARNING CONTEXT
The learning design quite clearly reflects the professional practice students will engage in during their clinical years and after graduation. The cases used in the clinical interviews were specifically selected to provide students with a mix of ethnicity, gender and age. Students can choose to be either a male or a female doctor reflecting the gender balance both in the cohort of students and the profession. The use of a variety of cases in the final version of the program will encourage students to consider how interviewing skills may need to be modified to suit different patients and situations.

Like the learning design, students' summative assessment in Introduction to Clinical Medicine (ICM) is authentic. Students are assessed through a video-taped interview of a simulated patient which is marked by observing examiners. Students are asked by the examiner immediately following the interview to reflect on their skills and comment on their interview. They then receive feedback from the examiner and patient, regarding their communication skills and the degree of rapport developed. The learning design of Communicating with the Tired Patient encourages reflection, partly by introducing the terminology used in reflection within the content expert comments.

Comment from the Evaluation Team...

The evaluators concluded that the learning design acknowledged the learning context well. Comments include:

"The focus of this program was to practise specialised professional skills. Links to broader contexts etc were not applicable."

"Breadth of experience was required and this was achieved through the range of examples provided. Links to key concepts and ideas were provided as appropriate."

"The content of the conversation is well scripted to reflect a realistic situation that offered many areas for exploration."

HOW THE LEARNING DESIGN CHALLENGES LEARNERS
Students are challenged predominantly through the use of self-reflection. Students are typically asked between 10 and 15 questions throughout their interview which are directed at the decisions they have made and the responses evoked in the patient. By giving students control over their interview they are challenged to conduct a clinical inquiry in an appropriate manner, relying on their own judgements and intuitions.

Comment from the Evaluation Team...

This point is iterated in the evaluation feedback:

"The intent was to illustrate different approaches to a clinical interview and their consequences. This was done particularly well in a non-confrontational way, that would allow for self reflection and behaviour modification if necessary. This was an excellent way of helping students to confront and challenge their own communication styles."

OPPORTUNITIES FOR PRACTICE
The "practice" of clinical interviewing is epitomised in the learning design of Communicating with the Tired Patient.

Students use the resource to complement their ICM tutorials and this is where they are required to directly practise interviewing skills. Again, a problem-based curriculum requires that students be mindful of issues associated with clinical interviewing on a weekly basis when they receive a new clinical case. Socio-cultural issues are often important attributes of the weekly clinical problems and students are often challenged to negotiate these issues.

Students receive feedback about their interviewing techniques and decisions in the program itself (through expert feedback and the interview summary) but they also receive feedback on their attitudes, perceptions and skills in clinical interviewing in other areas of the course (ICM tutorials and in their PBL tutorials).

Comment from the Evaluation Team...

It is important to note that the learning design includes the use of the interactive multimedia program plus the face-to-face tutorials. Thus, in this learning design students are exposed to multiple perspectives of practice. For example, the evaluators commented that models of "good practice" can be examined in the program:

"Models of good practice can be discerned from the scenarios; the expert comments and questions identify issues and elements of good practice; feedback is instantaneous and helpful; self-reflection is encouraged; causal relationships in terms of human responses can be investigated."

Furthermore, students then directly practise their interviewing skills in the face-to-face class sessions.