DISCIPLINE
Engineering

DURATION
Students are asked to spend a number of hours per week solving the problems – the number depends on the course. In our large first year subject Engineering 101, students spend three hours in lectures each week and also 3 hours in the computer lab.

Although our courses tend to consist of weekly topics, and although there are deadlines each week, in a sense this learning design is actually for a whole semester. The topics are not independent as students must use and re-visit earlier ideas while solving later problems.

ICT USED
The ICT is a custom web-based system called FlyingFish. FlyingFish is a web server that runs on a Windows computer on campus. It has many custom features to support the tutorial environment described here.

Students can use any modern web browser, on either Windows or Macintosh, and they do not have to have any special CD or installer. The system is truly web-based. The student must only enter the address of the server, from home or in our computer lab, and they can log in.

The client (student) end in some courses does make use of Java™ applets so the web browser must be Java enabled.

DELIVERY CONTEXT
Our tutorials are defined by the computer tutorial system. However there is still a traditional lecture series, delivered at a pace that matches the tutorial deadlines. Tutors are also present at the advertised tutorials. We warn students not to try to work entirely alone or from home as they would miss the valuable help of peers. So this learning design is really a face-to-face one with solid support from a custom-built computer system. We have no plans to reduce the current amount of face-to-face teaching even though we might get away with it.

TARGET AUDIENCE
First-year undergraduate

The University of Western Australia attracts an unusually large proportion of direct high school leavers. We have mainly 17 and 18 year old students. The form of our tutorial system reflects this to some extent (rigid, regular, short-term deadlines). Our students come from the very top of the TER band and our entry scores sometimes exceed those for Medicine. The students are therefore smart and able, but sometimes need help to set priorities and learn consistently through semester. We expect students to be good at maths and to have done some Calculus but in fact we find the general background in this area is weak, and we think many of them learn the Calculus concepts for the first time early in our course (by solving the problems).

COHORT
This learning design has been used in the course Engineering 101, which has up to 520 students. This is not the upper limit; we could probably cope with more if necessary.

Students attend the computer lab in groups of nominal size 64 or 128, depending on lab availability. Tutors are assigned to the lab according to the expected number of students. The students have the freedom to choose when they will go to the tutorials, so some classes during the week get very full. We respond to this by assigning more tutors to those popular times. In a very busy morning class with 128 seats, we might assign 3 tutors. An unpopular afternoon class with 64 seats will get either one tutor or it will be cancelled.

We are able to have a high student-staff ratio because some common student errors are identified automatically by the computers, and there is also an online FAQ (bulletin board) system called the Forum.

BROADER CONTEXT
Engineering graduates must have some generic skills such as the ability to work in teams, to communicate and so on. However they also must have the ability to solve some technical problems, or at least to learn how to solve technical problems. The foundations, the early years, of the engineering degree programs are thus quite strongly based around problem-solving classes. Students learn to solve problems in Dynamics, in Maths, in Physics and Electronics.

The learning design reported here is repeated 3 or 4 times out of 8 courses taken in the first year: Engineering 101, Maths 131 and 132, and Engineering Dynamics 106. It appears again in two later-year subjects: Applied Thermodynamics 200 and Thermofluids 311.

Students in the first year encounter a similar learning design in Physics (small problem classes) and in Electronics (the Mallard computer tutorial system from UIUC).

• Ability to predict the short-term behaviour of simple physical systems in motion using taught methods, models and equations.
• Ability to work in groups to solve problems, and the ability to seek help productively.
• Ability to write a clear and logical solution in a form suitable for an examiner to mark.

IMPLEMENTATION OF ASSESSMENT STRATEGIES
Most of the assessment comes from the final exam. Therefore we see the remainder of the teaching activities as preparation for that exam. We teach the students to look out for the common errors that are made, and to try to think like an examiner. We show the students examples of past exam questions and student work on them, to highlight how an apparently reasonable solution can in fact be very wrong.

After each of our exams we always give each student a full typeset solution, as they leave the room. We believe that many students learn a great deal in the minutes after the exam because they are very focussed on discovering whether they did well or not.

The computer system awards marks for the assessed problems in the tutorial set. There are typically two assessed problems each week. Each problem starts with 10 marks. If an incorrect answer is entered, the available mark is reduced to 8/10. Subsequent errors reduce the mark further. However, before the deadline, the minimum mark awarded is 2/10. After the deadline the mark on a given problem is automatically zero.

The log book requirement was introduced because we felt students were not getting enough practice writing exam-standard solutions. It is not good if students lose marks because their working is messy or badly set out. Students are told that they must have working in their log book or they will not receive help from tutors. Our young students would not bother to keep a consistent log book if we simply asked them to do it; we have to assess it in some way. This is a major administrative problem for us but we have not come up with a better solution. Each semester, as students leave the examination, they must submit their log books. The books are checked by tutors very rapidly (there are up to 500 of them!). Three problems selected at random are checked for all students. If the working is found in the book, and it looks reasonably original, the log book mark is 100%. If working is not found or if it looks as though it may have been copied from a solution, the mark is reduced. Students are given the chance to contest log book marks. The log book mark multiplies the mark awarded for the tutorial problems. So it is no good getting 100% from the computer system and zero for the log book – the resulting tutorial mark would be zero.

IMPORTANCE OF ASSESSMENT STRATEGIES USED
Our students are very young and have no idea about the subject matter and how they should learn it, let alone how they can best demonstrate their ability. Instead the staff have set up what are believed to be appropriate learning environments and assessment tools. From time to time students have made suggestions about the detail of the tutorial system, for example they asked us to have a modified mark penalty in the case of incorrect units, and this change was implemented. Students have rarely, if ever, asked for variation of the larger pattern (30% tutorial mark, 70% exam).

The tutorial environment is quite formative because most of the problems attempted are not directly assessed. There is also a summative part, the assessed problems, but even these are a vehicle for learning because feedback is given.

It should be noted that the average mark given by the computer system for the assessed problems is very high; more than 90% is typical. This is possible because the students work closely together to develop reliable solution strategies and these are then shared within a working group. Also there are some cases where students have used solutions handed down from older peers as the problem sets have not changed much from year to year. The very high marks awarded do give some students a false feeling of security about their level of understanding. They say, having failed the exam, "I thought I was OK because I got a good "Jellyfish mark".

The examination is the main way students demonstrate learning outcomes. Our exam problems are nearly all original and are of a high standard. It is not too difficult to pass, but it is hard to get a high mark. As mentioned above, we give out a solution after every exam and for some students this is a powerful formative experience.

Examples of classic errors on one year’s examination are sometimes used in teaching the next year. We call this "the Examiner’s Revenge".

WHY ICT IS USED
Up to the end of 1994 we had always had small group tutorials. In a class of 300 students, 15 groups of 20 were needed – a large departmental expense. Meanwhile we saw that attendance was initially excellent but fell to nothing by the end of the year. Annoyingly, the students who went to all the tutorials were generally those who needed it least.

We thought we knew why. An excellent student has the quality, perhaps a defining quality, of "having the guts" to seek help. A poor student often does not and in fact may have learned to hide ignorance in silence. It is very embarrassing to admit in front of your peers that you have not done any work for the past weeks and you have no idea what is going on. It is much easier to just slip away and not return.

The initial problem we were trying to solve was thus, "how do we force the students to do lots of problems and come to all the tutorials, without breaking the bank?"

HOW ICT USE HELPS
In a small class tutorial, because of limited time and tutor availability, the "problem sheet" tends to have just a few exam level problems. If a student cannot solve them there is a strong temptation to just give up. What such students – most students – need is a series of problems that start easy and get harder. Concepts should be introduced and explored one at a time before they appear in combination.
The usual approach in a small class situation is to provide a sheet with some optional "starter" problems. This is enough for an adult learner. However our students are very young and just were not doing such necessary but "optional" work. To be kind to them, they had other pressing tasks that were worth marks.

The computer system allows us to make the "starter" problems compulsory, without involving a huge amount of tutor time to mark them.

The usual small-class approach also has a time delay between assignment submission and assignment marking and return. By the time a student sees a mark, the topic is long past and there is little incentive to revisit the reasons for errors made. This is frustrating for the tutors who see that even when they do write feedback on submitted work, it is generally not read. The computer system does away with all that because the students know instantly when they are right or wrong. In fact we observe that students care very much about this instant "certification" of correctness and will continue to pound away at a given problem until it is achieved.

The Forum, which keeps a record of questions and answers about each problem, is also powerful and can only be done by computers.

MOST IMPORTANT ICT CONTRIBUTION TO LEARNING DESIGN
Regular enforced deadlines with marks given.

The automated diagnoses of errors made are also very important, especially in the Maths subjects where they are very complete and detailed.