Evaluation of computer-assisted learning courseware to reinforce the teaching of removable partial denture design theory

Robert A. McKerlie*

School of Dentistry, University of Glasgow

[email protected],

Tel: +44(0)1412119732

Donald A. Cameron

School of Dentistry, University of Glasgow

[email protected],

Tel: +44(0)1412119628

Robert G. S. Matthew

Learning and Teaching Centre, University of Glasgow

[email protected],

Tel: +44(0)1413303194

Abstract

This two-year study investigated the efficacy of a computer-assisted learning (CAL) programme at reinforcing the learning of removable partial denture (RPD) design. Thirty-nine dental students (year one of study) and thirty-five dental students (year two of study), at the commencement of their fourth year completed an RPD knowledge assessment, in the form of a 31 question online quiz (year one) or a 51 question paper-based assessment (year two) prior to the first CAL session and again after the last CAL session.  The students were split into groups of nine or ten (year one) and eight or nine (year two) and allocated fortnightly timetabled 1.5 hours (year one) and weekly timetabled 2 hours (year two) access to a CAL programme, four (year one) or six (year two) sessions in total.  After the conclusion of the CAL intervention the students were asked to complete an evaluation questionnaire. The RPD knowledge assessment showed a highly significant improvement in scores after the CAL intervention (P-Value < 0.001).  The students found the CAL programme to be both informative (year one n=35; year two n=32) and useful (year one n=33; year two n=24) with the majority finding the programme easy to use.  A computer-assisted learning programme can be effective at reinforcing the learning of removable partial denture design.

Keywords: Computer assisted learning: computer aided learning: computer aided instruction: computer courseware: E learning: Web-based courseware

Introduction and background

The use of computer courseware has increased dramatically over the last two decades, with powerful multi-media rich desktop computers and the emergence of the Internet as a plausible educational delivery method. The majority of developments of computer courseware have been as a supporting role, complimenting traditional teaching methods (Schittek et al. 2001). The benefit of the computer courseware is in convenience and accessibility to the student. It also allows for a more self-paced style of learning (Bachman et al 1998).  In the study by Lechner. S, Lechner. K, & Thomas (1999) a computer courseware was used in place of a series of lectures on removable partial denture (RPD) design.  On completion of the courseware students produced designs of a high standard under examination conditions.

At Glasgow Dental School removable RPD design is traditionally taught with a series of formal lectures in Year 3, followed by a tutorial/practical course in Year 4.  The tutorial/practical course has historically been taught by Dental Technology Tutors within the removable prosthodontic teaching laboratory.  As a result of the gap between the formal lecture course and the commencement of the tutorial/practical course, the tutorials have inevitably taken on the form of mini lectures, covering the theoretical aspects of partial denture design. 

“Difficulties with small groups: The teacher gives a lecture rather than conducting a dialogue, the teacher talks too much, students do not prepare for session …”(Ramsden, 2002: 157)

The aim of this study was to assess the effectiveness of proprietary computer courseware in reinforcing the teaching of removable partial denture design through independent learning, thereby removing the need for the tutorial ‘mini lectures’. 

Methods

In the first year of the study thirty-nine dental students at the commencement of Year 4 attended an induction session were they were informed about the structure of the study, timetable, the RPD knowledge assessment, evaluation questionnaire and introduced to the virtual learning environment and in particular the discussion area within the virtual learning environment.

The students were split into groups of nine or ten and asked to complete an online removable partial denture knowledge assessment in the form of a 31 question online quiz.  The assessment was delivered through the School’s virtual learning environment, Moodle.  This logged the students’ assessment scores.

The student groups were then allocated fortnightly timetabled (1.5 hours) access to the computer courseware, four sessions in total.  The courseware utilised was ‘Interactive Prosthodontics: Learning to Design Removable Partial Dentures’ produced by Lechner SK & Thomas, Sydney University Dental School (figure 1).  This alternated with fortnightly self-directed study sessions.  The self-directed study gave the students the opportunity to consult the recommended reading texts.  Because of the limited number of timetabled sessions the researchers made the courseware available to the students outwith the timetabled session in the school library.

At the conclusion of the computer courseware intervention the students were asked once again to complete the RPD knowledge assessment online.  The students were also required to complete a computer courseware evaluation questionnaire based on that developed by Lechner S, Lechner K, and Thomas, (1999).

This was followed by the laboratory-based practical course, which would give the opportunity for the students to highlight any misconceptions.

Figure 1  Interactive Prosthodontics: Learning to Design Removable Partial Dentures, Opening Screen (Lechner SK & Thomas GA, 1998, Sydney University Dental School)

On evaluation of the data collected from this study from both students and investigators it was felt appropriate to modify the study and run it again the following year.  Both staff and students commented that there was no monitoring of the online assessment and therefore no guarantee that it was the student’s own work.  As a result of this feedback, the investigators modified the RPD knowledge assessment.  The RPD knowledge assessment was increased from the 31 question online quiz to a 51 question paper based assessment delivered under controlled conditions.  One of the main criticisms from year one was the limited timetabled access to the computer courseware; this was increased from four 1.5 hour sessions in year one to six two hour sessions in year two.

The lack of feedback on completion of the RPD Knowledge Assessment before and after the courseware was highlighted.  In year two, interactive feedback lectures with staff were developed to go over each of the questions in the RPD Knowledge Assessment.  In year one, the courseware evaluation questionnaire asked the student to consider why their score had improved, stayed the same, or got worse as a result of exposure to the courseware.  With the introduction of the interactive feedback lectures the courseware evaluation questionnaire was modified, since the student did not receive a numerical score they were asked to comment on aspects of the courseware that they liked and disliked.

Following the completion of the courseware by the students an interactive series of lectures took place between the tutors and students. 

Results

Removable Partial Denture Knowledge Assessment

On analysis of the RPD knowledge assessment scores from year one of the study using a paired t-test, there was a highly significant difference (p< 0.001), with a typical increase in score after the courseware being between 3 and 5.7 points.  The mean score before the courseware was 24.5 out of a possible 39 and after access to the courseware this changed to 29 out of a possible 39.  It is interesting to note that some six students failed to increase their score after the CAL intervention, one student’s score remained the same and five scores decreased.  The decrease in score ranged from 0.2 to 5.4 points.

Figure 2         Year one Pre & Post RPD Knowledge Assessment Scores

Figure 3 shows the student scores in the RPD knowledge assessment pre and post courseware intervention for year two of the study.   The mean score Pre courseware intervention was 37.57 compared with 53.60 Post courseware intervention.  On analysis using a paired t-test the students showed a highly significant improvement in scores after the courseware intervention (p< 0.001) with the typical increase being between 10 and 15 points. 

Figure 3         Year two Pre & Post RPD Knowledge Assessment Score

Computer Courseware Evaluation Questionnaire

In year one when evaluating the computer courseware the students found it to be both informative (n=35) and useful (n=33) with the majority finding the courseware easy to use. 

The results from the year one evaluation questionnaire were very positive with regard to the courseware.  There were twenty-four (61%) students who disagreed with the statement: ‘I feel I could have learned more in lectures than I did with this courseware’.  All students found the courseware easy to use and only three (8%) students did not like the self-paced style of the courseware.  Thirty-one (79%) students felt the courseware improved their comprehension of RPD design while 37 (95%) students agreed that the computer was a useful learning resource.  Thirty-eight (97%) students would like to use the programme again for revision and 27 (69%) students felt they learned a lot form the courseware.  However 21 (54%) would prefer to learn RPD design in tutorials, Table 1.

Table 1          Year one Evaluation Questionnaire Extract

When asked why they thought that their RPD knowledge assessment score had improved examples of the type of responses made are:

            ‘I learned more about RPDs from the program’.

‘While working through the program I took notes of key points and rules which I used for revision’.

‘I understood the topic slightly better and could make more educated decisions’.

‘I learned a lot of new information from the program’.

In year two when evaluating the computer courseware the students found it to be both informative (n=32) and useful (n=24) with the majority rating the courseware as a ‘good’ learning tool.

The results from the year two evaluation questionnaire were also positive with regard to the courseware.  There were nineteen (54.3%) students who disagreed with the statement: ‘I feel I could have learned more in lectures than I did with this courseware’, seven (20%) students remained neutral and nine (25.7%) students felt that they would have learnt more in lectures.  Thirty-three (94.2%) students found the courseware easy to use and only three (8.6%) did not like the self-paced style of the courseware.  Twenty-seven (77%) students felt the courseware improved their comprehension of RPD design while 28 (80%) agreed that the computer was a useful learning resource.  Thirty four (97.1%) students would use the programme again for revision however, fifteen (42.9%) would prefer to learn RPD design in tutorials, Table 2.

Table 2          Year two Evaluation Questionnaire Extract

Examples of the type of response given when ask to identify one aspect of the courseware that they enjoyed:

‘I felt that the simplicity of the instructions in terms of how to use the programme, allowed you to focus on what you had to learn about i.e. RPD.’

‘The programme allowed you to study at your own pace.’

‘It was easy to understand and explained things well.’

‘Enjoyed the computers, but would have appreciated help from staff during the sessions.’

‘I enjoyed the fact that the programme was very comprehensive in RPD design.’

‘I thought it was good that it gave pictures of casts and tried to relate the fact that theory does not always work out exactly on a patient and tried to explain how you would go about modifying it.’

‘The program is a good idea, however it became laborious as my note taking started to take over.  I think it would have been better if there was a lecturer teaching as well as the CD.’

Examples of the type of response given when asked to identify one aspect of the courseware that they disliked:

‘Continuous note taking.’

‘I found a lot of the time I was trying to copy down as much information as I could, but maybe because of this I didn't take in as much as I would have liked to do.’

‘Some of the sections were a bit long and dragged on a bit.’

‘The lack of personal on-spec feedback.’

‘I didn't like the fact that if you didn't really understand something you couldn't ask questions.’

‘Repetitive nature of some exercises and there were no notes to allow revision after the session.’

‘The allocated time slot.  I would prefer to have more flexibility since no actual teaching is involved and it is self learning.’

Discussion

Although the analysis showed a statistically significant improvement in the RPD knowledge assessment scores after the courseware intervention, the actual improvement was less than anticipated by the researchers.  Possible explanations were:-

• Study time – the students treated the timetabled self-study sessions as ‘free-time’, i.e. no requirement to study. 
• Workload – students felt that the workload for each session was heavy.  The researchers were aware that the four timetabled sessions available could result in intensive workload and took steps to ensure that the courseware was available for access in the school library.  The students could use the study time to access the courseware if required.
• No Assessment – the students might have focused more if there was a formal assessment.  The dental students up to this point have been used to didactic teaching with little emphasis on self-directed learning apart from examination preparation.  A commonly asked question from students was ‘What is the point of learning this if we are not going to be asked questions on it in the exam?’

One of the main criticisms from students that emerged from the evaluation questionnaire in both years of the study was the perceived inability for the students to ask questions or seek clarification of any misconceptions.  The discussion forum within the virtual learning environment was not utilised possibly due to poor integration with the courseware as the courseware and the virtual learning environment are two separate entities.  It could also be said that the students had no previous experience in utilising a discussion forum within the course prior to this study; it was a new experience for them.

Computers are not for everyone; yet it is true that it is very hard today to avoid them totally.  It has to be kept in mind that people learn in different ways.  Recent research has commented on the fact that not all students like using computers to learn, some even find the use of computers as a barrier to their learning.  Care must be taken to offer a ‘blended approach’ when considering online learning resources. 

Students commented on the lack of notes and the time that they had to spend writing everything down and how that detracted from the learning experience.   This was despite the fact that the students were informed that the courseware was designed as an interactive learning tool and was not intended to be used for note taking.

In the process of outlining how the component of the course was being structured there was at least one student who was uncomfortable with the whole concept of working with the computer courseware.  This student highlighted their preference to being given information in a tutorial format to enable them to take notes, and revisit them in their own time.  On further investigation it became clear that there were a number of students who felt uneasy that ‘they were being left to learn for themselves’ using the computer.  These students went on to state that this amounted to a ‘dereliction of duty’ on the part of the tutors.  One of the authors (RMcK) took time to reassure the students in question that the CAL courseware was simply a tool to help reinforce the theoretical core content lecture series delivered previously.  When this is looked at in conjunction with the feedback from the evaluation questionnaire it is clear that a number of the students felt uncomfortable taking control of their learning in this situation and were unhappy with the lack of tutor supervision during the timetabled courseware sessions.  Rather than embracing the opportunity to take control of what they learned and at what pace this learning took place the students would have preferred someone to tell them what they needed to know. 

This feedback causes the authors some concerns when viewed in the light of the General Dental Council requirement that at graduation, dentists are life long learners.  Further Devitt and Palmer (1999) in their study comparing interactive CAL with CAL that provided didactic material, discovered that the CAL that provided didactic material produced significantly better knowledge outcomes.  They put this superior knowledge gain down to the fact that the students were more comfortable with didactic learning in a predominantly traditional lecture-based medical course. Again this reinforces the view that in providing blended learning opportunities designers of the material need to focus on both the knowledge outcomes and the process whereby those knowledge outcomes are used by the learner to change practice.

In a study looking at undergraduates and staff attitudes towards electronic learning Gupta, White and Walmsley(2005) reported that both staff and students see the courseware as a means to supplement rather than replace traditional methods of learning in the dental undergraduate programme.   Interestingly there have been studies that reported CAL to be as effective if not more effective as traditional learning methods (Bachman et al, 1998; Bissell et al. 2003; Carroll, & Schwartz, 2002; Howerton et al., 2004).  Could the need for staff contact and the request that CAL supplements rather than replaces traditional learning methods have more to do with staff and students’ perception of learning?  As a student you attend university to learn, lectures & lecturers are inextricably linked to this process, therefore no lectures or lecturers equals no learning. 

In this study no attempt was made to differentiate between those students who wished staff to be present to simply answer questions and clarify any misconceptions and those who would have preferred the whole process to be delivered in a more traditional way.  However, other work carried out in the school has shown that dental students on the 'old curriculum' appear to prefer more didactic forms of teaching, which may be influenced by a heavy timetable resulting in the classic surface approach to learning, whereas on the new curriculum there is some evidence of change in student responses, but it is very early to say anything definite.

Conclusions

The results of this two year study support the body of research that concludes that computer-assisted learning (CAL) courseware can be effective at reinforcing the teaching and learning by a traditional lecture course. 

However, when considering integrating computer courseware into the curriculum care must be taken to ensure that there are sufficient resources in place to cope with the increased demand for computer access.  It is also important to ensure that alternative resources are also made available, as not everyone is comfortable with using computers and can justifiably cite their use as a possible barrier to their learning.  It has to be a blend of traditional teaching/learning methods and online learning.  Correa et al. (2003:114) comment on the change of role for the teacher when integrating web-based learning resources; ‘the teacher was not an information and knowledge source, as occurs in conventional classes.  The computer takes on this role.  The teacher was important to eliminate doubts raised when reading and interpreting computerised text and images.  Principles of interactivity, navigation and legibility must be applied to support this new relationship’.

It is also important to remember the advice offered by Enyon, Perryer, & Walmsley, (2003:110):- “Teachers wishing to implement web-based teaching within their institution should be aware of the expectations and concerns of their students”.

The increasing incompatibility of the current version of the propriety software evaluated and the introduction of a virtual learning environment has led the authors to consider how best to offer this type of support in the future.  Further work is required to investigate ways of embedding the learning material into the virtual learning environment, therefore offering the students a one-stop-shop.

References

Bachman M.W., Lua M.J., Clay D.J., Rudney J.D.  (1998). Comparing Traditional Lecture vs. Computer- Based Instruction for Oral Anatomy.Journal of Dental Education, 62(8), 587-591.

Bissell V., McKerlie R.A., Kinane D.F., McHugh S.  (2003). Teaching periodontal pocket charting to dental students: a comparison of computer assisted learning and traditional tutorials.  British Dental Journal, 195, 333-336.

Carroll A.E. & Schwartz M.W.  (2002). A Comparison of a Lecture and Computer Programme to teach Fundamentals of a Draw-a-Person Test.  Archives of pediatrics and Adolescent Medicine, 156,137-  140.

Correa L., De Campos A.C., Souza S.C., Novelli M.D.  (2003). Teaching oral surgery to undergraduate students: a pilot study using a Web-based practical course.  European Journal of Dental Education, 7, 111–115.

Devitt P. and Palmer E. (1999). Computer-aided learning: an overvalued educational resource?  Medical Education, 33, 136-139.

Enyon R., Perryer G., Walmsley A.D. (2003).Dental undergraduate expectations and opinions of web-based courseware to supplement traditional teaching methods.  European Journal of Dental Education, 7,103–110.

Gupta B., White DA., Walmsley AD. (2004).The attitudes of undergraduate students and staff to the use of   electronic learning. British Dental Journal. 196(8):487-92.

Howerton W.B., Enrique P.R.T., Ludlow J.B., Tyndall D.A.  (2004). Interactive Computer-Assisted Instruction vs. Lecture format in Dental Education.  Journal of Dental Hygiene, 78(4), 1-6.

Lechner S.K. & Thomas G.A. (1998). Interactive Prosthodontics: Learning to Design Removable Partial, Sydney University Dental School             Dentures. School of Dentistry, University of Sydney.

Lechner S.K., Lechner K.M., Thomas G.A. (1999). Evaluation of a computer-aided learning program in removable partial denture framework designing. Journal of Prosthodontics, 8, 100-105.

Ramsden P.  (2002). Learning to teach in higher education.  London: RoultedgeFalmer.

Schittek M., Mattheos N., Lyon H.C., Attsrom R.  (2001). Computer assisted learning. A review.  European Journal of Dental Education, 5, 93–100.