Enhancing self efficacy
in experimental design and statistics using elearning technologies: an
interactive approach.
Maxine Swingler*
Department of Psychology,
+44(0)141 330 5088
Paul Bishop
Department of Psychology,
+44(0)141 330 8730
Abstract
A core area in any psychology degree is the
practical course that runs along side the lecture course and provides students
with training in running experiments and analysing data. However, the challenge
is to deliver this training in large classes, where many students express
anxiety about the statistics element of their course.
This article describes the design and evaluation of a web based
interactive tutorial that introduced students to the concepts behind basic
experimental design and statistics. The interactive component of the tutorial
focussed on understanding of experimental design, step by step feedback on
calculation of statistics and how to interpret and report results.
Students’ self efficacy in a number of statistical tasks was measured before and after completing the online tutorial and results showed a significant increase in self efficacy after the tutorial. Students reported the tutorial as particularly useful for understanding inferential statistics and for their assessed laboratory reports. Future developments of an online tutorial based on these findings are discussed.
Keywords: Elearning, statistics,
selfefficacy, interactive, online, psychology.
Introduction
One of the key sources of educational “friction” in an undergraduate Psychology
is the learning of statistics.
Students come to Psychology with a wide range of numeracy skills (Mulhern
& Wylie, 2004, 2006) and are often unaware of the importance of statistics in
psychology. As many psychology
undergraduates have not encountered statistical analysis of any form before, it
is unsurprising that statistics is a significant source of anxiety in psychology
students (Baloglu & Zelhart, 2003), and up to two thirds of students report high
levels of statistics anxiety in courses where statistics is a component (Onwuegbuzie
& Wilson, 2003).
Two factors exacerbate this situation: the key
role statistics has in a psychology degree and the size of classes.
For a psychology degree to be accredited by the British Psychological
Society the course must provide training in quantitative methods, research
design, and a practical component (British Psychological Society, 2008).
A key part of this is the collection and analysis of data; from early on
in their course psychology students have to learn to use descriptive and
inferential statistics both within the practical classroom and for course work.
Added to this problem is at the lower levels of the degree the classes in
Psychology are large as it remains a very popular subject (431 in the 200607
level 1 class at the
Although most departments provide an element of
formal lecture based statistics teaching, this is an aspect of the undergraduate
course at the lower levels that could be targeted for additional support. One
method of approaching this problem, given the large class sizes, is to use
online interactive tutorials that introduce the students to the concepts behind
basic experimental design and data analysis. In fact, use of Internet based
interactive tutorials to support the teaching of introductory statistics in
Psychology and in other subjects is a widespread practice (Aberson, Berger,
Healy, Kyle, & Romero 2000, Aberson, Berger, Healy, & Romero 2003; Bartz &
Sabolik 2001; Richardson & Segal, 1998). When evaluated, these online tutorials
had a positive effect on learning equal to more traditional lectures and
laboratory classes (Aberson et. al., 2000, 2003), especially when included in a
blended approach to teaching (Utts, Sommer, Acredolo, Mahler, & Matthews, 2003).
However, it is less clear whether the positive effect of online tutorials
extends to improving students’ self confidence in statistics.
To address this question, the present study will evaluate the
effectiveness of an online statistics tutorial developed at the
Development of the online resource
The aim of the online learning approach was to enhance level 1 students
understanding of statistics in psychology, and develop their skills in data
analysis as preparation for assessed coursework. An interactive, online tutorial
using example data from a psychology experiment was developed to cover
experimental design, calculation of the mean, ttests, and interpretation of
results.
Consistent and helpful feedback on students’
answers has been shown to be an effective teaching method in statistics (
An initial survey
in April 2006 highlighted that although many level 1 psychology students said
they would find an online statistics tutorial beneficial, they were not aware of
the online tutorial on the student portal (McCotter & Bishop, 2006). To increase
awareness and encourage use of the website, level 1 psychology students were
asked to complete a section of the online tutorial as a preparation exercise for
a face to face tutorial in statistics.
Students recorded their answers to the online exercises and these were
discussed and incorporated into the face to face tutorial.
Evaluation approach
Evaluation of the resource focussed on whether use of the online tutorial
improved students’ selfefficacy in statistics.
Selfefficacy is defined as the confidence a student has in completing a
specific task, and has been shown to be a good predictor of performance in a
variety of contexts (Bandura, 1997).
Current selfefficacy in performing statistical tasks is positively
related to statistics performance and negatively related to statistics anxiety
(Finney & Schraw, 2003). Given that psychology students express anxiety and lack
of confidence regarding statistics in psychology (Boluglu & Zelhart, 2003) it
seemed relevant to evaluate the online resource with a measure that included an
affective component (e.g., Ashcraft & Kirk, 2001; Piotrowski, Bagui & Hemasin,
2002; Schutz, Drogosz, White & Distefano, 1998; Trembley, Gardner & Heipel,
2000). The present evaluation used
selfefficacy statements similar to those used by Finney and Schraw (2003) but
adapted the tasks to the learning outcomes of the online tutorial.
Students merely have to indicate their confidence in their “current
ability” to complete statistical tasks.
The selfefficacy measure has advantages over traditional measures of comprehension or knowledge in evaluating the tutorial. Firstly, it measures the important element of students’ affective response to statistics, something that comprehension tests can only measure indirectly, if at all. Also implementing an additional test on students’ statistical knowledge when they have already expressed anxiety about statistics may only serve to increase anxiety levels and reduce confidence. In addition, psychology undergraduates vary widely in their level of mathematical qualifications and numeracy (Mulhern & Wylie, 2004, 2006). A test of statistical knowledge at this early stage in their psychology course may reflect students’ initial mathematical ability rather than the effect of a single teaching approach. Using selfefficacy as an evaluation measure avoids these pitfalls and provides a direct measure of confidence and to some extent an indirect measure of ability.
Evaluation Method
Participants
118 level 1 psychology
undergraduates participated as part of their psychology tutorial programme.
All gave consent for their data to be used anonymously.
Design.
A within subjects design was used.
Selfefficacy ratings on a number of statistical tasks were measured before and
after students completed the online tutorial.
Students’ feedback on the usefulness of the online tutorial, and problems
encountered were also recorded.
Measures
The evaluation measure was implemented online, before and after the tutorial and responses recorded on a database. The pre tutorial questionnaire consisted of 8 selfefficacy statements on current ability to complete a number of statistical tasks directly related to the content of the tutorial. Participants rated each statement (e.g., “Interpret the result of a ttest”) using a Likert scale from 1 (no confidence at all) to 6 (complete confidence). The post tutorial questionnaire asked participants to rate their current ability on the same 8 selfefficacy statements. Participants completed a further set of questions on how useful the tutorial was for understanding experimental design and statistics (1=Not useful at all, 5=Greatly improved understanding), how helpful the tutorial was, and any technical problems encountered. At the end participants were asked to enter comments they had about the tutorial. The selfefficacy questionnaire is in appendix 1, and post tutorial questionnaire in appendix 2.
Procedure
Students were asked to complete the online tutorial in their own time in the
week before their face to face statistics tutorial. Participants were asked for
consent for their data to be used and informed their data would remain
anonymous. Participants completed the pre tutorial questionnaire, and went on to
complete the ‘Design’, ‘Plot Averages’, ‘Paired ttest’, ‘significance’ and
‘Report’ sections of the ttest tutorial, which took approximately 20 minutes,
followed by the post tutorial questionnaire. At the face to face tutorial the
following week (approximately 10 students per tutorial group), students worked
in pairs on calculations of descriptive statistics using a similar data set.
Results
Although data
was from Likert scales and therefore could be considered ordinal, it was
analysed as interval data. This is a reasonably accepted practice (Nunnally &
Bernstein 1994). All analysis was
conducted using SPSS package, the analysis of variance was performed using the
General Linear Model.
Table
1.
Mean and median
selfefficacy ratings before and after participation in the online tutorial
(N=118).

Mean 
Median 

Current ability to successfully
Complete the following tasks. 
Before 
After 
Before 
After 

3.8 
4.7 
4 
5 

5.5 
5.7 
6 
6 

2.8 
4.0 
3 
4 

2.8 
4.3 
3 
4 

3.1 
4.8 
3 
5 

2.5 
3.7 
2 
4 

3.3 
4.0 
3 
4 

2.7 
3.8 
3 
4 
(1) no confidence at all, (2) a little confidence, (3) a fair amount of
confidence, (4) much confidence, (5) very much confidence, (6) complete
confidence.
A 2 way repeated measures ANOVA with factors of time administered (before or
after tutorial) and selfefficacy task (8 levels) found significant main effects
of time (F (1,117)=240.1, p<.0001), task (F(7,819)=136.9, p<.0001), and a
significant interaction between time and task (F(7,819)= 24.8, p<.0001).
The pre and post test efficacy scores for each task were compared using 8
paired ttests (the α was set at 0.001 following a Bonferroni adjustment for
familywise error). All tasks
showed significant improvement apart from Task 2 (calculating a mean),
explaining the significant interaction (all
ps<.001). To investigate the main
effect of task, 8 paired ttests (one per task) compared selfefficacy ratings
on each task in the before condition and found mean ratings were highest for
task 2, followed by task 1 and lowest for task 6 (all
ps<.001).
Selfefficacy ratings in the after condition were highest for task 2,
followed by tasks 1 and 5 (all ps<.001).
Table
2.
Perceived usefulness of online tutorial.
Percentage of participants who responded to each value of the Likert
scale on each item (N=118).
How useful was the online tutorial for understanding? 
1 
2 
3 
4 
5 
Median 
Mean 
1.
How hypotheses are used to make predictions. 
0.8 
21.0 
27.7 
33.6 
16.8 
4 
3.5 
2.
Identifying within subjects and between subjects experimental
designs. 
0.8 
10.1 
22.7 
43.7 
22.7 
4 
3.8 
3.
Identifying the dependent and independent variables in the
experiment. 
0.8 
18.5 
25.2 
34.5 
21.0 
4 
3.6 
4.
Calculating the group means from the data. 
3.4 
30.3 
19.3 
15.1 
31.9 
3 
3.4 
5.
Creating bar graphs, including titles and labels. 
4.2 
22.7 
26.1 
19.3 
27.7 
3 
3.4 
6.
Choosing the correct Ttest based on the experimental design and
hypothesis. 
1.7 
4.2 
19.3 
52.1 
22.7 
4 
3.9 
7.
How to calculate a paired Ttest. 
3.4 
4.2 
16.0 
52.9 
23.5 
4 
3.9 
8.
How to calculate an independent samples Ttest. 
4.2 
5.9 
23.5 
49.6 
16.8 
4 
3.7 
9.
Checking the significance of Tvalues. 
4.2 
9.2 
29.4 
42.0 
15.1 
4 
3.6 
10.
Reporting results of Ttest in the correct format 
2.5 
10.1 
26.9 
42.0 
18.5 
4 
3.6 
11.
Summarising the results. 
3.4 
11.8 
32.8 
32.8 
19.3 
4 
3.5 
1=Not
useful at all, 2=Did not add to my existing knowledge, 3=Added to my existing
knowledge, 4=Helped me to understand it better, 5=Greatly improved my
understanding. Mean and median
statistics for each item are reported.
A frequency analysis found the distributions of scores for some items of the
questionnaire to be skewed (skewness<1), thus nonparametric tests were used.
After the α was set at 0.001
for multiple comparison using a Bonferroni adjustment, sign tests (one sampled)
found responses to questions, 2, 6, 7,and 8 to be significantly higher than the
middle response (“added to my existing knowledge”) value of 3 (all
ps<.001).
Table
3.
Percentage
Yes responses to post tutorial evaluation questions (N=121).
Question 
Percentage Yes responses 
1.
Did you enjoy the ttest tutorial? 
70 
2.
Improve understanding of statistics lectures? 
90 
3.
(a) Helpful when completing lab report? 
91 
3.
(b) Which section of the lab report was the tutorial most helpful
for? 
Hypothesis 26
Design 29
Means and Graphs
34
Calculating the ttest
83
Checking significance
72
Reporting ttest result
78 
4.
Online feedback useful? 
86 
5.
(a) Any problems experienced? 
29 
5.
(b) Types of problem experienced 
Navigating
7
Entering answers
9
Calculating answers
13
Confusing layout
5
Instructions unclear
9 
Chi Square tests (one variable) were conducted for each question. For questions 1, 2, 3 (a) and 4, significantly more yes responses were observed than the 50% expected (all ps<.0001, α set at 0.005 after Bonferroni adjustment) For question 5 (a), significantly fewer yes responses were observed than expected (p<.0001). Chi Square analysis of responses to question 3 (b) showed that the online tutorial was rated most frequently as helpful in lab reports for calculating the test, checking significance, and reporting the ttest (p<.0001). Chi square analysis of question 5 (b) (types of problems experienced) showed no one problem was experienced more than any other (p<.0001).
Discussion
Self efficacy
The results of the evaluation suggest that the online tutorial improved students’ selfefficacy in their ability to complete statistical tasks. It is worth noting that while all selfefficacy ratings in all tasks improved before and after the tutorial, ratings between tasks were variable (see Table 1). In particular, ratings were already high for tasks 1 and 2 (experimental design, calculating a mean) in the before condition, and there was little room for improvement after the tutorial. This suggests that students were already confident in these areas. Indeed, these are concepts that have been covered in depth prior to testing, both in lectures and tutorials. Selfefficacy ratings for the more advanced tasks (calculating and interpreting a ttest, explaining a p value, and calculating the standard deviation) did increase after the tutorial. Overall, it appears that the tutorial had a positive impact in terms of student confidence in key tasks related to the level 1 practical course. This is encouraging as it shows that the self efficacy gains that have been found with face to face teaching (Finney & Schraw, 2003) can be replicated with carefully designed online resources.
Was the tutorial perceived as useful?
Students indicated that the tutorial was most useful for experimental design, choosing and calculating results of the ttest (see Table 2). However, ratings were lower for developing hypotheses, calculating descriptive statistics and creating graphs. This pattern is echoed by students’ feedback that the tutorial was most useful for the sections of the lab report involving calculating and interpreting a ttest (see Table 3). Combined with the results in selfefficacy, these findings suggest that students perceived elements of the tutorial where they were most confident (descriptive statistics) as least useful, and elements where they were least confident (inferential statistics) as most useful. The interactive element of the tutorial focused primarily on explanation of formulae and step by step calculation of ttests, which may explain the lower ratings for the sections on hypotheses and descriptive statistics. In addition, psychology students are known to struggle with use of formulae and symbols (e.g., Mulhern & Wylie, 2004), thus, students may have benefitted more from this component of the tutorial than others.
Interactive component
The interactive element of the tutorial provided immediate feedback, and
students reported this as useful. Examples of students’ comments include “Easy
to use, and very useful, especially when it says you have got the right answer”.
Relatively few problems were reported; although some reported problems in
calculation of the answers (see Table 3). The tutorial asked students to
calculate answers step by step, and provide feedback, but this inevitably
included some use of formulas.
Inclusion of formulas could be problematic for those students with a limited
mathematical background (Mulhern & Wylie, 2004, 2006).
Indeed, this issue may have been reflected by the lower self efficacy
ratings found with tasks involving calculations (see Table 1).
Future developments of the online resource could focus on providing
challenges to students that require them to go beyond the calculation of
statistical values (Ben Zvi, 2000, Mayer & Anderson, 1992; Park & Hannafin,
1993). For example, by providing a version of the tutorial where results of
statistics are precalculated and the focus is on interpretation of the results
and conceptual understanding.
Overall, combining the on line tutorial with traditional facetoface teaching
approach increased the numbers of students using the resource, and was workable
in a large undergraduate class. Of a possible number of approximately 431 level
1 students, over a quarter (118) completed all the online tutorial and
questionnaires voluntarily. This compares with 15 students from a sample of 98
in our previous evaluation (McCotter & Bishop, 2006). Actual participation was
in fact higher than 118 students, as an additional 154 students completed the
initial online questionnaire, but did not complete the post tutorial
questionnaire. Thus, a substantial number of students attempted to use the
tutorial, and either did not complete it, or did complete it, but did not fill
in the post tutorial questionnaire. While there is no data on reasons why these
students did not finish the tutorial, it is possible that some students found
the content of the tutorial either too simplistic or too complex. Future
versions of the tutorial could address this by providing versions of the
tutorial aimed at different levels of mathematical ability.
It is also possible that students were not motivated to complete the
online tutorial because participation was voluntary and students were not
rewarded with a grade or credit for participating.
If students adopt a strategic or surface
approach to learning (Entwistle, 1997, Mann, 2001), it can be difficult for
tutors to motivate students to engage in additional preparation for classes with
no clear payoff (Reader, 2007). Perhaps if an incentive was introduced to
motivate students to take part, uptake would increase.
Conclusions
Overall, the interactive approach to online learning of statistics seemed an
effective one for level 1 psychology students.
Student feedback suggests that future development of an online resource
that focuses on inferential statistics and underlying statistical concepts would
be beneficial. Furthermore, it may
be more effective for this resource to cover statistics at different levels of
learning, to suit the varying mathematical backgrounds of psychology students.
Although statistics selfefficacy is positively related to performance (Finney &
Schraw, 2003), performance per se was not measured in the present evaluation.
Previous research on web based approaches to teaching of statistics has
often included pre and post measures of comprehension (Aberson et al., 2000,
2003; Utts et al., 2003).
Evaluation of resources in future could correlate measures of comprehension,
initial statistics anxiety and selfefficacy to build a clearer picture of the
benefits of online resources from both affective and performance related
perspectives.
Acknowledgements
This project was funded by the
References
Aberson
C., Berger D., Healy M., Kyle D., & Romero V. (2000).
Evaluation of an interactive tutorial for teaching the central limit
theorem.
Teaching of Psychology,
27, 289291.
Aberson
C., Berger D., Healy M., & Romero V., (2003).
Evaluation of an interactive tutorial for teaching hypothesis testing
concepts.
Teaching of Psychology,
30, 7679.
Ashcraft
M. & Kirk, E.P. (2001). The
relationships among working memory, math anxiety and performance.
Journal of Experimental Psychology:
General, 130, 224237.
Bartz,
AE, &
Baloglu,
M. & Zelhart, P. (2003).
Statistical anxiety: A detailed review of the literature.
Psychology and Education: An Interdisciplinary Journal, 40, 2737.
Bandura, A. (1997). Self Efficacy: The Exercise of Control. NY: Freeman.
BenZvi, D.
(2000).
Towards understanding the role of technological tools in statistical
learning. Mathematical Thinking and Learning, 2, 127155.
BenZvi, D.
(2000). Towards understanding the role of technological tools in statistical
learning. Mathematical Thinking and Learning, 2, 127155.
British
Psychological Society (2008). Quality Assurance Policies and Practice for First
Qualifications in
Psychology 2008. Available from
http://www.bps.org.uk/careers/accreditedcourses/accreditation
criteria/accreditationcriteria_home.cfm
[Accessed October 10th 2008]
Entwistle,
N. (1997). Contrasting perspectives
on learning. In Marton, F.
Hounsell, D., & Entwistle, N (Eds); The
Experience of Learning: Implications for Teaching and Studying in Higher
Education. Scottish Academic Press,
Edinburgh, p322.
Garfield,
J. (1995). How students learn
statistics. International
Statistical Review, 63, 2534.
Finney,
S.J. & Schraw, G. (2003).
Selfefficacy beliefs in college statistics courses.
Contemporary Educational
Psychology, 28, 161186
Lane, D.
(2008). Online
Statistics: An Interactive Multimedia Course of Study,
http://onlinestatbook.com/index.html,
[Accessed 10^{th} Oct 2008]
Mann,
S.J. (2001).
Alternative perspectives on student learning: Alienation and engagement.
Studies in Higher Education, 26(1), 719.
Mayer, R.E., & Anderson, R.B. (1992). The instructive animation: helping students to build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444452.
McCotter,
M.V. & Bishop, P.. (2006). Enhancing
Understanding of experimental design and statistics using elearning
technologies: an interactive approach.
Poster presented at Psychology of Learning and Teaching (PLAT),
Mulhern,
G. & Wylie, J. (2004). Changing
levels of numeracy and other core mathematical skills among psychology
undergraduates between 1992 and 2002.
British Journal of Psychology, 95, 355370.
Mulhern, G. & Wylie, J. (2006).
Mathematical prerequisites for learning statistics in psychology: assessing core
skills of numeracy and mathematical reasoning among undergraduates. Psychology
Learning and Teaching, 2,
119132.
Nunnally, J. C. & Bernstein, I.H.
(1994). Psychometric Theory, 3rd ed. NewYork: McGrawHill.
Onwuegbuzie, A.J. & Wilson, V.A. (2003).
Statistics anxiety: nature, etiology, antecedents, effects, and
treatmentsa comprehensive review of the literature.
Teaching in Higher Education, 8,
195209.
Park, I. & Hannafin, M.J. (1993).
Empirically based guidelines for the design of interactive multimedia.
Educational Technology Research and Development, 41,
6385.
Piotrowski, C.,
Psychology Department Virtual Learning Environment (2008) Psychology student
portal. http://portal.psy.gla.ac.uk/
[Accessed July 22^{nd} 2008].
Reader,
W. (2007). Nonparticipation in
seminars: free rider avoidance and value maximisation.
Psychology Learning and Teaching,
6, 121129.
Richardson W, & Segal D (1998).
Teaching the analysis of interaction in the 2×2 Factorial Design.
Teaching of Psychology, 25,
297299.
Schutz
P.A., Drogosz L.M., White V.E., & Distefano C., (1999) Prior Knowledge Attitude
and Strategy Use In an Introductory to Statistics Course.
Learning and Individual Differences, 10, 291308.
Swingler,
M.V. (2006). Online Statistics Tutorial.
http://www.gla.ac.uk/sums/ttest/
[Accessed July 22^{nd} 2008].
Trembley.
P.F.,
Utts J,
Sommer B, Acredolo C, Mahler MW & Matthews H. (2003).
A study comparing Traditional and Hybrid Internetbased Instruction in
Introductory Statistics Classes. Journal
of Statistic Education,.11, 110
West R.W.
(1996). Java applets for teaching.
http://www.stat.sc.edu/~west/javahtml/
[Accessed 10^{th} October 2008]
Appendix
1: Selfefficacy
Please rate your confidence in your current ability to successfully
complete the following tasks. The item scale has six possible responses: (1)
no confidence at all, (2) a little confidence, (3) a fair amount of confidence,
(4) much confidence, (5) very much confidence, (6) complete confidence. For
each task, please mark the one response that represents your confidence in
your current ability to successfully complete the task.
Note 1 is No confidence
at all to 6 Complete confidence
Distinguish between
different types
of experimental design
1 2
3 4
5 6
Calculate a standard
deviation
1 2
3 4
5 6
Explain what a pvalue is
1 2
3 4
5 6
Explain the calculations
of the ttest.
1 2
3 4
5 6
Select the correct ttest
based on an experiment’s design
1 2
3 4
5 6
Calculate a mean
1 2
3 4
5 6
Explain how to calculate
degrees of freedom
1 2
3 4
5 6
Interpret the result of a
ttest
1 2
3 4
5 6
Appendix 2:
Post tutorial Questionnaire
How useful were the online exercises for understanding Experimental Design and
Statistics? Rate each of the
statements below from 1 to 5.
1=Not useful at all
2=Did not add to my existing knowledge.
3=Added to my existing knowledge.
4=Helped me to understand it better.
5=Greatly improved my understanding

Rating (15) 
How hypotheses are used to make predictions. 

Identifying within subjects and between subjects experimental designs. 

Identifying the dependent and independent variables in the experiment. 

Calculating the group means from the data. 

Creating bar graphs, including titles and labels. 

Choosing the correct Ttest based on the experimental design and
hypothesis. 

How to calculate a paired Ttest. 

How to calculate an independent samples Ttest. 

Checking the significance of Tvalues. 

Reporting results of Ttest in the correct format 

Summarising the results. 

Was it helpful?
1.
Did you enjoy
the Ttest tutorial ? Yes/No
2.
Did the Ttest
tutorial improve your understanding
3.
Do you think
that the Ttest tutorial will help with completing
Hypothesis/Design/Means and Graphs/Calculating the Ttest/Checking
significance/Reporting Ttest results.
4.
Was the feedback
in the tutorial useful? Yes/No
5.
Did you
experience any problems when using the tutorial? Yes/No
If yes,
circle the type of problem(s) you encountered from the options below.
Navigating to each web page/entering answers/calculating answers/confusing
layout/instructions unclear/other.