Evidence & HE

One of the real challenges I face in trying to convince people that there are better ways to approach education is an attitude towards evidence that I don’t understand. I was talking to one educator about the evidence from psychology about how to motivate students to engage in their academic studies. Her response puzzled me, but it is something I have heard before and since. She said: that’s all right if you believe in that kind of stuff. When I asked about the stuff she was referring to, she said she was referring to research, as (according to her) we all know, researchers can find any outcome that fits their agenda.

Needless to say, that was the most extreme example of the dismissal of evidence, but certainly not a rare one.

In my research methods class, when I used to talk to the first year students about rational thinking and evidence, I used an audience response system to poll the students about various aspects of their understanding. One of the questions I used to ask was:

Should the major decisions in our society be based on (a) solid evidence gathered using the best research methods available, or (b) feelings, beliefs and just “knowing” when something should be a certain way?

As it was during a lecture on rational decision making, of course I would get 98% responding with “a” as the appropriate response.

I then showed the following slide.


During this slide, I explained to the students that a placebo-controlled randomised study is about as good as it gets in the clinical scientific world, and that the homeopathic society was saying that the best science couldn’t measure the effects of homeopathic medicine. I then repeated the question:

Should the major decisions in our society be based on (a) solid evidence gathered using the best research methods available, or (b) feelings, beliefs and just “knowing” when something should be a certain way?

To my surprise (the first year I did this) those responding with “a” dropped to about 55%. These are students who enrolled  in University to obtain a BSc in psychology from one of the five top psychological research departments in the UK. Suddenly, there was something they wanted to believe in, and the idea of using science to answer a question wasn’t that important to them.

I have always hoped that by the time the students graduated with their degrees, they would, once again put science and evidence back into a premier place for answering questions in about the world. And yet, I have my doubts.

The Right Answer

Roger Shank wrote something last week that I think is worth looking at:

Math and science are meant to teach thinking (or so it is said). They could actually teach thinking of course, but when the scientific questions are given to you, and the right answers are taught to you, science ceases to be about observation, experimentation, hypothesis creation, and reasoning from evidence, and becomes memorization to get good scores on multiple choice tests.

Does constantly coming up with the right answer mean that we don’t learn to think. I can expect individuals who are uneducated to undervalue the power of rational thinking and the scientific method; evidence, to the uninitiated, is nothing better than opinion. But an education, at the very core, should about thinking, rational thinking, and critical evaluation of evidence. If a person has been trained to understand the process and rigour that accompanies the proper application of the scientific method, and the strength of properly obtained evidence,  how can scientific findings be something that you can simply dismiss as though they were nothing more than opinions.

Scientific discovery has laid the foundation for much of what we enjoy in the world today. However, conservative influences in society, just as in the past, use whatever power is at their disposal to ensure that science only supports the worldview that is already established. Delivering well educated, thinking individuals is needed to counterbalance the antiscientific influence that has arisen in recent years. Unfortunately, well educated has come to mean great memorisation.

I would suggest that our obsession with content and getting the right answer has meant that rational thinking has become an optional extra in HE.


How have we made something as exhilarating as learning as oppressive as education?


Information Abundance and Teaching

I received an e-mail from a friend today, and I’d thought I’d share some of it:

Dave said that he would like to talk to “us” about training of computer/technical information.  Apparantly they are getting lots of enquiries about how to better teach IT-like skills…. he said (and, I swear I am not saying this just for Jesse):  “….people are contacting me asking things like ‘how can I teach kids who know more than I know? and, what am I supposed to teach them when they can just look up any answer at any time?  how can I teach in an age of information abundance?”  [ok, maybe I added that last one…..].

The “us” referred to is our Psychological Scholarship Innovations group – we look into teaching and learning from an evidence based perspective (kind of some of the stuff I write about).

At the risk of stating the obvious – we are in the age of information abundance. If we insist of maintaining our place as the expert on something that we just teach, and are not truly an expert on, we will constantly be upstaged by those we teach. Especially if they are taking a class in something that they are interested in. If there is an intrinsic motivation to learn about something, they will probably have already looked into the area, and sometimes this is to some depth.

When I am teaching adults, there are some classes where I would defy anyone to challenge what I am teaching. There are other classes that I am assigned to teach, and I have to go out and do some work in preparation of what I want to cover – because I’m not really an expert. I am a good teacher, and I have a strong background in psychology, but that doesn’t make me an expert across the entire area.

when the information is available, literally all around us, why do teachers resist asking the students to contribute? Why is the conversation still, almost always, one way? I was reading some stats about high school classes, and it was saying that something like 95% of the dialogue (including questions) comes from the teacher. If you are in a traditional university lecture, the numbers have got to be even higher. Learners today think this is what makes good learning. No matter what we say to the contrary (as a community), our actions are drowning it out.

At a recent meeting, someone said to me that as learner centric classes become the norm – become the norm. This statement is in response to having a single core class with a learner centric philosophy, and a couple of optional modules doing the same. Out of about 15 mandatory classes, and 20 optional module offered in our department, I’m not sure how three or four is beginning to represent the norm.

The internet is here to stay. Access is becoming ubiquitous. Information is freely available.

I don’t hear anyone arguing anything different, so when are teachers going to stop spouting and start curating?

Higher what?

I read Harold Jarche’s review of Marina Gorbis’ new book, and was thinking about what to write about it (it struck a chord) and then Clark Quinn came up with something brilliant. It was his comments about our unique human skills and specifically our abilities for sensemaking, and  novel and adaptive thinking that hit me. In a nutshell, these are our higher order thinking skills. Higher, as in higher education. Higher as in the manipulation of the abstract in order to find emergent patterns. Higher as in taking the mundane and creating something new. Higher as in bringing together disparate strands and synthesising information to produce a new, emergent thing (property, thought, theory etc.). This is what sets us apart as different. We can do these things, and we can do them well. Even at the most basic level, we learn to decipher recognisable thought patterns from a series of sounds (speech), and we learn to decipher thoughts from abstract symbolisation (writing).

Early in our history, we figured out how to fashion tools that would augment our physical abilities. We learned to hunt with implements, farm with tools, and lift and move with technology. The augmentation of our physical abilities has allowed us to build and create on an unprecedented scale. We are building a new arts and innovation centre, Pontio, at Bangor University.

New arts and innovation centre at Bangor University, scheduled to open in 2014.
New arts and innovation centre at Bangor University, scheduled to open in 2014.

As I watch the building grow from my office window, what I find amazing are our abilities to lift and move the hundreds of tons of materials and carefully put them in place. We can do this in a relatively short time period because we rely on tools that amplify our natural lifting and moving abilities.

In the last 50 years or so, we have developed some equally impressive tools to amplify some of our thinking abilities. Not the higher order abilities, but many of the routine thinking abilities we possess. We have machines that can do routine and complex calculations, memorise and regurgitate with impressive speed and accuracy, and with storage and searching facilities that are immense and fast. And, yet, we don’t have machines that can do what makes us really special: synthesise and create.

The sad thing is that in our higher education institutions, we still focus on the routine calculations, the rote memorisation, and the regurgitation skills that we needed in the pre-computer age. I hear my colleagues say to me that if our students don’t learn all the content, and memorise all the ideas, they couldn’t function well if all the computers in the world suddenly stopped working. I wonder if construction workers worry about their buildings remaining unfinished if all the cranes in the world suddenly stopped working.

Instead of challenging our students to sharpen the skills that only we (humans) possess, we assemble them in massive groups and lecture them, bringing them back together several months later to go through an examination exercise; the routine memorisation/regurgitation exercise. Is this the best we can do?

By the way, our beautiful new Pontio building is going to have a massive lecture theatre (over 500 seats), accompanied by a number of smaller (200+) lecture theatres so that we can perpetuate and expand our carefully honed memorise/regurgitate routines. How sad.

Keep Calm and Carry On

In July, Jordan Weissmann gave us all a warm fuzzy blanket to make us feel better about higher education and the direction we are going. Whispering a soothing lullaby to university heads and administers everywhere, he rocked the cradle of complacency with reassurances that all is well, and we have nothing to fear.

However, the fear is still there. The unsettled feelings of disquiet just won’t go away. I’m sure the apologists who still lecture in the same way they always have will be relieved to hear that the internet isn’t really going to change anything. We should keep calm and carry on.

Contemporary rendering of a poster from the Un...
Photo credit: Wikipedia

The idea that researchers make the best teachers is still resoundingly true, I am the centre of my teaching world because of my expertise, lecturing is a fine art refined over centuries of practise, students are learning as well today as they ever have, our graduates can think better, and have better skills than ever before, and we have new funding models and resources that will answer all our problems.

Except that all of this simply isn’t true. The world is changing. Just because we want to hear that everything is the same as it always has been, and aways will be, that doesn’t make it true. The world has moved into the age of information abundance in as little as five years. Our world (the world of universities) is firmly rooted in the age of information scarcity. EDx isn’t going to change that. EDx is based on a flawed model. EDx is simply taking what we currently do and digitising it.

There are answers out there. There are models of learning that work. There are reasons for universities to still exist. We just haven’t embraced them yet, and I have doubts that we (in the West) will, which is why I read with interest about Africa and India will lead the education revolution. I agree, and am excited to see it happening.

Learning and Education

When did education stop being about learning and turn into a performance art?

I was reading over some of my students’ blogs from last semester, and one of the things that jumped out at me was their observation that education was about grades, degrees, and getting ahead and not about learning (they weren’t happy about it – often pointing out that this is what is wrong with education today).

I blogged last year about how Bjork talks about the conditioning cycle that moves both students and learners into a self reinforcing cycle of performance and reward. Students are rewarded for doing what the teacher wants (high grades) and teachers are rewarded for increasing the number of students who achieve high grades (promotion opportunities, institutional acclaim). This becomes a virtuous (vicious) cycle of mutual rewards as students learn to perform (who said passing a test had anything to do with learning), teachers recognise the performance with academic currency (grades) and institutions reward “good” teaching with recognition and praise. Who is fooling who?

Jack Rogerson (one of my students) blogged about student cheating and why. He noted work by Dweck & Vandewalle who identified performance goal oriented students as:

  • Maladaptive Students – Quickly become disillusioned with tasks and tend to discourage themselves from developing their academic abilities/skills. They instead focus their attention on the opinions of others – they are mindful of negative judgement and are therefore more likely to resort to cheating as means of maintaining a positive image of capability amongst their peers.

It is all about appearances.

I believe that there are many students who start their studies actually excited about learning, but eventually, most find themselves caught up in the performance and reward cycle.

For me, this is one of the damning features of lectures. I will stand up and tell you something that I think you should know (and/or record it as a podcast and post it to the world), and then, in the name of assessing your learning, ask you about what I told you. The better you are at fetching the information (including some tidbit that I didn’t actually share with you), the higher the reward you will receive.

I wonder what Socrates would think of our civilised approach to learning today?

It doesn’t have to be that way. Present something both interesting and useful, care about the students success, empower them to direct their own learning experience, and help them believe that they can succeed. Provide learning tasks that allow students to match their ability with your expectations, and then reward real success, not a momentary performance.

We have the know how and the tools to liberate the learning experience. We can really have students centred learning – for which lecturing is the antithesis – at every learning  opportunity. Using an information abundance model to underpin learning design, I have scaled student centred learning up to 60+ students at a time. We don’t need to have seminars and discussion of >10 students to have a real learning experience, it can be available now with reasonable resources.

Given what we have available and what we can do now, I despair at the cost of inertia.

Teaching with Podcasts – A Great Success Story

Teaching labs for statistics classes is one of those labour intensive, not very much fun, teaching jobs. In Bangor, we enjoy 300+ students a year studying psychology, and teaching the students to use SPSS to analyse data is one of those difficult and thankless tasks no one really wants.

For many years, we divided our numbers up into groups of about 50, and then repeat taught six sessions for 90 minutes for about eight weeks each semester over three semesters. Even with an average of three postgrads helping out in each lab session, the results were never very positive – the most able were bored, the less able were lost, and the middle felt pushed, but kind of got it.

My colleague, Mike Beverley, was responsible for teaching the labs across both the first and second years while I taught the first year classes. The topics covered ranged from simple data entry to complex ANOVAs and factor analyses. Students were set assignments, analysed data, and turned in regular work. The experience was not very enjoyable for either the students or the teachers. Our satisfaction ratings from the students was low when it came to the labs (“Burn every copy of SPSS on the planet!” was typical). We needed a new model.

We initially decided to podcast Mike’s first session each week, and then he could ensure that the presentation to the students was at least consistent, and he didn’t have to repeat himself endlessly. Podcasting was new then (spring of 2005), so we were just trying things out. At first we recorded audio podcasts, but after a few weeks, we recorded the screen capture for the students. By chance, we stumbled onto a model that really worked. In the labs, we would usually provide about five minutes of instruction, and then let the students work at it for a few minutes before introducing something more. As a result, all of our podcasts were about five minutes long, demonstrating how to carry out a procedure with a voiceover. The initial fumbling about was successful enough that we decided to prepare podcasts over the summer to cover every topic we taught for use the following year.

In the Autumn of 2005, we scheduled the labs, employed the postgrads, and demonstrated the podcasts to the new, incoming students. To our surprise, no one ever came to another scheduled lab. I fib here – there were about six students who insisted on coming every week. After about three weeks, we rolled the six into a single session, and let the rest of the students know that we were only going to be in the lab for that single 90 minute session. If they had question, they needed to come along then.

The students learned SPSS – better than they had in previous years. Their feedback in the module evaluations was uniformly positive about learning SPSS that way, and we changed the way we did things (we still use the same basic model six years later).

Cost Savings

The savings from this have been great for the Department. Teaching the traditional labs went something like this:

  • 9 hours/week per instructor
  •         4 instructors = 36 hours/week
  •    8 weeks teaching across 3 semesters
  •    864 hours/year of instructional time
  • Stats support surgeries for 3 hours/week across 15 weeks involving 2 or 3 people
  •    Approximately 75 hours/year
  • Total of about 940 hours year teaching and supporting stats

Using podcasts for instruction, the cost went someting like this:

  • 22 weeks with 1 hours of support available per day (1 person currently)
  •      110 hours
  • Savings of 830 hours

Podcast Development

  • Estimate about 3 – 4 hours per lesson
  •       2 or 3 podcasts per lesson
  •       23 lessons
  •    70 podcasts in total
  • About 100 hours in total to make the initial podcasts

Bottom Line

Massive savings overall (830 hours less development time), and the podcasts are reusable. The instructor was happier (for a time), the students were happier. The students have the podcasts available throughout their entire undergraduate programme so they can refer to them anytime. They have control over their learning,and use the labs when they choose to do so. A real win – win solution!

The single biggest problem has been the updates of the programme (SPSS). This has meant that we have re-recorded the podcasts twice in six years – not too bad an investment given the long term benefits.

Success Secrets

I think there were a few things we did (and continue to do) right when we made the teaching podcasts. They were:

  • Have an expert in Stats & SPSS teaching make the podcasts (at least the first time)
  • Keep the podcasts short (5 – 10 minutes)
  • Don’t obsess about quality
  • Be prepared to release updates quickly if there is a need for clarification

In the area of teaching and learning, given the promises of efficiency and performance that technology has alluded to over the years, it is nice to see something that turns out to really work – along with some quantifiable evidence to illustrate just how well.

We started this way back in 2005 – I just haven’t put this out there since. I have presented it at a couple of conferences, but not put it out there where anyone could refer to it, so here it is.

Teaching Resources

I went to a presentation about open access teaching resources at ALT-C this year, and it made me think about the resources for teaching that have become available as a result of the internet. There is a massive resource to be used. I think it is unusual for a teacher in HE today not to use the web as a resource to find teaching supplements (too bad for the teaching supplement business). Isn’t this really what open access teaching resources are.

I was chatting to a colleague earlier about how we are quick to use the resources that are there, and she reminded me that we have begun using public blogs in a big way in our teaching.

In my module last year, I required the students to present evidence, followed by informed opinion, on a public blog. As a part of the weekly exercise, the students had to comment on each others work. This has formed a network of blog entries (600+) of a reasonably high standard discussing evidence based education. This year, we have started blogging with our psychology research methods students in Years 1 & 2. I think that it is early yet to see what the outcome is, but I know that, as we continue, there will be thousands of entries about studying psychology, and issues around how and why we conduct research. My colleague is using blogs in her MSc module studying consumer psychology, and is really pleased with the engagement of the students.

My colleague reminded me that these blogs are both public, and informative, and do add to the resources available for teaching. We all add what we have to the pool, and the pool just gets deeper and richer.