Formative Assessment

In one of my earlier posts, I talked about effort in recall being a fundamental principle of learning. The testing effect is a good example of effortful recall – you have to work hard to remember the answers for a test.

I was giving a presentation to the faculty of the business school about using clickers in teaching, and demonstrated the importance of using something like clickers as a method of building in formative assessment in the lecture.

From the Roediger & Karpicke (2006) paper, I did the following to make the point. First I explained the study very simply.

There were three groups in the study (see Figure 1). In one group, the subjects read through a piece of information and studied it for ten minutes, took a five minute break, and then studied again, repeating the process four times. After a final break, they were tested on the  material. The second group did the same thing, but took a test in the final ten minute slot. The third group studied the material for one ten minute slot, and then after the break were tested on the material. This was then repeated twice more. Finally, all the subjects were brought back after a week and tested again.

After the session was over, the subjects were tested on the information. Their results are in Figure 2

I asked the faculty at the business school to guess at how well the subjects did on their tests a week later. Needless to say, the figure of their results was a real surprise to most (if not all) of them (see Figure 3).

I think this study makes a compelling case for formative assessment. Asking the students to recall information shortly after they have learned it makes a significant difference to their long term retention.

A few years ago, I used my weekly statistics tests as a means to measure the effect of participating in a formative session prior to taking their test. I asked them, on their test, to indicate whether or not they had been to the formative session that week, and whether or not they had studied before the test (they were open-book tests). Across the 350 students, there was a massive effect of attending a formative session (see Figure 4).

For years formative assessment has been pushed as an important aspect of education. I think that the concept of effortful recall is one of the underlying psychological principles that makes formative assessment so powerful as a leaning tool.

Refs:

Roediger, H. L. & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.

The Tyranny of Content

Stuff – that’s what we teach – stuff. Stuff and more stuff.

We live in the information age, where information (and good quality information at that) is widely and freely available to more and more of us. Certainly the availability of information to university students is at unprecedented levels. And yet, our teaching models are still largely based on presenting information.

I have colleagues who complain about how full their modules are, how they can’t do anything different than straight lecture or they won’t cover the material, feel overwhelmed  by new material that needs to be incorporated because there is just so much old material that the students need to know. I call this the tyranny of content.

In HE, we decide the content. As professionals and experts in our respective fields, we decide what is important and what is not. We put together a syllabus, we assemble the learning objectives and declare the learning outcomes. We decide the content. If someone else is doing that for us, we become trainers rather than professionals.

Given the amount of information that is available to teach, we can never hope to design a programme that would cover it all. So why are we trying so hard to do just that?

We need to remember that the ‘H’ in Higher Education stands for higher order skills (critical thinking, critical analysis, synthesis etc.). How can cramming more stuff into a syllabus help students gain higher order thinking skills? I know that there are a few enlightened lecturers out there who focus on the higher order skills, but most of the faculty at HE institutions are focused on stuff.

There are simple reasons why this has happened. Stuff is easier to teach and assess. Stuff is what the students demand – they want a drip feed of facts that they can spew forth on demand. Stuff easily fits into the measurable metrics administrators demand. Stuff rules! We are slaves to the tyranny of content.

Higher order skills are incredibly difficult to teach (or can be). They are very hard to assess when the paucity of lower order skills gets in the way of evaluating what the students are trying to say. Students find higher order skills meaningless and boring (or think they do). As my last post indicated, we are shaped by the systemic expectations and reinforcers that surround us – and these rarely include higher order skills.

When someone succeeds in teaching and assessing higher order skills, we gaze on in admiration, awe-struck by their accomplishment, and envious of their success – shaking our collective heads and wishing we could do something like that before turning back to our stuff.

I every field of knowledge there is expertise. An expert knows a lot of stuff, but more importantly, they approach problems both within and outside of their area of expertise in a different way than a novice would. It is how they look at the problem, how they approach the problem, how they organise the problem that confirms to you that they are an expert in some related field. This critical examination of the problem, the organisation of the information, and then careful approach to addressing the problem is what we really need our students to learn. It is how we study the problem, not the problem itself that is important, and yet, we focus, almost exclusively, on the information around the problem, the right answer, the solution.

We need to somehow rethink what we do in HE so that we can shake off the tyranny of content. Our students have access to as much information as we have, and yet we often insist on repackaging and presenting it in our our own image. We need to refocus on the higher part of higher education.

Performance and Learning

I mentioned in my last post that Bjork presents ways to make knowledge transference an integrated part of learning. In his chapter, Bjork alludes to conditioning as a primary problem in why transference does not happen naturally.

Transference is a difficult thing to learn. It’s easier to just learn the facts, and parrot them back again. In order to learn transference, the learning needs to occur across different times and settings. Students don’t like that.

In psychology, one of the principles of learning is shaping. Using shaping, you can get a bear to ride a bike (not any more, but it used to be done all the time). How shaping works is you provide a reinforcement (reward) for behaviours that are close to the behaviour you want. Once that behaviour becomes commonplace, you change the criterion for when the reinforcement is given to something that is closer to the behaviour you want. This cycle continues until the bear is riding the bike. So in our example, you give the bear treats when it goes near the bike. Once the bear associates the bike with treats, you only give them a treat when they touch the bike. Then you change the criterion to picking the bike up, and on, and on… Pretty soon (or maybe it takes a long time) you have a bear riding a bike.

What does this have to do with teaching?

Behavioural scientists have shown that all organisms (including us) respond to conditioning techniques. Shaping is a conditioning technique. We find ourselves in a shaping cycle with our students every time we deal with an assessment. You set an assessment to provide a learning opportunity for your students, and their performance on that assessment comes back to shape your behaviour as a teacher. If your students do well on their assessment, there is positive reinforcement provided to them for learning what you wanted them to learn (they get good grades). As greater numbers of your students achieve your desired assessment outcomes, you receive positive reinforcement through feedback from students, feedback from colleagues ect. that says you are doing a great job. In many institutions, student feedback is made public and plays a direct role in promotions and tenure. That is a powerful reinforcer. You want to do well.

As a result of this powerful reinforcement cycle, your teaching and assessment is tailored to what the students excel at, and they excel at the assessments you set. What we end up doing is maximising performance, not maximising learning. In order to maximise learning, difficulties (desirable difficulties as Bjork calls them) must be introduced so that students can learn things in new settings and times. This is what transference is all about.

Our whole education system is designed to maximise performance rather than promote learning.

Teaching – to what end…

One of my students wrote about teaching for transfer. The evidence is dismal for secondary school achievement in this area. I looked for the OECD reference, and couldn’t find it, but 98.2% of UK 15 year olds unable to generalise their knowledge – pretty abysmal. I wonder if we do any better in HE? I doubt it, as we tend to use the same methods of teaching and assessment that are used in secondary schools. Bjork presents some great research about how we can improve transfer in our teaching, and why most teaching doesn’t succeed. I have to agree with Butterfield and Nelson (1989) who said that “educational institutions that do not promote flexible use of their teaching are inadequate”. We don’t just teach for the sake of teaching.