To understand. Simple enough.

I would be willing to wager (based on inside knowledge here – I’ve had the privilege of sitting on dozens of validation, revalidation, and quality assessment panels) that at least 80% of all learning outcomes use the word – understand.

We can define the word – perceive the intended meaning, significance, explanation, or cause of something. We teach it. We talk about it, or the lack of it in our students. We decry current study techniques that ignore it. It lies at the heart of what we do, bit can we operationally define it so we can measure it?

One of the first things I teach my students about the measurement of any psychological phenomenon, in the importance of operationally defining it. A good operational definition allows two observers to rate an interaction with high inter-rater reliability. Two people will return the same score when making the same observation. Sometimes you have good operational definitions, sometimes you get poor operational definitions, but you need one in order to have consistency in the measurement.

As an example, you might operationally define an interaction between two people as occurring every time their eyes met. This ignores any other kind of interaction (working together, body contact etc.) and only allows for a single item to be classed as an interaction, but you will get good, reliable measures when two raters observe the same event. It somehow feels phoney, but that’s what is done – I’ll come back to this later.

Understanding is a psychological phenomenon. It is something that takes place within an individual, and has to be inferred through some kind of observable behaviour. In formal education, we infer understanding through an assessment of some kind. There is a hierarchy of assessments that demonstrate understanding with true/false of multiple choice questions at the bottom, and oral examinations at the top (where you can probe for understanding until satisfied). The usual candidate for assessing understanding is an essay, either as coursework or an exam. However, what are we looking for when we say we are looking for understanding? What is our operational definition? How do we tell a novice marker what it is that we want them to reward?

Is what you are looking for in understanding the same thing that I am looking for in understanding? If we aren’t looking for the same cues to convey understanding in our students, which one of us is right?

According to Wiggins & McTighe, there is a matrix of 30 cells that have to be taken into account in order to represent understanding. These include such clear and precise statements as “a powerful and illuminating interpretation and analysis of the importance…” or “generally unaware of one’s specific ignorance”. Given that understanding is usually one of several attributes that are included in any good marking rubric (along with originality, synthesis, flow, grammar etc.), and each one of these attributes would be multifaceted, how many cells would we have to keep active in our working memory while reading through a piece of work. Good solid research tells us that we can only keep 5 ±2 items in our working memory at once – although I have met a number of lecturers who are particularly gifted, and insist that they can, and do keep all 1,987,392 cells of information necessary to perfectly grade a piece of work active in working memory simultaneously, in addition to their perfectly attuned scale so that they can differentiate between a 56% and a 57% on a piece of work – I’m afraid I must admit that I’m merely human, and 5±2 is my limit.

When something as ephemeral as understanding is pushed into an operational definition, the result is as phoney as defining human interactions solely as when two peoples eyes meet. Yes, you get good inter-rater reliability, but no one really believes that what has been carefully operationally defined captures real understanding.

And so we measure recall. If you can accurately recall the information that has been provided in the lectures and notes, you do okay. If you demonstrate understanding – whatever that is – you get higher grades. Might as well cut out all the reading and just go directly to MCQs.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s