What does Non-Mastery Mean?

When we measure learning, we can measure it in two fundamental ways. Is the learner able to recall/demonstrate understanding/synthesise/critically evaluate – whatever – to a pre-set standard before the learning is finished (mastery), or is the learning graded (as in a gradient) and declared finished? The question I am asking is, if we don’t expect mastery in learning, what do we get?

The reason I am asking is because of a recent experience in my final year class. I asked the students to synthesise what they had learned about various topics for me. When I read their attempts at synthesis, I was sorely disappointed. At least 80% of them simply listed the various sub components they had covered, and submitted that as a synthesis. How do you evaluate something when your students don’t actually do what you ask them to. It was apparent to me that the students had never actually been taught what it means to synthesise information. In an ideal world, I would have returned them all, talked to them about synthesis, and then asked them to try again. I don’t live in an ideal world – I live in a world where the deadlines and forms of assessment are decided a year in advance, and no changes are allowed.

This leaves a marker in a quandary – do I mark them on what I asked them to do, or do I mark them on their attempt? I have talked to markers in other disciplines and in a number of institutions, and have found that I am not alone in this. What we do is look for somewhere that we can justify giving them marks. I know that we should all mark to a carefully constructed multidimensional matrix of a gradient of attributes, but as I have written before, humans simply can’t physically keep 30 or 50 or 80 cells (adequate on originality, or excellent on structure) in working memory while we read a document – although I have met a number of educators who swear they can – supermarkers (I humbly bow to their powers).

So, what do grades actually mean? What does a “C” grade on a synthesis exercise mean? I know that when my students write blogs, I can easily judge them on a couple of dimensions (critical evaluation of evidence and how well the information is presented). I also know that when I mark something more complex, like a final year project, the culmination of 18 months work, I am challenged to clearly articulate what a “C” grade actually means.

If I stick to my original problem, a synthesis blog, what does a “C” grade mean? Does it mean that the students failed to synthesise, but managed to list all of the content they were supposed to synthesise (that’s kind of what I did). How can I award credit for something that was asked for, but wasn’t done? What is the message that we give students when we give graded credit for their work? How does a poor essay differ from an excellent essay? Which dimensions are the critical dimensions that need to be evaluated.

The articulation (or lack thereof) of the critical dimensions of evaluation is the fundamental problem with blind double marking. It is almost impossible for two markers to agree on the subjective weighting given to the various dimensions that make up an average piece of work (grammar is more important that structure etc.).

So, what does a “C” grade say to the student, and what does that component say to a future employer? How has it become acceptable for us to say that someone is educated, they have a qualification, when they can get that qualification by only doing (write a synthesis) some of what the qualification says they need to do?

Shouldn’t an ideal system expect mastery? Shouldn’t we be able to say what a graduate has demonstrated the ability to do something? Shouldn’t we support a student in their trying until the succeed instead of giving them credit for their failed attempts and then pass them on?

I worry about what non-mastery really means…