The media has put me in a foul mood today. Preliminary results have been released reporting on NAPLAN, and the reporting on it is pretty poor. For example, a story from the ABC, South Australian NAPLAN results behind the national average, relies on terms such as ‘below average’ and ‘worst performing’ without any sense of explanation of what that means or providing any adequate context.
Surely half of the students were below average, and half above average. Every teacher knows the truth that there is no such thing as average. So what does the report mean by average? Do they mean ‘meeting the acceptable standard for their age’? Do they mean, being in a middle band of achievement, with the parameters defined by ACARA? We don’t know, because they don’t say.
They don’t ask, or answer the question: What does this mean our kids do or don’t know? What, in real terms, can children in years 3, 5, 7, and 9 not do that they should be able to do?
These are important questions. If students are demonstrably failing to learn in schools, that is a problem. But are they demonstrably failing to learn? What skills are they failing to learn? How does the NAPLAN test demonstrate that failure, or success? Are there skills that students may or may not be learning that NAPLAN does not, and maybe cannot, report on?
These are also important questions, that once again are not being asked.
What is reported on is that particular groups are below some unspecified average. That some perform worse than others.
We could say that without NAPLAN, couldn’t we?
So ask the question – why don’t we? Is NAPLAN really an effective tool? Do we need it to build a successful education system? The much lauded Finnish system does not have it, and they are MUCH LAUDED.
Scharlach’s START framework (Students and Teacher Actively Reading Text) breaks down comprehension skills into 8 distinct and linear stages:
3. making connections
5. determining main idea
7. checking predictions
8. making judgments
Students also utilize a metacognitive reading journal where they are prompted to record their thinking according to these 8 stages. In her study, Scharlach concludes that it is the metacognitive step of the reading process that sees the greatest gains in later test performance.
The Engagement Problem
Unless the texts chosen for reading comprehension practice are of particular interest to the students, it is hard to find anything truly engaging about preparing for NAPLAN. If you consider engagement according to Schlechty’s model, borrowing from Daniel Pink, student motivation that hangs upon scoring well or simply following teacher instruction because that is what one does, will lack the long term effectiveness a teacher should strive for (or a student will enjoy).
Learning any skill ‘because it is on the test’ will always come a distant second place to learning a skill because it is in one’s immediate interest to learn it.
Games-based-learning is a education trend I have been wary of adopting, but I think there is real potential for interactive fiction to give kids engaging games to play that provoke an immediate need to learn comprehension skills. If kids want to win the game and progress through a text adventure successfully, the same comprehension skills required by NAPLAN are the ones most likely to help them solve the various puzzles they will come across.
Modifying START for IF
The START framework provides prompts for the reading journal to help students with their thinking as they read. In order to successfully modify this framework to suit interactive fiction, I propose adding alternative prompts to aid with interactive fiction problem solving. In a classroom, I would likely have these on a poster, with their original START counterparts, so students can be made aware of the cross over these skills have into all reading.
NB: In the context of IF, the reading comprehension strategies become useful at the point when a player is stuck. This usually means they have encountered a puzzle of some kind.
In this chapter I think…
Predicting / Inferring
I think my immediate objective is…
In my mind I see…
If I explore the environment I see…
This reminds me of…
The objects I can access can be used to…
People usually do or say things like that when…
I think the most important thing…
My priorities are…
In 10 words of less…
The current problem I have is…
My original prediction…
I have tried…it did not work because…
My favourite part…
I am / am not pleased with that solution because…
I would recommend playing through one text together as a class, with the teacher modelling these strategies. I think Lost Pig is a wonderful place to start (play it online here). Once the thinking / comprehension strategies have been conveyed and practiced, I’d further recommend continuous free exploration of other titles throughout the year, letting kids choose their own adventures. If you are in the habit of assigning weekly / fortnightly comprehension passages and accompanying worksheets / quizzes, stop those immediately and replace them with interactive fiction.
Standardised testing is a problem. There’s a lot you can read about that, particularly Testing Times: The Uses and Abuses of Testing, by Gordon Stobart. My particular bugbear is with NAPLAN, and Ian Hardy at QLD Uni has written some interesting stuff criticising that, too. Track down his paper, ‘The Logic of Enumeration’, or ‘Catalyst data: perverse systemic effects of audit and accountability in Australian schooling’, by Lingard and Sellar, for a detailed critique of Australian testing.
To my understanding, the chief problem that NAPLAN gives rise to is that what is measured and reported on from the perspective of teacher accountability is the thing that will be seen to matter most, and therefore receive the most attention from teachers. The reason that this is a problem is because the literacy and numeracy focus of the testing is remarkable narrow. Despite having a brand new, and quite good, standards based national curriculum, what matters most according to these tests are skills that receive footnoted attention in some cases (see http://www.australiancurriculum.edu.au/Glossary?a=E&t=comprehension%20strategies for an example).
Another problem is that preparing for these tests is boring. Kids hate it, and quite rightly see no value in it (there’s nothing in it for them, really – it’s to keep their teachers and schools accountable, not give them any tangiable, transferable reward like an ATAR score, for example). I can assure you that teachers hate it, too. Sure, we can use the data to see how our kids are going, but we have many more reliable tools in our toolbox than standardised testing. Standardised testing is not objective, it’s not reliable enough, it’s far too simplistic (multiple choice reading comprehension? You’ve got to be kidding?!), and it is far too narrow. Kids should be learning critical thinking and analysis, collaboration skills, authentic problem solving skills and a whole host of other desireable 21st century skills (despite whatever the politics of that phrase might do for you…). These don’t test well. They don’t end up on standardised tests. They never really get the attention they deserve. Instead, our kids perform tasks on tests that they will never be called upon to repeat again, and only for political purposes.
As much as I would like to see the demise of this form of testing in Australian schools (and worldwide, but I’m a NIMBY at heart, so lets do it here first), I also doubt the current policy makers to make such a sharp U-turn on what is still a newish venture for them. So my current thinking is that the best approach to the tests, particularly the reading comprehension test, is to refuse to drill and kill, but to add so much sugar to the spoon that the medicine going down better be for diabetes.
I am aware that I need to work on my metaphors.
The standard approach to prepping for the reading comprehension paper is to use student workbooks from various educational publishers and go through their worksheets over and over again. Lots of reading short prose and then answering multiple choice and short answer questions. Drill the skills over and over, and kill any enthusiasm a child might have for reading.
The approach I propose is to replace all the time spent working through those workbooks with allowing the kids free rein to play through interactive fiction titles of their own choosing. There is scope within this to teach practical comprehension strategies, like those proposed by Scharlach in the START framework, but teaching them as methods for solving in game puzzles rather than scoring well on tests. The prediction, visualisation, inferential and empathetical skills required to score well in NAPLAN are the very same skills required to be successful in playing IF. And IF is a lot more fun. More resources on this approach, and others, to using interactive fiction can be found at Brendan Desilets’ excellent site, Teaching and Learning with Interactive Fiction.
More on this subject to follow over the next couple of months…