At the start of the school year my drive to work changed. It changed from an Auckland motorway drive of 20 minutes to a drive through South Canterbury farmland. My drive currently includes several majestic peaks covered with a light dusting of early season snow.
What hasn’t changed is Morning Report. The morning news radio show introduced to me by Granny Boon during my stays with her in the 1970s. Back in those days the political bacon definitely got grilled. These days there isn’t so much grilling – maybe the occasional braising – but not so much grilling.
It was fantastic the other day to hear Kim Hill back in the hot-seat grilling away like a George Foreman. In particular her pasting of our esteemed minister Hekia Parata following the release of national standards.
Of particular fascination is the portion towards the end of the interview where Kim clearly asks Parata what the margin of error in the standards are. The minister’s answer is telling.
“I don’t know.”
Sorry? You’ve just released what you consider to be the most important statistics of the educational year. Thousands of children measured, loads of data imputed, and hours and hours of work by teachers and bureaucrats later and you can’t tell us the margin of error.
I remember 6th form statistics. The margin of error is one of the first things you learn about. Admittedly this is a very high sample, so you would expect the margin of error to be fairly low (although when you think about how the measurements are made, serious statistical analysis, and common sense, tell us that much, if not all of this information is bullshit).
A quick 10 minutes on the Education Counts webpage sheds no further light on the margin of error for these statistics. There are plenty of statistics featured, but no mention of the margin of error.
If we look at the table of “statistics” opposite we can see the much trumpeted (not by me) lift in overall achievement of all our students. Unfortunately for the government, these statistics are about as meaningful as a line of dialogue from The A-Team.
And this point must be made: if our missing margin of error is above 2% then the so-called “rise” in the number of children meeting the standard is non-existent. The figures could have stayed the same or even gone down. We will never know.
So much trumpeting and so little questioning (except from Kim). These are the questions I want answered:
- What is the margin of error?
- If the margin of error is more than 2%, does that make these figures even more meaningless?
- How do overall teacher judgements, unmoderated between schools and regions (and possibly across schools), impact on the margin of error or the figures as a whole?
- Has there been any statistical expert comprehensively analyse these figures?
- If so, who are they and where is their analysis?
- If not, why not?
I’m sure regular readers have many other questions, none of which were asked by our main media players. This isn’t surprising since they suck up whatever Steven Joyce, Cameron Slater, David Farrar, Matthew Hooton and Simon Lusk think they should.
To conclude, here is quick off-the-top-of-my-head prediction for the release of the 2013 national standards at this time next year: add two-ish percentage points to each of the above figures.
And there’s your election sell from National right there. Look! The standards work. We work. You can trust us with your children.