JEFF JOHNSON'S PISA ENVY

The headlines have past and we can now settle down again and get back to the real work of helping students learn, engaging with communities in support of that learning and supporting our teachers who, each day, work hard to ensure that high quality learning takes place.

The headlines on Tuesday were focused on the OECD assessments of students – the results of the Program of International Student Assessment (PISA). The analysis of the 2012 results suggests that Canada was among the high-level achievers, though performance of the country's 15-year-olds in math has declined, with a 14-point dip in the past nine years. Manitoba and Alberta have really seen the greatest decline in math scores. Performance in reading has remained relatively stable, and there has also been a decline in science performance - dipping from an average of 534 in 2006 and 529 in 2009.

The good news for Canada, especially Alberta, is that we do well on measures on equity – access to and success in education: we are not a post-code edutocracy. Many other countries (like the US and Britain) are. Also good news is that the global education reform movement countries (GERM) do especially poorly when compared to those which pursue equity as the cornerstone of their educational strategy.

Now before we get excited, a note of caution. For an international test to work, all students tested have to answer the same questions, or at least questions of similar difficulty and intention. In one obvious sense, they don't: the questions are translated into different languages which, according to one Norwegian academic, "results in rather strange prose" in his country and for several others. Danish academics, when they analyzed the 2006 Pisa tests, found that eight of the 28 reading questions were deleted from the final analysis in some countries. Moreover, about half the students participating that year weren't tested on reading at all. The OECD, which runs PISA, says it calculates "plausible values" for the missing scores, and this is a standard statistical device. But it's a hard idea for most of us to get our heads round, and many statisticians dispute its validity, suggesting that the results are nonsensical and meaningless.

Further, a study by Keith Baker published in 2007 showed that there is no relationship between a nation’s economic productivity and its test scores. Also, test scores did not bear any relationship to quality of life or democratic institutions or creativity. Indeed, the U.S. “clobbered the world,” with more patents per million people than any other nation even though it has poor scores on PISA and other measures. Baker concluded that a focus on such things as PISA is an unhealthy and distracting basis for educational policy.

The PISA results often produce “PISA envy” amongst Ministers of Education, who jockey for insight and explanations on the “big PISA day” every few years. Our own Alberta Minister, Jeff Johnson, is no exception. He says he is looking around the world to understand how we can improve math education and continue to improve overall standards and he is already convinced that the issue is teacher quality. Conveniently, given that he already has plans to act on this “issue”, he sees this as the major task for 2014.  In part he is right – we already have excellent teachers who now need the right conditions of practice and support so as to be able to fully utilize their skills. Rather than “transform teacher quality and teacher education” we need a genuine partnership at the level of the school and the district  between students, teachers, administrators and parents to ensure we have great schools for all students in all areas of Alberta. This is not what Johnson has in mind. He intends to tinker and change.

But we shouldn’t let league tables, PISA or standardized tests drive what we do. We should let teachers and in-school administrators, supported by district colleagues, drive what we do. Ministers of education should ensure an adequate supply of resources, appropriate over-arching conditions of practice, an agreed curriculum framework and then get out of the way. Schools is where learning takes place and schools are the place that makes a difference. Give our schools back to the teachers and stand by to be amazed at what they can do.