“You don’t follow rabbit tracks when you are hunting big
game” said Jeff Johnson, Minister of Education in the Government of Alberta at
a conference run by the Canadian Education Association. No one knows what the
big game is in education in Alberta anymore, but we do know that Jeff Johnson
is not the right man to lead the hunt for excellence and school transformation.
In response to a simple question asked by Naresh Bhardwaj (a
former teacher) about class size this last week in the legislature he said this:
“when we’re looking at quality of education and the success of the
student, the size of the class is not the
most important thing to track or to try to affect. Obviously, the engagement of
the parent is the most important, but second to that is the quality of teaching.”(my
emphasis)
Where to start?
There is a mountain of data dating from the early 1970’s
and Michael Rutter’s study of school effectiveness and the work of David
Reynolds (both of which I was involved in) looking at the factors which impact
school achievement. What we know is that there are a range of factors which
impact school performance. While parental engagement is a “nice to have” it is
not amongst the most critical in shaping school outcomes.
In terms of what we know after this forty years of very
rigorous work is that the five key factors which shape outcomes are: (a) prior
educational performance of the student – a student who has done well before is
likely to continue to do so; (b) social class and economic status – students from
impoverished backgrounds do less well than the students from high income
families; (c) school size and culture, both overall size of the school and
class size matter for a range of complex reasons; (d) whether the school is
urban or rural – which is basically a matter of curriculum choice and ability
to attract and retain quality teachers; and (e) whether the school is public,
private or Catholic and the level of support it receives for its work from its
funding source. This list comes from a comprehensive study of this question
undertaken by the Government of Australia (2004).
Class size is a complex question, as Harvey Goldstein and
Peter Blatchford pointed out in the 1990’s
(here)
– it is not just size, it is what happens in class, who is in class and the
degree of student engagement that impacts performance. There is not a simple
cause-effect relationship. However, a US analysis and
synthesis of almost 200 empirical research studies have shown that, for
specific targeted purposes, reducing class size improves academic performance.
The targeted purposes are: (a) for primary education; (b) for schools with a
high intake of students from poor economic backgrounds; (c) where there is a
high level of students in class with special needs; and (d) where the subject
being studied is known to be challenging for many students. A key condition of
success for small class size is that teachers have been trained to teach small
groups.
A meta-analysis of 77 studies published by Smith and
Glass in 1978 found that small classes were associated with higher achievement
at all grade levels. The major
benefits of reduction occurred where the number of students in the class was
fewer than 20. They concluded that small classes were superior in terms of
students’ reactions, teacher morale, and the quality of the instructional
environment.
The key findings to review are those from the detailed
analysis of PISA. These show that school climate and culture are far more
significant that specific organizational measures. While they are related,
culture and context speak more to the engaged environment of all aspects of
schooling than any specific organizational measure, such as class size (see here,
especially at page 37).
The question of teacher quality, which has preoccupied Jeff
Johnson for some time (he favours merit pay for teachers – he made this clear
when he worked with Dave Hancock on Inspiring
Education) is an important question. A variety of studies indicate
that measures of teacher preparation and certification are by far the strongest
correlates of student achievement in reading and mathematics, both before and
after controlling for student socio-economic status and that teacher quality
and qualifications count (see here).
In particular, subject matter knowledge (those who have a degree in mathematics
or science are much better at teaching math or science than those who do not), knowledge
of teaching for student engagement and skills in the effective use of learning
technologies are all seen as key ingredients for student achievement.
Surprisingly for some, the key here is knowledge of teaching and learning
processes – it is far more important than knowledge of the subject (Ferguson
and Womack, 1993).
Not all teachers are great teachers all of the time. The
biggest critics of bad teaching are good teachers. As part of any review there
is a need to look at sensible professional practice, including peer-to-peer
performance measures and continual performance assessment of teaching,
including clear metrics for raising performance year after year. There is also
a strong case to introduce a requirement for continuing professional
development (CPD) for teachers as a requirement for continuing certification. The
important thing is to have a clear, effective system that school Principals can
use flexibly but is also fair to staff.
Merit pay for teachers is not a smart idea. A 2004 study
in Tennessee showed that it had mixed success in rewarding teachers who
increased student achievement. Assignment to career-ladder teachers increased
mathematics scores by roughly 3 percentile points but generally had smaller and
statistically insignificant effects on reading scores (here).
Several meta analysis show that merit pay has little or no impact on student
achievement, but does start to change adult behaviour in inappropriate ways. Schools
have many educational goals – not only easily tested basic skills in math and
reading, but the sciences, history, good citizenship, appreciation of
literature, the arts and music, physical fitness, good health habits, and
character. In any institution with complex or multiple goals, incentive systems
that reward achieving only some of those goals (usually those most easily
measured) will inevitably distort that system’s output. Rational agents,
responding to incentives, will ensure that resources, time, and attention are redirected
to goals being rewarded, and away from those (perhaps equally important but
more difficult to measure) not being rewarded.
Thirty years ago, the methodologist Donald T. Campbell framed what he called a ‘law’ of
performance measurement:
“The more any quantitative social indicator is used for social
decision-making, the more subject it will be to corruption pressures and the
more apt it will be to distort and corrupt the social processes it is intended
to monitor.”
Since then, social scientists have documented how simple
accountability or incentive systems based on quantitative output indicators
have actually harmed the institutions they were designed to improve – not only
in education but in business, health care, welfare policy, human capital
development, criminal justice, and public administration. Merit pay is a rabbit
track.
As for parental involvement or engagement, it is the
prior education of parents and their economic status which has an impact on student
achievement, not their engagement with the school. Parents who read a lot are
more likely to provide an environment in which their children read and parents
with a post-secondary education are more likely to be able to provide learner
supports to their children than parents who did not complete their high school
education. But this is more about socio-economic status and income levels than
about engagement. According to the PISA data – consistent over time - about
50 per cent or more of differences between schools are
jointly explained by the school climate and student characteristics and the
school context. Parental engagement does not appear to be a significant factor.
So, I suggest to Naresh Bhardwaj MLA that he pay attention
to the evidence and not much attention to Jeff Johnson, especially if Naresh Bhardwaj
is seeking to make informed decisions on the basis of evidence. As for Jeff
Johnson, well he should go out and shoot some rabbits – he seems to be tracking
them.
References
Ferguson, P., & Womack, S .T. (1993). The impact of
subject matter and education
coursework on teaching performance. Journal of Teacher Education, 44 (1), 55-63.
Government of Australia (2004) School performance in Australia:
results from analyses of school effectiveness.
