Month: March 2018

Part 2: Unintended consequences and perverse incentives

In this second of a series of three blogs posts, I want to argue that one of the consequences of an increased reliance on standardized tests can be more privatization of education. That is the way it has played itself out in Britain, and has led to a quasi-privatized 2 tier education system. In Nova Scotia, Bill 72 sets the stage for the same process. Although the Glaze recommendation for a separate office for standardized assessment may have been jettisoned after negotiations with the NSTU, other changes have set the stage for more testing in the future – the demise of school boards, the centralization of authority, and making principals managers.

As Grant Frost says in his latest article, https://frost18.com/a-voice-3/, the privatisation wolves are in the door, and are curled up on the carpet. The passage of Bill 72 has set us on the road to adopting a neoliberal agenda for education which has been in think tank AIMS’ sights for years, and has been implemented in countries all around the world. AIMS promotes market based thinking, just like the Global Educational Reform Movement (GERM) which foists a business model on education. The policy proposals include competition (both between schools and among teachers), choice, test based accountability, performance related rewards and the weakening of teacher unions. The claim is that a smaller role for government will produce “efficiencies” and “innovation” – but the British experience is very different. All of these are invitations for private enterprise to jump in to claim a slice of the pie.

Britain has for a long time had a system of high stakes examinations that decided which children would do academic work in high school (the 11+), which children would have to leave school at 15, O (Ordinary) levels), and which would qualify for higher education A (Advanced) levels). In the 1990s, education reforms resulted in adding a new set of exams for age 7, standardizing them across the country, all the while publishing results in League Tables. Once exam results were available for public view, tests went from being high stakes to stratospheric as parents in larger centres became like consumers, choosing the “best” schools for their children. And because of their traditional reliance on exam results, few outside the education system in Britain questioned their merits.

Once Britain had achieved “test based accountability”, the next step was intended to raise standards, particularly for those children considered “disadvantaged”. A Labour government was in charge, and there was a lot of rhetoric around making sure that every child had “equality of opportunity”.  The poorest performing secondary schools were to be closed and then reopened under the management of “trusts” which would inject new money into them. Sponsors who could invest at least 2 million pounds ($4m) were sought,  and these new “academies” were given autonomy over finances, staffing and admissions. Public funding would continue to flow, but otherwise, these academies were like private schools within the public system, outside the authority of the Local Education Authorities (LEAs). Successful academies were encouraged to expand and take over more schools, thus becoming Multi-Academy Trusts (MATs). When the Conservative coalition took over in 2010, their education policy included expanding MATs to any secondary school that wanted to convert, with special provisions for those schools deemed “outstanding” who could convert without oversight (now called Converter academies). The Labour government had previously given up the idea of requiring sponsors to invest money, as there was a shortage of people willing to do that, and all trusts were now officially charities.

As the population rose, new schools were needed, and in order to continue to supply more choice to parents, the Conservatives borrowed the idea of “free schools” from Sweden. These were basically charter schools, set up by groups of community members and were publicly funded, but with full autonomy, just like academies. They were also free to implement their own curricula, often religiously based.

By 2017, there were over 3000 MATs including some primary schools. The government has decreed that all schools will be academies by 2022.

Julian Astle, researcher with the Royal Society of the Arts, has described the British education system as a game of “whack-a-mole… Our school system, with its focus on tests, targets, league tables and inspections, is full of unintended consequences and perverse incentives. It has become such a game that it is forcing teachers and school leaders to choose between helping pupils and helping themselves.” https://www.teachingtimes.com/articles/cheatingteachersslt84.htm

He was referring to the phenomenon of teachers “cheating” on tests to help their students – something that becomes more and more prevalent as standardized tests become more high stakes. It could, however, refer to the British system as a whole, because every day’s newspaper brings new headlines about scandals in the operation of these “trusts”.

Have the intended consequences have been achieved? Have there been unintended disasters? The key issue is what is the opportunity for profit – “OP” for short:

Have standards been raised? Although results on internal British exams have been steadily rising, with a greater percentage across the board achieving “proficiency” in the core subjects, PISA (international comparisons) results have improved only slightly. See previous post: https://progressiveeducationnovascotia.wordpress.com/2018/02/22/joy-in-learning-good-for-students-and-teachers/ And because of the government’s decision to limit the subjects that will be listed in the League Tables, schools don’t have any incentive to offer a wide range of subjects that will not be “counted”. Thus, the narrowing of the curriculum within subject areas (teaching to the test) that I have referred to elsewhere has now grown to include whole subject areas, especially the arts and foreign languages. (OP – private arts classes, independent schools with enriched arts programs serving affluent families)

Several recent studies have shown no difference in equity between schools managed by Local Education Authorities and those managed by MATs. Other studies show that there is more segregation between advantaged and disadvantaged students in areas with a prevalence of academies, “and this is especially true of the more recent Converter Academies. Converter Academies, on average, take far less than their fair share of disadvantaged pupils.” Gorard,2014  http://dro.dur.ac.uk/12119/

Has “academization” made the U.K. education system more efficient? When there were 152 LEAs, there were 152 leaders, making the top salaries. Now with 3000 MATs, there need to be 3000 leaders, making the cost per pupil for Mats leadership almost 10 times as for LEA leadership. In addition, by leaving it up to MATs to pay head-teachers what they want, head-teacher salaries can also be astronomical. (OP – headhunters employed to find CEOs, “super heads”). “Efficiencies” are instead made at the classroom level, with less money for resources and teachers (recent budget cuts have had enormous impacts on the classroom).  Although there is a recommended salary scale, performance pay (based on pupil scores – again standardized tests) can be used to depress teacher salaries. Just today, the Guardian reported that some of the worst gender pay gap offenders are MATs, with their large proportion of female workers. https://www.theguardian.com/news/2018/mar/25/gender-pay-gaps-in-academy-school-chains-among-the-worst-in-uk.  In addition, because each trust is responsible for its own financial accounting, contracts and legal work, there is lots of new work for accountants and lawyers. One trust was reported as paying over $100,000 for 6 months accounting work. (OP – legal, accounting firms, “information management” firms eg SIMS)

Britain’s firmly entrenched culture of test–based accountability certainly does set up a competitive dynamic between schools, and the reforms have created lots of “choice” to fuel it, but this has led to the “whack-a-mole” problem of the unintended consequences. Because parents want their children to go to the best secondary schools, they apply for the ones with best average test scores. The new academies are allowed to manage their own “oversubscription” policies, ie choose children most likely to boost their ranking on League Tables. In order to get their children into the “best” secondary schools, parents who can afford it engage tutoring agencies to boost their marks (OP – tutoring agencies, private tutors – perhaps teachers leaving the system?). When the government bans the use of marks and interviews for admissions, academies can become “specialist” schools i.e. claim to focus on STEM subjects, the arts or foreign languages, so they can then devise ways to test for aptitude in those areas.  This leads to the question – what’s the difference between aptitude and parents with money for private lessons? Another quality that schools might look for is well off parents who will be able to contribute to school fundraising…so they start having “parties” for prospective families (thus killing 2 birds with one stone, since they can suss out the students at the same time). So the government bans parties. Sometimes schools even resort to changing the boundaries of their catchment areas to cut out less advantaged neighbourhoods. And so it goes…

 

To be continued…. Next post: the sorry, 2 tier state of British education today – winners and losers. And what NS can do to prevent a similar fate.

The large number of rough sleepers amidst all the wealth of the colleges is a testament to the inequality which plagues Britain (and its school system) today. This doorway is someone’s home.

Part 1: The Problem with High Stakes Tests

One of the recommendations in the Glaze report, was for an external office to oversee standardized testing, a “Student Progress Assessment Office”. But what do we mean by standardized tests? How are they different from ones the teacher makes up?

Standardized tests are created outside the classroom, and are used to compare students, classes, programs, and policies. They are often created by departments of education, or by private business interests (textbook publishers are major players in the industry). A useful distinction is based on what happens with the results. High stakes tests are those whose scores are used to assign serious consequences for educators, students or schools. Test scores which are used for “streaming” children into various levels or to determine students’ career prospects at an early age are a prime example of “high stakes”. High stakes tests can also be used to determine consequences for schools. In Britain, where children sit standardized exams known as SATs every 4 years, League Tables publish every school’s results, and those schools at the bottom can be deemed failing, and closed. Teachers can be judged based on their students’ performance on these tests, and sometimes paid accordingly. Children can be denied access to schools and programs based on them – in Britain, SATs are extremely high stakes. More detail on the negative effects of high stakes tests can be found in previous blog posts. (Joy in Learning)

The purpose of low stakes tests is diagnosis – the objective is to assist learning by providing information to administrators, to teachers and to the takers of the tests which can help them improve. The main comparison is between an individual’s or program’s progress over time. For example, a teacher may administer a diagnostic reading assessment to an individual child, looking to assess their strengths, progress since the last assessment and their needs. When these results are acted upon, there is a direct benefit to the student. Sometimes tests are administered to a representative sample of the population to assess how a program or a teaching approach is working. These can be very valuable to educational planning and curriculum development, but the results of individual students or teachers are not singled out.

My criteria for a good assessment tool is simply this: does it help students learn? Most high stakes tests do not help the students who take them improve their own learning – and are not intended to. Either students get results too late to do anything about them, or the results are simply a grade or number, with no explanation of how it was achieved.

Low stakes tests, on the other hand, can help students, teachers and administrators learn. Large scale, randomly sampled tests can give us valuable information about policies and programs that work. The Programme for International Student Assessment (PISA), administered by the OECD for the past 18 years, is an example of a test that tries to do precisely that. It is given to 15 year olds around the world every 3 years and measures “what students know and can do” in reading, math and science. Because it also gives students and teachers questionnaires to gather data on many other aspects of their lives, such as socio-economic status, attendance records, attitudes towards school etc., PISA data can be used to analyze which factors matter most for learning. And because it is low stakes, and is only administered to a sample of students, there are no adverse repercussions of low scores for the takers of the tests or their teachers – which means that results are not skewed by “teaching to the test” and drilling students in how to maximize test scores.

During my university days, I took courses in test design, and an important thing I learned is how difficult it is to design a test that is truly objective – cultural, class and intellectual biases are inherent in most standardized tests. I was happy to teach at a school for most of my career that did not rely on them, and in fact used a variety of “authentic assessment” measures designed to help students learn from their work. I look at all standardized test results with healthy skepticism – they can be a useful diagnostic when well designed, but even low stakes tests on randomized samples are blunt instruments for measuring a school’s or program’s worth.

Because of these experiences, I am always amazed when I hear government or business people talk about standardized tests as if they were some kind of holy grail of measurement. I find it remarkable when a business leader (who has no personal experience of education within the last forty years) can talk about a “national emergency” because test results drop by 2%. https://www.theglobeandmail.com/news/national/education/canadas-fall-in-math-education-ranking-sets-off-red-flags/article15730663/  Apparently, to some people, these tests are more “objective” than teachers’ feedback – an attitude that has contributed to a lack of trust of teachers. Educators know how unreliable they are; plus, they see the fall-out of high stakes tests in the narrowing of the curriculum, children’s stress, and the sometimes drastic consequences that follow. Educators around the world are leading the charge against high stakes tests.

PISA, on the other hand, is meant to be low stakes and has over the years produced evidence-based, statistically significant connections between policies and academic results. Each round of testing produces about 6 huge volumes of analysis.  For example, they have found that countries where “streaming” (based on high stakes tests) is practiced perform less well on average. Years of early childhood education are positively correlated to PISA results. “PISA 2012 also finds that the highest-performing school systems are those that allocate educational resources more equitably among advantaged and disadvantaged schools and that grant more autonomy over curricula and assessments to individual schools.” https://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf  PISA has even pointed out the positive impact of principals and teachers collaborating – something our government has chosen to ignore.

Sometimes there are problems with the way PISA compares countries, particularly when certain cities in China are considered as separate countries, and then not surprisingly get the highest results in the world (Shanghai for example). Mary Campbell’s article in the Cape Breton Spectator points out some other problems. https://capebretonspectator.com/2018/02/21/pisa-assessment-glaze-ns-schools/  However, PISA has produced some valuable directions for educational policy makers, if they choose to listen, and is a rough barometer of how our schools are doing compared to the rest of the world. Canada does exceptionally well on PISA, and Nova Scotia holds its own within Canada as a small, less well-off province.

Test scores can be seriously misinterpreted – as I have discussed in a previous blog post, “Why a College of Educators?”. We have just seen how the Glaze report has used them to justify its recommendations, and how the government has incorporated them into Bill 72, the Nova Scotia Education Reform Act, which was passed yesterday.  But nothing in Bill 72 will improve standardized test results (nor will it help children learn, which is not the same thing). The biggest impact may well be an increase in the government’s ability to impose more tests on the school system, to try to control teachers and their “managers” in finer detail, to impose new programs on schools and soon to justify more privatization. It has happened in Britain, and it’s not pretty. It will happen here if we do not fight the implementation of this bill.

Oxford lecturers, on strike for several weeks, have now won their battle…the university will keep their pensions the way they were.