July 17, 2014

STANDARDIZED TESTS EXIST TO TEST THE TEACHERS, NOT THE PUPILS:

Sweden's School Choice Disaster : Advocates for school choice might be shocked to see how badly the country's experiment with vouchers failed. (Ray Fisman, 7/15/14, Slate)

[S]wedish school reforms did incorporate the essential features of the voucher system advocated by Friedman. The hope was that schools would have clear financial incentives to provide a better education and could be more responsive to customer (i.e., parental) needs and wants when freed from the burden imposed by a centralized bureaucracy. And the Swedish market for education was open to all, meaning any entrepreneur, whether motivated by religious beliefs, social concern, or the almighty dollar, could launch a school as long as he could maintain its accreditation and attract "paying" customers.

For a while, at least if media accounts of the reforms are any indication, things looked like they were going pretty well. Voucher school students consistently outperformed their counterparts at government schools; in 2008, the London Telegraph described the reforms' impact as "tremendous." The number of private schools increased tenfold in less than a decade, with a majority run as for-profits.

But in the wake of the country's nose dive in the PISA rankings, there's widespread recognition that something's wrong with Swedish schooling. As part of ongoing efforts to determine the root cause, the Swedish Schools Inspectorate (the equivalent of the U.S. federal government's Department of Education) called for a regrading of a subset of standardized tests administered during 2010 and 2011. In total, nearly 50,000 students at all grade levels from more than 700 schools had their tests in English, Swedish, science, and math re-evaluated.

Two Stockholm University economists, Björn Tyrefors Hinnerich and Jonas Vlachos, have been analyzing the data, and their findings to date demonstrate the many ways that things can go wrong under a market-driven education system. As in many countries, Sweden has standardized tests that are administered to all students nationwide. Performance matters for both students and the schools they attend. Students who do well will have brighter admission prospects. Schools that do well attract more (and perhaps better) students; the ones that perform poorly risk losing accreditation. However, unlike, say, the SAT, which is sent off to be graded at the testing service's headquarters, in Sweden grading is done locally, often by teachers, often at the school where the test-takers are enrolled. (This setup is not uncommon in the U.S. as well: The New York state Regents Exam is administered to all students and graded by each student's own teacher.)

It's easy to imagine teachers going easy on their own students, possibly unconsciously, motivated by nothing more than a desire to help them--a study co-authored by my colleague, Jonah Rockoff, found exactly this result in an analysis of New York Regents Exam grading. In Sweden, according to the study by Hinnerich and Vlachos, the scores issued by external evaluators were indeed harsher than those assigned by internal graders. And after accounting for things like a school's location, along with basic student characteristics, it turned out that the external evaluators had downgraded the scores for students at voucher schools much more than for students at government ones. In fact, a sizable portion of the much-vaunted outperformance of voucher school students could be chalked up to nothing more than easy grading. More surprising still, the voucher school grade inflation is almost as high for math and science (where you'd think an answer is either right or wrong) as it is for Swedish. By contrast, Rockoff et al.'s Regents Exam analysis found very little evidence of test score manipulation in quantitative subjects.  
Posted by at July 17, 2014 3:48 PM
  
blog comments powered by Disqus
« THE EASINESS OF THE FIXES THWARTS REFORM: | Main | WHICH IS WHY THE SOCIAL SAFETY NET NEEDS TO BE DEFINED CONTRIBUTION...FROM BIRTH: »