Friday 22 March 2013

Fallacies in the Practice of Rating Schools

Four years ago, we were in the position of moving to a new city. We were homeschoolers then, but expected that our kids would likely attend brick-and-mortar school for their high school years. We had the option of choosing the school they'd attend based on our choice of neighbourhood. We were not familiar with the schools in the area, and it was suggested by our real estate agent that we look at the ratings based on the Fraser Institute. We made a point not to do this, and here is why.

When the Fraser Institute rates schools, they do so based on the standardized EQAO test results for each school. I could write a whole series of blog posts on the lack of validity of this sort of criterion, but instead I will focus on what I consider to be the most important aspects of a school.

This has a great deal to do with my philosophy of what role education should play in our lives.

I value:
Creativity over standardization
Cooperation and collaboration over competition
Depth and breadth of study over short surveys in a given topic
Learning new concepts and applications over drill-and-kill and repetition
Problem-solving over computation
Reflective thought over summarization
Flexibility over standardization and textbook learning
Exposure to a variety of ethnic, religious and socio-economic groups over self-segregation
Acceptance over labeling
Divergent thought over convergent thought

When my kids go to school, I want them to be challenged to think of things in new and different ways, I want them exposed to a variety of subject areas, I want them to be encouraged to problem-solve and find their own solutions. I want them stretched and I want them to be encouraged to take risks with the support of caring teachers.

I am more interested in what is learned at school than what the test scores have to say. I want my children to be more than a series of numbers on a sheet of paper that can never describe the true value they may have to an employer or educational program.
I would rather they have a 75% in a course that made them think and challenge their assumptions than a 99% because they memorized any facts they didn't already know. I value learning over test scores, and growth over stagnation. I don't want my kids spoon-fed information, I want them getting messy, getting involved, asking questions, testing assumptions and knowing because they have seen and done it for themselves, rather than because a textbook, teacher or standardized curriculum told them so. I want them to have active, intelligent conversations with their teachers and peers. I don't want them to become "zombies" or "bots" who do not practice thinking for themselves.

I have several friends who are professors or lecturers at local universities, and they lament how students are less and less prepared for the practical and academic rigours of university education. These are selective schools, and students that enter have averages in the mid to high 90's. And yet--they are unprepared. Plagiarism is now at epidemic proportions. Although some argue that it's because it's now easier to catch, that it was always there, there is also the argument that it is now also easier to plagiarize too. I can't imagine why someone would want to spend tens of thousands of dollars to take courses only to cheat themselves out of the learning they provide. Even if your professors don't catch you, employers will see through this.
Why are students so unprepared, and only focused on that degree at the end? There are likely many factors at work here, but one that cannot be ignored is the fact that their previous (standardized, test score-based) education has not prepared them well for academia, or for life.

My sons are now at that age where I've been looking closer at Ontario's standardized curriculum at the high school level. What I see horrifies me.

In general, the curriculum has been greatly watered down. What passes for an "academic" level course covers less material and less depth than was required twenty years ago. Looking back into the elementary years, it gets even worse. Topics are glanced over, then repeated over and over again in subsequent years. What might have been a "spiral" approach in which the foundation of a concept is explored, then expounded upon in subsequent years has just turned into a confusing merry-go-round of repetition. The concern about introducing abstract math too early has caused an over reaction that makes students repeat the same boring arithmetic for years on end.

It is true that some kids aren't yet ready for highly abstract mathematical concepts early on, but if you can graph it and/or use manipulatives to show it, you decrease the level of abstraction substantially, and possibly keep the interest of students who will scream if they have to work through yet another worksheet on long division. In fact, there aren't many mathematical topics that can't be presented this way, but since they are not in the official curriculum, teachers are not compelled (allowed?) to teach them.

But I'm digressing here.

Let's look at what EQAO is and isn't.
1. EQAO is a series of standardized tests. They are designed to hold teachers and schools accountable (through public pressure brought on by making the scores public) and to measure "areas of need" which are (at least to me) very vaguely described.

2. EQAO is not a measure of creativity, problem solving or critical thinking skills.

3. EQAO does not take a baseline measurement for students.

4. EQAO is not a valid reflection of the average ability of the students of a school.

5. Many students do not write the tests for a variety of reasons, some of which are:
- part time students might not take them
- some parents refuse the tests
- some students may not fall into the category to take the tests, such as those who test out of the subject in high school years (grade 9 students who take grade 10 math, for instance)
- students who opt to take the literacy course instead of writing the literacy test
There are other issues with the validity of scores, some tied into magnet programs, ESL populations, special education, etc., and some more to do with the tests themselves.

6. EQAO is a snapshot of a student's ability to take a standardized test on a given day in a given amount of time. It does not take into account: test taking ability, issues specific to the student (sleep, health, etc.), how many hours were spent drilling and teaching to the test in class, was the sun shining in their eyes during the test, etc. It may or may not be an accurate reflection of a student's accomplishment in the area it is designed to test.

7. Like many other standardized tests, questions on the EQAO tend not to be open-ended and also tend to rely on a single type of approach to a given topic. This makes the tests easier to mark in a consistent manner, but decreases the potential for evaluating the student's thinking strategies.

8. EQAO scores are used for other things, such as to drive real estate markets and put pressure on schools to conform to the standardization model. By making scores public, the Ontario government uses the community, particularly parents, to put pressure on schools to perform.

9. To those who still say "they test the skills our kids need" I would challenge them to define for themselves what those skills really are, and to look at the tests themselves (past tests are available online) as well as any statistical data they can find and see if that, in fact, is truly the case. I can't tell you what you want for your own child; I only speak for my own and my own educational philosophy.
The validity of standardized tests as a measure of cognitive achievement in general is often questioned, and it is best if you look at the data based on these particular tests.


In order to better measure the caliber of a school would require an initial assessment of what the students know and understand upon entering to create a baseline condition. Then, evaluating the student's increase in understanding based on the courses studied would better show the actual learning that has happened over that period.

Of course, there are problems with this as well, as there are some types of learning that are not easily measured, and in many ways, these are the kinds of learning that often best serve students throughout their lives. Things like empathy, social awareness, self awareness, critical thinking skills, creativity, collaboration, problem solving, etc. are not easily measured by any standardized test.



I would also argue that instead of the literacy test, a more useful approach to addressing the issue of literacy would be to have students write a shorter version of the major essay required in the IB (International Baccalaureate) program. And, as in the IB program, essays would be sent to random areas to be marked by a total of 3 teachers per essay to ensure fairness, consistency and validity of the evaluation.


My kids have not written an EQAO, and I would not be surprised if they never do. I will leave it as their decision in high school, since not writing the literacy test means either wasting a credit taking the literacy course or forfeiting a secondary school diploma. Contrary to popular belief, one does not need the diploma to enter most post-secondary programs, but some scholarship opportunities do require one. Grade 9 math teachers sometimes count the test scores as part of the grade (often about 10% of the overall mark), but my kids will not be in a position for that to be an issue.

The move towards standardization is also a move towards what happens in education in the United States. In the U.S., students spend so much time writing high-stakes standardized tests, that the teachers spend nearly all of their class time prepping the kids for the tests. In some areas, recess has gone by the wayside as teachers find their jobs threatened if their student's scores fail to rise. In such a situation, there is little room for exploration and true learning.
It is true that the EQAO tests do not happen nearly as often as the U.S. tests, but there still is a tendency by teachers to throw sound pedagogy out the window in order to drill kids and try and boost a school's scores, and "reputation" since so many people unfortunately buy into what the Fraser Institute has to say about these things.

As far as I'm concerned, you can have your test scores, and your ratings. You can keep your curriculum too. We have found creative ways to bypass the worst of the system, and I challenge you to find ways to make it work for you and your kids too, whether it means homeschooling, or working with the schools to help make them better.



No comments:

Post a Comment