Researchers question Fraser Institute school rankings

June 29, 2006, volume 36, no. 5
By Ruby Ng

Document Tools

Print This Article

E-mail This Page

Font Size
S      M      L      XL

Related Stories

Three education researchers at Simon Fraser University advise the public to be cautious about how they interpret the Fraser Institute's B.C. ratings of public schools.

Using well-known methods from statistics and the science of measuring psychological attributes, professors Phil Winne and John Nesbit, and research associate Carmen Gress, illustrate how such composite ratings may not be precise enough for practical decisions like choosing a school.

Their report Cautions About Rating B.C.'s Schools highlights some of the issues.

The trio points out that such school ratings inevitably have some built-in errors. “As a result, any particular school's rating is somewhat unreliable,” says Winne.

He and his colleagues demonstrated that “the odds are 19 to 1 that any particular elementary school's rating could be about 1.1 points lower or higher than what was published in the Fraser Institute's report about elementary schools in British Columbia.”

The Fraser Institute also rates secondary schools in B.C. based mostly on students' scores on Grade 12 provincial exams. Winne and his team show that ratings based mostly on just one kind of measurement may be misleading.

For example, using straightforward statistics involving provincial exam scores and teachers' grades, a student with an average score on the Grade 12 English provincial exam might receive a grade from their teacher as high as A or as low as F in that subject.

“Teachers can observe a student's work over a much broader range of the curriculum and can use a wider variety of methods to assess knowledge than is possible in a one-shot test,” says Winne.

“But the standardized test has some technical advantages over the teacher's grade. So which measurement should you believe? It's important not to over-interpret what a single score can reveal.”

“We don't claim ratings, grades, and standardized test scores are useless,” Winne observes. “What we urge is not to overlook the particular limitations of any kind of rating of students or schools.”

The SFU researchers offer several practical reminders to parents: “Don't rely on a single indicator when judging a school. A variety of factors contribute to a successful learning experience. And, bear in mind that every measure has some drawbacks.”

They obtained the data used in their analysis through a research agreement with the B.C. Ministry of Education.

Search SFU News Online