NewsWatch Canada

Characteristics of the Methods

[Click here to return to the main Methodology page]

As with any empirical research project, our methodological choices and procedures influence the findings that we are able to report. In the spirit of "truth in advertising," we freely acknowledge that the process of producing the list of Top 10 stories has distinct characteristics and limitations. The following are the major methodological challenges we have faced to date:

The process of gathering nominations for under-reported stories is undoubtedly still influenced by our geographical location in Vancouver (story nominations are collected and distributed through the SFU office) and by our cultural and linguistic location within the English language. While we invite entries from French-language media, we would very much welcome a parallel project based in Quebec's distinct cultural milieu. On the other hand, the public attention the project has attracted since 1993 has helped widen the range of nominations we receive.

The characteristics of nominated stories also continue to be influenced by our extensive use of 'alternative' periodicals. Their broadly 'progressive' orientation, and especially their lack of resources for long-term, ground-breaking investigative journalism, influence the kinds of stories these journals offer. Their use as a particularly rich source of stories is justified, however, by their explicit attempt to cover events and perspectives marginalized by the dominant media. In future, we will attempt to balance this with a closer examination of publications identified with the 'right-wing,' such as The Fraser Forum and Alberta Report.

Our research is not designed to investigate the full range of possible ways in which people perceive that the news may be 'biased' - from slanted language and selective use of sources, to the overall ways in which issues may be 'framed'. Rather, our research focuses very strongly on significant omissions or blindspots in the news agenda. However, while it is not the project's primary purpose, its findings may well be relevant to debates about news bias and objectivity. Such debates, after all, often concern what the news didn't cover.

The project does not itself have the mandate or resources to undertake extensive, original investigative journalism; instead, it functions as a conduit to publicize stories which have been published or broadcast at least once, somewhere. In theory, this means that a story so effectively 'censored' that it has never seen the light of day would escape the project's attention as well. In practice, we feel that in a relatively democratic society, information which is significant and valid but also discordant and discomfiting is far more likely to be ghettoized to the back pages or the marginal programs, rather than suppressed altogether.

Our research on coverage of 1994 stories was considerably more thorough than in the 1993 version of PCC. Still, as previously noted, there are real limitations of cost and availability, particularly of full-text press reports and of broadcast news. CD-ROM disks of The Globe & Mail and Maclean's magazine are much less costly than on-line services, but a time-lag of many months makes their use awkward. Notwithstanding these limitations, our research strategy should have revealed stories which were well-covered in the major print media (which arguably help set the news agenda for most of the broadcast media as well as the other print news outlets).

In the Project's evaluation of nominated stories, as in the world of journalism itself, subjective judgements inevitably come into play on several dimensions - in defining what constitutes a distinct "story", rather than a set of related stories, a trend, an issue, a rumour, or an opinion; in assessing the significance and credibility of stories, and the reliability of sources; in deciding what counts as a national rather than regional story. (It is interesting that Globe & Mail editors have also had ongoing and unresolved debates on the national/regional distinction, according to David Hayes in his book Power and Influence, p. 178) Such judgements may reflect collective as well as individual differences; there were apparent differences, for example, between the Windsor and SFU seminars in deciding how much press attention a story could receive and still be considered under-covered.

Since the start of the project in 1993, and with the help of SSHRC funding since 1994, we have taken a number of steps to make the research as clear, reliable and replicable as possible. In the seminar discussions, the conscious use of the seven criteria listed above, the collective discussion of each story, and the conscientious attempts to counteract personal biases were all efforts to ensure that judgements were broadly reflective of shared and explicit criteria rather than simply individual idiosyncrasies. The same is true with the effort (invariably flawed, as any Prime Minister trying to select a federal Cabinet can attest) to ensure that our national judging panel is broadly representative. (Interestingly, two judges at opposite ends of the political spectrum each felt that the panel was weighted in favour of the other side!) During 1994, research assistant James Compton prepared detailed and explicit guidelines to the criteria and procedures.

During 1995 we are taking further methodological steps forward. Robert Hackett (principal investigator of the SSHRC grant), Katherine Manson and Michael Karlberg are conducting a survey of approximately 300 associations and advocacy groups of various types across Canada. These organizations are being asked for their perceptions of the news media's performance, and of blindspots in the news agenda. Their collective replies will be a useful benchmark for our analysis of what's missing from the news. The same will be true of planned interviews with journalists, and a review of claims in scholarly and popular literature concerning biases and omissions in the news.

Our range of nominated stories will be further expanded by scanning a broader range of periodicals, including the business press, as well as recently published Canadian books on social and political issues. Next year's student seminarians will be asked to explore not only individual under-covered stories, but also more general hypotheses about omissions or double-standards in the news agenda.

In this ambitious expansion, a complementary division of labour has been devised between the Windsor and Simon Fraser seminars. For instance, the Windsor students will create a "junk food news" list of stories whose news coverage has vastly exceeded their importance, while the SFU students will concentrate on the under-covered stories. All told, we are looking forward to a busy and productive year.

[Click here to return to the main Methodology page]

NewsWatch homepage