Garbage in Garbage out: How bad data will cripple the future of news

(Note I haven’t been doing much blogging for the past several months. I took early retirement from CBC News and moved back to my old hometown of  Kitimat, British Columbia, a process that took much longer than I anticipated and is still ongoing as I wait for electricians to finish some electrical upgrades on my new house. I am now resuming my quest to find hints on the long term future of news and so the blog and related projects will slowly appear here.)

The old adage from the earliest days of computing, Garbage In Garbage Out still holds.

The beleaguered news industry is obsessed with metrics, too obsessed in my view. That obsession also seems to be based on the idea that the data being gathered is good data, not junk. 

Yet this week, up popped on my iPad a sad example of what is wrong with the efforts to save journalism, sad because it comes from one of the United States’ most respected journalism schools. It is a survey, a survey that shows just how out of touch with reality some studying the future of news are, a survey that is so seriously flawed that when I was teaching journalism at Ryerson University in Toronto, in the 1990s, I could have used as an example to teach students what to avoid.

When I lived in big cities, and being part of the generation raised on print, I would devour the morning paper along with my breakfast, mostly the Globe and Mail in Toronto, but  in the  various other cities I have lived, also the Ottawa Citizen, the Times and the Guardian in London, the South China Morning Post in Hong Kong, the Vancouver Sun and The Province.

Now, in Kitimat, British Columbia,  there is no newsprint waiting on my doorstep at dawn.

The iPad (as opposed to a netbook computer) is the next best thing.  I can prop the tablet up on the breakfast table and still get my morning news fix (I mean update) without having to go to the computer in my home office.  I check the Globe and Mail, AP, the BBC,  New York Times and the Daily Telegraph. I enjoy the Guardian’s Eyewitness best of the day  photo gallery. (And I would actually consider paying for a Guardian iPad app, but for some mysterious reason, it  is  only available for the iPhone and I’m an Android user.)

So there I was Saturday morning, scanning the Associated Press app, when there appeared at the bottom of the screen, a very enticing ad.

210-ipadfuture1.jpg.

Since I am interested in shaping the future of news, I tapped.

The first page was both a further enticement and the usual academic disclaimer needed when surveying  human subjects.  The survey was from the University of Missouri School of Journalism, one of the best institutions south of the border,  so I was quite optimistic.

There were warning signs. The disclaimer added a further enticement for ongoing participants, but only to Americans,  saying that to win the goodies, an  iTunes gift card, you had to be United States resident over 18.

212-surveydisclaimer-thumb-450x487-211.jpg

I am interested in shaping the future of news. So I tapped.  The first screen came up. My heart sank.

The first question asked for the subject’s five digit US zip code.

214-zipcode-thumb-450x98-213.jpg

That meant immediately that everyone outside the United States didn’t count. I filled in the field to let me see if  I could continue.  I could, but now the survey has no idea where I’m from.

So much for shaping the future of news.

Here’s the first problem, just the day before the ad popped onto my iPad, on Friday, October 22, the Wall Street Journal reported.

Morgan Stanley estimates that about 13 million Apple tablets will be sold this year, out of 15 million total tablet sales world-wide.
For 2011, Morgan Stanley estimates that Apple’s number will rise to 30 million, while non-Apple tablets will skyrocket to 20 million.

So assuming the figures are correct (and if you check these other links, the Morgan Stanley figures appear to be in the right ball park), 15 million people around the world use tablets at this moment.

iPad Impact: Tablets contribute to PC market pain

iPad/iPhone shipments drive up Apples Q4 profits

Tablet Sales to Hit 19.5M in 2010,

 Associated Press, even though it is based in the United States, is a world-wide news organization with staff, stringers and affiliates in every spot on this planet. Even if a lot of those 15 million users don’t read the Associated Press app, you must assume  that the majority of the people get their news from their tablet and many use multiple news sources.

So how can a genuine survey aimed at shaping the future of news exclude the majority of  iPad users?

When I worked for CBC.ca, (which unfortunately doesn’t yet have a news app) our audience figures showed consistently over the years that 20 per cent of our  web audience came from the United States and another 10 per cent of the audience came from the rest of the world.  The BBC site has a huge world-wide audience, and I can only presume that is reflected on the audience figures for the BBC app on the iPad.

When Guardian editor Alan Rusbridger spoke about the future of news in the Hugh Cudlipp lecture earlier this year, he said:

More Americans are now reading the Guardian than read the Los Angeles Times. This readership has found us, rather than the other way round.

So as a Canadian, I sighed, it looked like just yet another case of American exceptionalism and parochialism.

Unfortunately, it got worse.

The next question asked the birth year of the participant. But the drop down menu stopped at 1994, excluding everyone younger.

217-agesurvey-thumb-450x641-216.jpg

Again one has to ask what is going on here?   We know from multiple studies that news reading habits are formed at an early age.  Yet the university survey is excluding everyone under the age of  16. If a lot of  12-year-olds are suddenly getting excited over the journalism they see on their iPad, good news for shaping the future of news, you would never know it from this survey. (I hope a lot of 12-year-olds are getting excited about news on their iPads)

So why the exclusion?  I wondered for a moment if it was the US law  where for many internet activities magic age is 13. So if so, the survey is still excluding three years of users. Or one must ask is it something to do with the giveaway of the iTunes gift card, rules set by who. Apple?  A university ethics committee? The state of Missouri? The state of California? Then why not 18​?

Are these giveaway rules skewing and distorting the survey?

One more note, about the United States resident restriction, especially if it is tied to the giveaway. Again it conflicts with what we do know about web journalism and probably tablet journalism.   That people who move away keep in touch with local news by checking hometown web sites and often that it  is a huge part of a news site’s web audience (even if the corporate side is reluctant to point it out to local advertisers). So once again this survey is excluding all those American ex-pats around the world who may be reading the AP and New York Times apps. And what about all those members of the American military deployed overseas,  most of whom technically are not on US soil, checking their local news on tablets? Do they count?

220-giveaway-thumb-450x349-219.jpg

Finally, while the questions were fairly routine, asking about how I consume news on paper, on my smart phone and in my iPad, I detected a bias (or perhaps over simplification) in the questions.  It may be an urban bias, despite the fact that the University of  Missouri in is in the heartland of a largely agricultural state.  (I was at Missouri for a beautiful August week in 1993 when I attended IRE computer-assisted reporting boot camp).  There was no way I could tell the survey that journalism on paper is not as available in the far Pacific northwest as it is in downtown Toronto, New York or St. Louis. It assumes an almost “either or attitude,”  that the one reason you give up a newspaper on paper is to switch to electronic delivery because you just love your iPad.   Yet there are other reasons, for giving up reading papers on paper. A lot  of the more environmentally active refuse to read paper (even though the electronic versions are probably just as bad for the environment).  As mentioned, for many people today, your home town paper isn’t delivered in Kandahar or Kuala Lumpur or Kingston-upon-Thames.

There is also a second factor, it is much easier to read a newspaper on the way to work in a city with a good public transportation system, where you can read the paper on the bus, subway or train. So how much has commuting habits to do with consuming news on a smart phone or tablet?

It’s unclear if AP has anything to do with the survey (i.e. Was the ad on the AP app paid for by the university or is  it a “house ad,” with AP participating and working with the data? Perhaps someone can clarify in comments)

But this is clear, with the crisis in journalism, this type of survey is not a help, it’s a hindrance.  If it’s American parochialism,  same old same old  will continue to fail with a world-wide audience and increasing world-wide tablet sales. If it’s the iTunes giveaway setting the survey parameters, it’s a bad as any survey that  is paid for by a corporate sponsor with a pre-determined outcome.  If it’s because of a lack of budget due to current restraints that make it impossible to crunch numbers from 15 million tablet owners, then the survey should have been done like someone like Pew who can look at the planetary picture.

Garbage in, Garbage out has no place in shaping the future of news.

NOTE and UPDATE
:  I can get Canadian news, via The Canadian Press (an organization I occasionally string for) via the AP app on my Android smart phone.   But the Local News function on the AP Ipad doesn’t work, My Ipad insists that I live in Manhattan, not Kitimat.

Two days after I made the initial post, on Oct. 27, 2010, I sent a note via my Ipad to AP tech support asking why can I get Canadian news on the Android and not on the iPad? I got a reply  “A Customer Support representative will respond within the next 24 hours.”
It is now Nov 14 and I still haven’t had a reply,

Enhanced by Zemanta