The response to my earlier post about the Project for Excellence in Journalism's study How News Happens makes me think I should clarify a few things. But first, if you haven't already, please read Steve Buttry's critique.

I said earlier that it was a waste of time to criticize the study's scope when the thinking behind it was so sloppy. I still believe it's unnecessary, but it's not a waste of time; the omission Buttry found reveals how high journalism prejudices undermined the study's validity.

Mark Kawar said my criticism of the study's first question ("Where does the news come from in today’s changing media?") was semantic. Technically, since I am talking about meaning, it is, but I don't accept that as pejorative.

Kawar asked if I would have preferred, "In what medium do widespread, fact-based storylines begin?"

Yes.

The rephrased question is superior to the PEJ's because it dares to offer a definition of news. But it sounds a little silly, right? It makes me think "news" isn't what I want to look at. It also makes me look askance at the word "medium," when "media" kind of slipped by. It also reveals the question as a bit of a tautology. We can predict the conclusion: newspapers. Everybody knows that!

Here's where I say something about stuff everybody knows: They don't really know it. And I don't object to research that does nothing but support or contradict what is assumed to be well-known. Now, I did write that "every beat reporter in the known universe" already knew what the study had concluded, and I did quote a very short extract of Jeff Jarvis' reaction to the PEJ. Let's just say it's because I felt the boulder come rolling back down the hill. I mean, great, now we have a club to wave in the face of the next raving amateur blogger who claims he's outdoing us. I could use one of those. Or a study to plug into a literature review. That's useful, sure.

But I also fear it's fuel for another few months of smug smiling while we wait for print circulation numbers to pick up. If we build it, they will come! Just wait until the economy turns around!

Kawar also wrote that it sounded like Buttry and I wished the PEJ had tried to answer a different question. (Note: These were Tweets, so Kawar probably has more to say or might have said it differently. It's just all I have to go on right now.)

If I had my druthers, yes. But there are two questions in the PEJ's study, with one of them rephrased several times. So, which one?

I'm interested in what a post-newspaper information market would look like, but the PEJ didn't get us there. I'm a little leery of the question, "Who really reports the news that most people get about their communities?"* It sounds moralistic, and I'm more interested in how people get information, share it, use it, etc. I think people who run newspapers should be, as well.

But I would have been happy with any question, so long as it was clear and well-defined, rigorously modeled, and properly related to the conclusion. How News Happens is none of those.

*Why are all these questions so prolix? There's nothing wrong with "Who reports the news?" The rest doesn't add gravitas; it's just weight. Pace Steve Buttry, but if the PEJ had an editor, how did "The answers are a moving target; even trying to figure out how to answer them is a challenge." survive? Figuring out how to answer questions is the challenge.

A fundamental flaw of the Project for Excellence in Journalism’s study of the “news ecosystem” in Baltimore is revealed in its first sentence:

Where does the news come from in today’s changing media?

To a perfectly reasonable question the study’s authors have appended the misguided notions that information is made by news organizations, and that understanding what’s done with it requires a look at new and old media in competition, rather than at the way everybody else uses and shares that information.

Worse: There are a mess of questions in the study’s first few paragraphs, but its ultimate aim seems to have been to explore what would happen to information if newspapers were to die, and its conclusion was, simply put, that most news stories originate with newspapers. So, aside from being a non-sequitur – a really bad sign, by the way – the study’s conclusion confirms what every newspaper beat reporter in the known universe already knows. In the words of one pundit with whom I never would have expected to agree so completely, no shit.

It is a waste of time to criticize the study’s scope (Six stories tracked over one week!) when it is structured so poorly. The theoretical links between the study’s questions and its design are not explained; a theoretical justification is not so much as suggested. The conclusion does nothing more than provide a sliver of information that might be used to justify the design of the study itself. An analogy:

Question: How would people get around without motor transportation?

Answer: Our study finds that a lot of people drive cars.

If the study’s aim was solely to describe the current “ecosystem,” then what are readers in this ecosystem, space aliens?

PEJ: “…most of what the public learns is still overwhelmingly driven by traditional media.”

Learns. Learns! The study did nothing to describe how people use or share information on their own, yet somehow has explained both of those things. “Driven,” in this context, is a weasel word that protects the claim from being wrong, and simultaneously ensures that it is completely without insight.

David Carr revealed, perhaps a little too cleverly, the study’s absurdity in a meta-post at the New York Times blog Media Decoder.

And in the protocol of blogging, rather than calling others who can speak to the efficacy of the information — in this case, where the “news” in 53 new outlets in Baltimore comes from — the writer surfs his way to the stories and opinions of others, and then links and refers to those. If you're feeling really fancy, you can always drill into Twitter as well.

The activity has its merits, but truly kicking the can down the road and advancing the story is not generally one of them. Instead, we depend on the source material for insight, sometimes treating it as our own — the technical, legal term for that is stealing — or sometimes excerpting[…]

The back-slapping congratulations with which the study has been received by some newspaper folk shouldn’t baffle me, but it kind of does because the conclusion seals their irrelevance; if newspapers do the best job of collecting, synthesizing and publishing information, and, as we know, readership declines anyway, then…what? This should horrify, not comfort.

They should be wondering what it means that people are reading their stories and reading about their stories, but not reading the newspaper. Hint: It doesn’t mean you should be doing your best to protect a product fewer and fewer people want, at the expense of a product they do.

A better study wouldn’t have assumed that media were the best subjects for describing how information is discovered and shared, particularly if the goal is to imagine what might happen without them. Newspapers and blogs and networks don’t just compete with each other in a vacuum; they compete with each other and against many other stimuli in the landscape of people’s lives. They compete for time in a person’s life. David Carr called this the pursuit for “mindshare” in the post excerpted above.

To really explore what the future might bring, we should look for recent changes that haven’t yet rippled across our lives. The newspaper was the king of an age when city council meetings weren’t archived online, when court records couldn’t be downloaded at home, when police agencies didn’t report incidents online in real-time, and when the expense of publication was not zero. A newspaper didn’t just publish information; it provided the labor to collect that information. What will it mean when the cost of that labor approaches zero? What will it mean, for example, when the police blotter, one of the most popular offerings of local newspapers everywhere, must compete with near real-time updates on a police agency’s Web site?

It might mean professional journalism will have to do things differently, but it won’t mean people will no longer be informed. Or was I really supposed to be persuaded by the PEJ’s research that without newspapers people will know nothing?