The new news business

For most of the 20th century, any list of America's wealthiest families would include quite a few publishers generally considered to be in the "news business": the Hearsts, the Pulitzers, the Sulzbergers, the Grahams, the Chandlers, the Coxes, the Knights, the Ridders, the Luces, the Bancrofts -- a tribute to the fabulous business model that once delivered the country its news. While many of those families remain wealthy today, their historic core businesses are in steep decline (or worse), and their position at the top of the wealth builders has long since been eclipsed by people with other names: Gates, Page and Brin and Schmidt, Zuckerberg, Bezos, Case, and Jobs -- builders of digital platforms that, while not specifically targeted at the "news business," have nonetheless severely disrupted it.

On the national level the owners of the big legacy news businesses have fought fiercely against the disrupters, often with the effect of a frustrated ocean swimmer flailing against a fierce rip current. But with each digital click upward in Moore's law (processing power) and Metcalfe's law (network power) the tide of technological disruption has only risen, washing many of the legacy swimmers further out to sea, or at least diminishing their financial prowess. Perhaps it is another law, Amara's, that best describes the results. That law states, "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run."

To document the transformation of the news business, we created an oral history project, from which this article is excerpted. We gathered the personal recollections of a broad but select group of principals who faced the choices, made the decisions, placed the bets, and now have the benefit of hindsight as to how it could, or couldn't, have played out differently. The full project, curated at Harvard by the Joan Shorenstein Center on the Press, Politics, and Public Policy in conjunction with Nieman Journalism Lab, will be published at the website digitalriptide.org on Sept. 9.

* * * * * *

In the early 1990s, David Graves, an executive at Reuters in New York City, spearheaded an investment by Reuters in Yahoo (YHOO, Fortune 500). Reuters, based in England, was a wire service whose main competitors were financial news companies like Dow Jones and Bloomberg. It had relatively few clients among U.S. newspapers, many of which were members of the Associated Press. Reuters started putting its news on Yahoo, and did so in part for an entirely novel reason -- to help protect its investment in the fledgling technology company even as Reuters (TRI) hoped to build significant new licensing revenue streams.

David Graves, former Reuters and Yahoo executive:

It would've been '93. We had just reached agreement to put a couple of million dollars into a little company called Yahoo, pre-IPO. As a concession for us doing this really risky investment, they agreed that Reuters would have exclusive rights to provide content to Yahoo for five years, in terms of what content we had. If Yahoo wanted a news product or a financial news product, they had to come first to Reuters and say, "Do you have this?" Then, if we didn't, they could go buy it from someone else.

Of course, every time they came and asked for something, if we didn't have it, we figured out how to make it. "Oh yeah, we have that." We created things that we didn't have before. U.S. sports and a lot of other things like that.

There was never any question of charging users for content, because Yahoo wanted to increase its audience as quickly as possible.

Graves:

For the first part of the Internet, your stock valuation was driven by your audience size, because nobody was making any rational decisions about what things were worth. So for some number of years all you cared about was how many millions of people were clicking on Yahoo in a given month. Fifty thousand a day kept getting added ... Certainly any kind of pay situation would've interfered with that thing.

Reuters did well with its Internet investment precisely because it was not competing for American consumers in its main business. Because of that lack of conflict, the company was able to pursue free content online without threatening its protected revenue stream from its financial news. CNN was in a similar position: Its main business was fueled by cable fees, which allowed it to find a new audience online without cannibalizing itself. Newspaper-subscription revenues were disrupted by news organizations that had no stake in the consumer paying for news.

* * * * * *

The very core of the revolution was technological, with every new advance being driven by innovations in computer engineering, something most publishing companies never bought into. They viewed engineering as a means to an end -- plumbing -- a way to enable their core journalism to travel around the world through this magical new distribution channel called the World Wide Web.

Eric Schmidt, executive chairman, Google:

I think there's a very simple explanation. The industry has no engineers ... You cannot innovate and build new products without engineers in your field. If you don't have them, you have to find somebody who does and partner with them in a clever way.

Will Hearst, chairman, Hearst Corp.:

I agree with Eric on that ... You needed to celebrate engineering as a creative craft, as a journalistic, creative profession, like photography, like illustration, like editing. These skills don't grow on trees.

* * * * * *

From the resurgence in Internet advertising, beginning in 2003, to the Great Recession that began at the end of 2008, virtually all the swimmers in the news flow had a swift current at their backs. But the global financial meltdown brought a fierce reversal in the riptide.

Premium publishers attempted to hold on to high rates for their web advertising inventory, but it was a mostly futile effort. By 2012 some of them began creating their own (so-called native) advertising solutions, moving away from the banner ads that had defined Internet advertising since the mid-'90s. Often these publishers were criticized for violating what had been a sacrosanct church (newsroom)/state (commercial) divide. Jonah Peretti explains and defends this movement at Buzzfeed.

Jonah Peretti, founder, Buzzfeed:

I don't think banner ads are a solid enough basis for [the business model]. And just like this shift [toward social distribution of content] has happened on the editorial side, the same shift is happening a little bit later in advertising -- where the leverage you get from massive networks can mean the same piece of creative and the same size media buy can get much more distribution.

The church/state separation is very important ... The thing I don't like about the church/state division, as someone who sits above the divide, is that it can lead to a two-tiered system where the journalists are seen as the whole purpose and the greatness of everything, and that the people in advertising are seen as a necessary evil ...

In general, print newsrooms didn't innovate nearly fast enough. Television outlets fared better, at least economically, continuing to garner the largest share of total marketing expenditures from advertisers while holding on to more brand advertising dollars. Moving ahead, however, it will certainly take significant innovation to alter the strong flow of digital advertising away from print and, as audiences continue to migrate online, eventually away from television as well.

Dick Costolo, CEO, Twitter:

My perspective is that we're building this global town square. What I mean by that is, if you went back to ancient Greece, the way that news and information was passed around was, you went to the agora after lunch in the town square. There was this unfiltered, multidirectional exchange of information. I might go into the agora and say to Martin, "Hey, my aunt died." Martin might say, "Euripides' goat passed away." We would exchange some information. By the way, the politicians were there. The musicians were there, etc. There was this multidirectional, unfiltered exchange of information, which was interesting in all sorts of ways.

* * * * * *

When we began constructing this oral history we raised some big questions about "what really happened to the news business," and over the course of nearly 70 interviews, our subjects shed light on all of them -- save one. And that is the big one, really: What is going to happen next to the news business, and how is that likely to affect the quality of life in a democratic republic?

Tim Berners-Lee, inventor, World Wide Web:

There's a need for journalism. People are desperate for it. People are fed up with spam. They're fed up with just searching, using a web search tool to find a medical article, then realizing only after they have gone to the bottom of the article, and followed the advice, and bought the drugs that the whole thing was produced by the same pharmaceutical company, with an extremely slanted view ... People are fed up with that, and journalists have got the skills and the motivation. It's their job to solve that problem ...

One of the solutions may be to get payment protocols on the web -- new payment protocols -- so it's easy for me, as I read your blog or as I read your journal, the output of your journalism, I might be able to tell my browser, "You know what? Whenever I really enjoy an article, I'm going to hit this button, and I want to pay the guy who wrote it, and I want to pay the guy who pointed me at it," because I really appreciate that.

Maybe that's right. But the transition from what's left of the old legacy news business to whatever comes next is likely to be the swiftest undertow yet of the digital riptide. We're in a period of fierce creative destruction where we are both awash in more news than ever -- with glimmers of new business models emerging, such as paywalls on some news sites starting to yield significant revenue streams -- and feeling the ebb of independent, verifiable, and sustainable reporting. The next full moon, the next hurricane, the next digital media breakthrough seems bound to take down a lot of familiar swimmers before it all works out somehow.

About the authors: John Huey, a writer, editor, and publishing executive, recently retired as editor-in-chief of Time Inc. Martin Nisenholtz managed digital operations for the New York Times Co. for 17 years and is now a senior adviser for the company. Paul Sagan is executive vice chairman of Akamai Technologies. They collaborated on this project as fellows at the Joan Shorenstein Center at the Harvard Kennedy School during the 2013 spring term.

This story is from the July 22, 2013 issue of Fortune.