By Marianne Aubin Le Quéré (DLI Doctoral Alum, Cornell Tech)
In 2020, Newsday, a local Long Island newspaper, teamed up with the Associate Press to automate their coverage of the area’s school districts. The cooperation was supposed to be a win-win-win: Newsday would expand their coverage and attract larger audiences; Long Island residents would get much needed local information; and the team at Quartz would prove that automated text generation can be scalably deployed to local newsrooms. One year later, they had succeeded—and yet they had failed.
Some of the promises of automation had paid off, as Newsday was now able to produce automated coverage of over 100 school districts each day. This change saved journalists time, and provided the public with information they sought. And yet, although Newsday boasts a relatively large engineering team for a local newspaper, the project took over a year to conclude. The team started with lofty ambitions of creating a scalable infrastructure that could be re-used for school coverage, real estate updates, or to report on local crimes. By the end of the project, though Newsday had created an infrastructure that suited their needs well for school coverage, they now knew extending to other beats would not be a simple task. If Newsday wanted to start automating other coverage, they would have to embark on another multi-month process to create local automated content suitable for publication.
As institutions turn with consternation towards the waning state of local news today, more philanthropic efforts are emerging, geared at integrating artificial intelligence tools to help local news businesses thrive (or, more often, survive). The Knight Foundation, for example, launched its Knight-Lensfest Initiative in 2015, recently kickstarting new efforts with a list of seven projects seeking to incorporate AI to change local news business models. Google and Facebook, who themselves are responsible for large losses to local newspapers, have reinvested in the industry through the Google News Initiative and Facebook Journalism project. Academic attention is also turning to the question of AI in local news, with a JournalismAI team emerging out of the London School of Economics and the Brown Institute at Columbia launching an “AI and Local News” initiative.
There’s no denying that local newspapers need help and investment. Just in the U.S., over 2,500 newspapers have shut down since 2005. The dissolution of local media results in less availability of local news, giving rise to the term “news deserts,” areas where residents have no access to a local newspaper at all. According to latest estimates, over 200 counties in the U.S. no longer have a local news source, while almost half have only one. The local newspapers that remain are also losing revenue, letting go of reporters, and conducting less investigative journalism.
It is perhaps only natural, then, that we’ve turned towards AI to fix some of the problems underlying the local news industry. The question is simply—will AI solve anything?
Newsday’s story is one example where AI technology might perform well for large, well-funded newsrooms, but adapting technology to local newsrooms is not just a matter of copy-and-paste. A new report published by AP about the state of AI in local journalism confirmed that there is a sizeable ‘gap’ in the adoption of AI technology between large and small newsrooms. Local journalists have a number of additional concerns that arise from belonging to oft-underfunded organizations.
In a recent CHI’22 workshop paper with fellow Cornell Tech PhD student Maurice Jakesch, we grouped AI tools that have been deployed within the local news creation process into key categories. Through the stories of individual tools, journalists recounting their experiences, and companies and academics that provide AI solutions for local journalism, we described how adopting AI forces journalists to weigh different aspects.
AI is great at optimizing processes. As Nick Diakopoulos highlights in his 2019 book Automating the News, many large newsrooms have successfully incorporated automation into their news production processes, from the analysis of sources, to monitoring social media for leads, and including automated content generation. Often, the types of AI that are deployed in large newsrooms make tedious processes incrementally faster. AI tools may help a journalist group large sets of documents together to be able to sort through them faster. Anomaly detection methods can be used to find out about an emerging story faster than the competition. And yes, large language models can be used to fully automate templated stories like sports reportage and financial analytics.
However, the problems that local journalism is facing will not be resolved through increasing publishing speed or optimizing processes. For example, the Reuters News Tracer was developed to tap into real-time verified eyewitness testimony on social media, and allows the company to gain an “8- to 60-minute head start” over the competition. But whereas an international newsroom may compete to break a story first, local newsrooms hardly even have competition. AI initiatives that port solutions developed for large news organizations to small newsrooms may find that they have limited impact.
Ultimately, the problems local journalism faces revolve around finances and ongoing sustainability rather than around competition and speeding up reporting. And when it comes to business models, AI is not such an obvious solution.
There are some efforts to use AI to increase local newspaper subscriptions, for example through the use of “smart paywalls.” These systems try to algorithmically predict when a user will be most likely to subscribe to a newspaper, to implement paywalls in moments with the highest chance of subscriber gain. Recently, the Brown Institute at Columbia University has partnered with 11 local news organizations, to test their hypotheses that these smart paywalls could be part of a solution for local news sustainability. This is one example where an AI-driven solution that focuses on the problems of local newsrooms may lead to an impactful outcome.
However, one underexplored area in the AI and local news space is to think about the ways that AI may be deployed to re-orient resources. For example, we know that Google News and Facebook feed rankings play an important role in directing web traffic to local news organizations. Other tech and large media organizations have not considered the potential benefits of sending their users to local news sites. For example, the New York Times might consider leveraging AI to personalize external local news recommendations for people visiting their sites. This type of arrangement could not only benefit local news, but may also provide a good user experience for visitors of the New York Times when their coverage depth runs out in a specific locality. More tech-driven local websites, such as NextDoor (a neighborhood social media platform), may also help get vital information to their users by identifying and promoting high-quality local journalism in their newsfeeds.
AI and automation may also present an opportunity to fundamentally alter local news business models. Local news entities help people complete many purchases, such as deciding which films to see, concerts to go to, or businesses to visit. If local news outlets were able to get a financial kickback for recommending the right local business to attend, there would be an incentive to invest in optimizing local recommendations. These business entities may then also find ways to direct user attention back to the local news outlets; these processes can become mutually beneficial and self-sustaining. In other words, it’s not sufficient to change and optimize the newspapers themselves, but we must think about building an entire media-and-internet ecosystem that helps drive more money to local newsrooms, and helps those newsrooms feed money back into the local economy.
In the end, it is not sufficient to deploy AI tools that were developed for large journalistic institutions and hope they might help in a local and financially scarce context they were never designed for. Let’s rethink how AI can help local journalism – and if more technology is the right avenue to pursue – from the ground up.
Today, most of the AI efforts deployed to local newsrooms have been incremental, not transformative. It’s time to change that.
Marianne Aubin Le Quéré
DLI Doctoral Alum
Cornell Tech
msa258@cornell.edu
Cornell Tech | 2022