Twitter’s new data access rules will make social media research harder

Twitter’s headquarters in San Francisco. The company’s latest change will make it harder to researchers to study the platform.

David Odisho/Getty Images

hide caption

toggle caption

David Odisho/Getty Images

Twitter’s headquarters in San Francisco. The company’s latest change will make it harder to researchers to study the platform.

David Odisho/Getty Images

Much of what we know about social media discourse is thanks to Twitter’s longtime policy of allowing free access to its data. That has made Twitter data a treasure trove for researchers eager to study online behavior, including how falsehoods and conspiracy theories circulate. Kate Starbird remembers how Twitter research dominated the field between 2010 and 2015.

“You look at some of the conferences we attended, you know, 50% of the social computing papers would be about Twitter and sometimes even more, because that was the data that we had access to,” says Starbird, a researcher at the University of Washington who studies online information dynamics during crises, including disinformation.

But in the latest change to the social media service since billionaire Elon Musk bought the company last year, Twitter announced Thursday that it would start charging users at least $100 a month for using its data pipeline starting Feb. 13, with one exception – users that tweet less than 1,500 times a month, an average of twice an hour or less.

Twitter did not say how many tweets users can download or post at the $100 a month level. Those who need additional access will have to have to pay more though the company didn’t disclose the pricing.

The move will make it more expensive to run many automated accounts, known as bots. Some bots promote scams and propaganda, while others are useful or fun for many users, such as those that highlight every change the New York Times makes to its story headlines or flag an earthquake.

Musk has long expressed his desire to rid the platform of “bot armies.” When Twitter first announced last Thursday that it will start charging for API usage without information of pricing or exceptions, bot watchers on the platform bemoaned the imminent demise of creations they loved. On Saturday, Musk tweeted that “responding to feedback,” bots “providing good content” will keep free access.

If some bots were spared, no researchers were. The change will also limit what is possible for researchers such as Starbird who have relied on that pipeline, known as an application programming interface or API, to study user behavior and information operations on the platform for years.

Earlier this week, after Twitter first announced that it will start charging for the API, a group of research institutions, advocacy groups and individual researchers from around the world issued an open letter calling on Twitter to maintain <<API>> access for researchers so that public-interest research could continue. In a statement, U.S. Rep. Lori Trahan (D-Mass.) said that Twitter should be making data access easier, not harder. As of Wednesday, Twitter did not respond to a request from NPR sent last week for more information about its decision.

Despite its recent upheaval, researchers still need Twitter

While researchers have used Twitter data for all kinds of research, Twitter’s platform design gives it a unique role for researchers focusing on information operations and conspiracy theories. Content can go viral in a way that’s impossible on other platforms, and provides metrics like retweets and likes that make tracking the impact and spread easy to understand.

Users’ timelines are shaped not only by who they follow but also algorithmic recommendation, so players seeking influence can game it to amplify its message.

While the social media landscape has splintered in the past few years, Twitter still serves as a guide post because narratives brewing in smaller platforms could bubble up on Twitter.

“Twitter was a way to kind of at least see, like, what you know, where is that iceberg in the water and what’s going on there,” says Starbird.

Since taking over Twitter late last year, Musk has reinstated the accounts of thousands of conspiracy theorists and white nationalists while cutting the company’s teams that monitored the platform for hate speech and enforced its rules.

Big picture research could be hampered

Users with access to the Twitter API can upload and download data in bulk to and from the platform using a computer program, bypassing the main user interface.

Currently, many Twitter API users can download up to two million tweets from the past seven days for free every month. Academic institutions can download unlimited amounts from the entire archive for free. With large datasets, researchers can make intricate maps of how clusters of users relate to each other, which is invaluable for understanding online communities, including those that spread falsehoods.

Without access to that wealth of data, researchers will have a less comprehensive picture and less ability to go back and investigate narratives that they’ve missed in real time, Starbird says.

By giving users well-documented API access, Twitter’s data has been more transparent than other major social media platforms. Meta’s offering, CrowdTangle, does not provide straightforward ways to download data in real time and in bulk the way Twitter does. Moreover, the company is reportedly winding it down and has not announced whether it will offer a replacement. Meta did not answer questions from NPR about CrowdTangle’s future.

TikTok announced last year that it’s testing a research API, and is “planning to expand availability in the US in the coming weeks.” the company told NPR in an email. The company has come under criticism in the past year for allowing disinformation to spread on its platform. It has also faced bipartisan scrutiny due to its Chinese ownership.

Starbird’s team is throwing ideas around what they can do with Twitter if their current level of access ceases. They intend to focus on Telegram, TikTok and Reddit along with Twitter for the 2024 presidential election while collaborating with teams that monitor other platforms.

“We’ve tended to work within the constraints we’ve had for so long.” Starbird says, and maybe there will be new creative ways to use Twitter data. “Unfortunately, I think a lot of that creativity is going to be better spent on other platforms.”