Experimentally Measuring Effects of Recommender Systems
As social media continues to have a significant influence on public opinion, understanding the impact of the machine learning algorithms that filter and curate content is crucial. However, existing studies have yielded inconsistent results, potentially due to limitations such as reliance on observational methods, use of simulated rather than real users, restriction to specific types of content, or internal access requirements that may create conflicts of interest. To overcome these issues, we conducted a pre-registered controlled experiment on Twitter's algorithm without internal access. The key to our design was to, for a large group of active Twitter users, simultaneously collect (a) the tweets the algorithm shows, and (b) the tweets the user would have seen if they were just shown the latest tweets from people they follow; we then surveyed users about both sets of tweets in a random order.
Our results indicate that the algorithm amplifies emotional content, especially those expressing anger and out-group animosity. Furthermore, reading political tweets from the algorithm leads readers to perceive their political in-group more positively and their political out-group more negatively. Interestingly, while readers say they prefer tweets curated by the algorithm in general, they are *less* likely to prefer algorithm-selected political tweets. Overall, our study provides important insights into the impact of social media ranking algorithms, with implications for shaping public discourse and democratic engagement.
Smitha Milli is a Postdoc at Cornell Tech, working with Nikhil Garg and Emma Pierson. Smitha received a BS and PhD in EECS from UC Berkeley where they were advised by Anca Dragan and Moritz Hardt. Their Postdoc is supported by the Open Philanthropy Project.