Does Algorithmic Monoculture Increase Systemic Exclusion?

By Kenny Peng (Cornell Tech)

Colleges and companies increasingly use algorithmic tools when evaluating applicants—and consequently, may be evaluating applicants in an increasingly homogeneous way. What are the risks of this homogeneity? This basic question is the focus of emerging research on algorithmic monoculture [1-7].

One intuitive concern stands out. Imagine being a high schooler applying to college. The student—already subjected to a complex and seemingly arbitrary process—now has an additional fear: if one college rates her poorly, then all colleges likely will as well. Monoculture, intuitively, has the potential to result in greater systemic exclusion—that is, being denied all opportunities.

As a starting point, this post examines and challenges the basic premise that algorithmic monoculture increases systemic exclusion. From there, I hope to clarify—building on a recent paper by Nikhil Garg and I—what we should expect to change (and what we might worry about) as decision-making becomes more and more homogeneous. In the process, I’ll also suggest how we may sharpen our definition(s) of systemic exclusion.

Monoculture in Equilibrium

Let’s consider the claim that algorithmic monoculture increases the the number of students who are rejected from all colleges. Suppose the total number of seats available in colleges is fixed. Then, the number of students who enroll in a college remains constant, regardless of how colleges evaluate applicants. This implies that the number of students who are rejected from all colleges also remains constant. In other words, the amount of systemic exclusion does not change as a result of algorithmic monoculture.

How does the rate of systemic exclusion remain unchanged? When colleges make more homogeneous evaluations, they are competing over the same group of students. This means that colleges each need to make more offers in order to fill capacity. To really understand the effects of monoculture, we need a way to handle these kind of market effects, in which colleges respond dynamically to fill capacity. Nikhil and I present such a model in our paper Monoculture in Matching Markets. The model provides some additional insights, as I discuss next.

Benefits of Monoculture

One, perhaps unintuitive, result our paper demonstrates is that monoculture actually results in better outcomes for students. More precisely, all students are more likely to end up at their top choice school under monoculture. Under monoculture, a student who receives one offer is likely to receive many offers; this means that students have their pick. Viewed another way, monoculture increases the amount of competition between colleges over students, increasing the welfare of students.

Analogs of this result have been shown in other contexts. For example, in New York City and other school districts, there has been debate over whether or not high schools should admit students all using the same lottery number, or using different lottery numbers. Again, perhaps counterintuitively, it has been demonstrated that using one lottery number across all schools improves the likelihood a student receives her top choice.

Viewed in this way, monoculture seems to be beneficial from the student perspective. In the next section, we complicate the picture.

Meritocracy and Equality of Opportunity?

The above analysis only considers student outcomes overall; it doesn’t, however, consider which students match. Suppose we want, for example, for the most capable students or workers to be matched. This can be justified from a meritocratic or social welfare perspective. Is monoculture worse from this perspective? The answer isn’t so clear. In other recent work, Nikhil and I show that depending on the type of noisiness by which colleges or companies make decisions, independent decision-making (the opposite of monoculture), can either make the outcome more or less noisy—independent decision-making doesn’t necessarily result in selecting higher-quality applicants. 

I think there is also a more fundamental way to understand why monoculture may be problematic. To see this, let’s turn to the problem of hiring. If we’re looking only at how many people receive a job offer, as we showed above, this number remains constant. In fact, monoculture increases the amount of competition, potentially benefiting workers. 

Consider instead the set of people who receive a chance to interview. Here, if the set of applicants who pass the screening stage is more homogeneous across companies, then the total number of applicants who receive an interview is smaller. In other words, algorithmic monoculture may very well increase systemic exclusion at the screening stage. Here, the concern is not that fewer people receive an eventual job, but rather that fewer people have the opportunity to demonstrate their qualifications. 

Kathleen Creel and Deborah Hellman have an excellent article in which they raise the moral concerns raised by systemic exclusion arising from algorithmic monoculture. As they write, “Exclusion from a broad swath of opportunity in an important sector of life is likely to be morally problematic.” The more economics-style analysis presented here can further sharpen this claim: that algorithmic monoculture may increase exclusion from the opportunity to demonstrate one’s capabilities in an important sector or life, and that this is likely to be morally problematic.

A broader takeaway is that the claim that algorithmic monoculture increases systemic exclusion requires a more precise definition of what we mean by systemic exclusion in the first place. Sharpening this definition is important both to understand (1) what type of systemic exclusion is likely to arise under algorithmic monoculture, and (2) what type of systemic exclusion is problematic. Doing so, will allow us to understand when potential interventions, such as randomization, will be most helpful.

Further reading

[1] Bommasani, Rishi, et al. “Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?” Advances in Neural Information Processing Systems, vol. 35, 2022.

[2] Creel, Kathleen, and Deborah Hellman. “The algorithmic leviathan: Arbitrariness, fairness, and opportunity in algorithmic decision-making systems.” Canadian Journal of Philosophy, vol. 52, no. 1, Jan. 2022, pp. 26–43, https://doi.org/10.1017/can.2022.3.

[3] Jain, Shomik, Kathleen Creel, et al. “Scarce Resource Allocations That Rely On Machine Learning Should Be Randomized.” Proc. 41st Intl. Conf. on Machine Learning (ICML), vol. 41, 2024.

[4] Jain, Shomik, Vinith Suriyakumar, et al. “Algorithmic pluralism: A structural approach to equal opportunity.” The 2024 ACM Conference on Fairness, Accountability, and Transparency, 3 June 2024, pp. 197–206, https://doi.org/10.1145/3630106.3658899.

[5] Kleinberg, Jon, and Manish Raghavan. “Algorithmic monoculture and Social Welfare.” Proceedings of the National Academy of Sciences, vol. 118, no. 22, 25 May 2021, https://doi.org/10.1073/pnas.2018340118.

[6] Rémi Castera, Patrick Loiseau, Bary Pradelski. Correlation of Rankings in Matching Markets. 2024. hal-03672270v5f

[6] Peng, Kenny, and Nikhil Garg. “Monoculture in Matching Markets.” Advances in Neural Information Processing Systems (NeurIPS), vol. 38, 2024.

[7] Peng, Kenny, and Nikhil Garg. “Wisdom and Foolishness of Noisy Matching Markets.” Proc. 25th ACM Conference on Economics and Computation, vol. 25, 2024. 

Kenny Peng

PhD Student, Cornell Tech

DLI Doctoral Alum

klp98@cornell.edu

More Info >

Cornell Tech | 2024

Download PDF >