By Kristen Engel
On 13 February, the Digital Life Seminar hosted the University of Haifa’s Professor Tal Zarsky to discuss his research on the implications of ‘Small Change Makes a Big Difference’ (SCMBD) outcomes on decision-making and the determination of how and when to intervene. SCMBD outcomes occur where small changes in inputs produce large changes in outputs, determined by SCMBD testing of the differences between inputs and outputs. The SCMBD phenomenon is often unavoidable and exploitable by those in the know surrounding the gaps and decision points that lead to specific outcomes. In his talk, Professor Zarsky considered the implications of SCMBD outcomes, how to respond to them through testing, and where to take the conversation going forward.
The Implications of Linear Thinking in a Non-Linear World
The rise of automated processes across the public and private sectors is exacerbating concerns about the fairness of decision-making. In both machine and human-driven processes, questions are raised about the presence and mitigation of the lack of statistical parity, inequality in error rates across social groups, as well as biased systems and data. The questions of fairness in the selection of inputs as well as regulatory and algorithmic decision functions that contribute to SCMBD outcomes highlights the larger issue of the effectivity and accuracy of man-made frameworks in modeling the natural world. SCMBD outcomes undermine the philosophical notion of desert, where there is a clear and connection between work and the deserved, earned reward. If someone works twice as hard, we expect them to do twice as well. Humans tend to apply such linear frameworks to decision-making, underscored by this concept of desert, which is a cornerstone of many governing influences from regulations to religion. However, the natural world is not linear. Manmade regimes, such as thresholds and binning, can defy the natural order of things and lead to inaccurate and unfair decisions. For example, the difference between an income of $10,000 and $10,010, with all things holding constant, can cause a greater than 1% difference in resulting outcome.
Responding to SCMBD
Despite this gap, existing linear decision-making processes still have the potential to yield helpful insights. Assessing for SCMBD outcomes, by observing similar inputs to a decision-making process and testing for a disparity of the outputs, is easily conducted and already being implemented in a variety of contexts. Consideration of the individual circumstances of these processes and outcomes is key in determining if the resulting SCMBD situations are symptomatic of the presence of an unfair and possibly illegal fact or proxy indicator or just a reflection of the natural world. Due to the murky connection between outcomes and decision-making processes, any SCMBD testing would need to consider the concerns of the parties on both sides, balancing system explainability with political issues and the protection of trade secrets. Since SCMBD testing does not require ‘breaking open the black box’, it can be tested by institutions, litigants, third parties, or self policing without compromising tradecraft and still return valuable insight such as flagging a problematic process or a flaw in decision-making design.
The Path Forward
The challenge is determining in what situations SCMBD testing should be applied. Awareness of the potential of outcomes where small changes make a big difference is a great first step and prompts the consideration of further questions. One must determine whether or not a SCMBD outcome reflects a problem or a natural occurrence and whether or not to take corrective action. It can alert us to data that need to be diversified and thresholds that need to be redefined. This has implications extending beyond the decision-making process itself, as some SCMBD outcomes will require policy intervention. As technology enables more decisions with less human intervention, consideration must be taken to design systems that can reduce the delta between the modeled and observed world and evaluate outcomes in the context of their inputs and the arbitrary decisions built into the process. Decisions are becoming more hidden, obfuscating the role of flaws in design and arbitrary thresholds in altering outcomes. The identification of SCMBD outcomes needs to be a consideration in the decision-making so that the right questions can be asked to ensure processes result in fair and accurate determinations.
Kristen Engel is a MS Student in Connective Media at Cornell Tech.