Individual reflections by Paul-André Bisson and Sanjay Thallam (scroll below).
By Paul-André Bisson
Cornell Tech
During his talk on Thursday, February 15th, Professor Ryan Calo (University of Washington School of Law) spoke about the topic socio-digital vulnerability that he worked on in recent years. More precisely he talked about how technological designs matter and impact our behavior, the three ways they interfere with us, and how law can be applied in this context.
Calo started the talk with an impactful example of a design’s influence on human behavior: the January 6th Attack on the U.S. Capitol. Despite the chaotic nature of this event, the people were still walking between the stanchions while inside the Capitol. Needless to say, this small example was sufficient as an opener to make us quickly realize just how much design influences our behavior. He then explained how similar designs can as easily be used in technology to influence our actions and decisions. He also clarified the concept of socio-digital vulnerability, which is the susceptibility of individuals and groups within mediated environments to decisional, social, or constitutive interference, rendering them vulnerable.
The three fates
The three ways technology and its designs interfere with us and make us vulnerable, also known as the three fates, were explored extensively during Calo’s talk. The first interference induced by technology is decisional interference: our ability in decision-making and autonomy. Calo talked about how different companies no longer need to explore cognitive biases and have sufficient data to figure out how someone is irrational. It is now possible to create persuasion profiles to figure out how someone can be irrational, and vulnerable, and exploit it. They can know how much someone specifically would pay for an ice cream.
The second interference explored during Calo’s talk is social interference: our social and emotional state. Calo mentions that many people have a powerful response to anthropomorphic elements. An example of this is Replika. This application creates an AI designed to build relationships. As it was observed with the users of this application, most of them become socially attached to it. However, it would lead its users to develop romantic feelings towards the AI, pushing them to buy their premium plan for romantic options. Eventually, the Garante created a case against them. This example alone shows how technology can have a very strong impact on people’s emotional state and use it against them.
The final interference explored is constitutive interference: our ability to self-discover and self-actualise. Although this topic was not as explored as the previous two during Calo’s talk, he has made it clear that our ability to self-discover is more limited now than ever.
With so much interference, filtering, and management about what and who someone encounters, Calo mentions it becomes hard to live an “authentic” life. Although multiple affordances are brought by technology, they also come at the cost of our ability to self-discover.
Where is the law in all this?
During his talk, Calo made it clear that technology can interfere with multiple aspects of what defines us as a person. However, how can we leverage the law in this context? From everything that was mentioned during the talk, the concept of vulnerability is the first that should be revisited. In many places, the concept is binary and status-based. However, vulnerability can also be circumstantial. The point made by Calo is not that everyone is vulnerable, but that the nuances are not explored enough in law to have an accurate representation of what is happening. Furthermore, Calo said during his talk that there should be a distinction between power rules and harm rules. As of right now, most rules address, prevent, and alleviate harm. However, it is also possible to have rules affect power. In other words, add or remove power from different groups of people. Regulating firms and companies that currently have access to a lot of information for many individuals, giving them a lot of power over us, should be one of the first steps in the context of socio-digital vulnerability.
How do we move forward?
In his conclusion, Calo mentions that once reliance on a robot has been created, it cannot be taken away immediately. In other words, the process of reducing the reliance of an individual on technology is gradual, not immediate. Although the recent advancements in technology have given rise to multiple affordances and drastically changed our lives to be more comfortable and connected than ever, they also interfered with our lives in more ways than one. The law remains imperfect in regulating those interactions, which pushes us to think deeper about what should be considered lawful and what shouldn’t. Moreover, when interacting with technology, one must try to take a step back and ask themselves if there would be a better way for them to accomplish a task with technology, or if they are being convinced into doing something they would not normally do.
By Sanjay Thallam
Cornell Tech
For decades, science fiction has dreamed about being able to speak to our devices as if they were people. Famous characters such as C3PO from Star Wars and Jarvis from Iron Man became fan favorites, using their futuristic AI technologies to help their characters along their journeys. Just a few years ago, it would have been unthinkable that such capable bots would become prevalent in society but today we are seeing the advent of Large Language Models and Personalized Chatbots. These new elements have many abilities to act as sources of knowledge and as a spot to vent, however, there is an unintended side effect: the safety and privacy of vulnerable people.
In his paper – Socio-Digital Vulnerability – and his speech to our class Dr. Calo emphasized the different elements responsible for Vulnerability and how laws & policy integrate with this broad field. At a high level, vulnerability refers to the idea that a certain group of people may be more inclined to exploitation by another entity – whether intentionally or unintentionally – with manipulation being the action of harnessing an individual’s vulnerability to one's own gain. Typically in society, it is seen as wrong to take advantage of the vulnerable and to render others vulnerable, but, new social technologies are breaking down this socio-normative aspect. One key example of this from Dr. Calo’s talk that stuck with me was Replika AI and the impact it had on its consumers. Replika is a chatbot that a user can customize based on their preferences and interests and was originally seen as a benign virtual friend. However, after building a “bond” with the user, Replika would propose taking things one step further than friends to lure vulnerable people into springing for the premium version of the app. When the Italian Garante discovered this, they banned these more adult advances from Replika stating they were trying to protect children on the internet but with the unintentional consequence of ending dozens of “relationships” that users had with the bots, causing an outcry in the community. This example highlights the impact that seemingly harmless virtual connections could have on real-world people and exemplifies the impact of human-digital connections.
Currently, our law system sees vulnerability in a binary way, however, in today's technological world it may be that we need to move to a system with more nuance and shades of gray when approaching this topic. Traditionally, one major group of vulnerable individuals is late-in-life seniors who may be susceptible to coercion regarding their will from outside parties such as children, thus our systems have protections to protect these seniors. This is a very static way of looking at vulnerability as it imposes it upon certain groups or archetypes but are more people than just the traditionally vulnerable considered vulnerable? If everyone is considered vulnerable, is anyone vulnerable? These are some of the tough questions facing policymakers today.
To better understand socio-digital vulnerability, it may be helpful to understand more established forms of this topic. The first of these is Decisional Vulnerability where people in power – such as a merchant – can subtly manipulate a person’s view of a decision to skew it in their favor. One such famous example of this is with prices ending in $0.99 to make individuals feel like they are spending less money. After all, 9 dollars is single digits whereas 10 is a lot bigger number. Taking this into the digital space, researchers are noticing a trend of “invisible influence” where ML algorithms are slowly affecting the way our decision-making online without us even knowing, slowly withering away our autonomy. A second form of vulnerability is Social Vulnerability which refers to the susceptibility of communities to the impact of outside hazards such as poverty or limited networks. However, with the rise of chatbots, they are further amplifying these effects as we are unable to find vulnerabilities in AI whereas they can easily determine our weaknesses as humans. The third element of vulnerability is Constitutive vulnerability which is more focused on who we are and what we want to be as people. This is where the inner psyche itself is manipulated by finding the deepest aspirations of people and using them to promote a company’s commercial or political intentions.
Seeing all of this, it may seem as if we are destined to be taken advantage of by technology in the end with no end in sight, however, with evolutions in law and approach, we can cause change to help society. One of the major concepts is the idea of power versus harm laws. Right now, a lot of the technology-centric laws are “harm” laws focused on anticipating harm and then trying to deter it. For instance, trying to deter robberies with the threat of many years in prison. The alternative to this is the idea of “power” laws focused on trying to make the victim more powerful by evening the playing field. Going back to the aforementioned example, one way of giving the cashier more power is by reducing the number of firearms present which may even the playing field with the robber. Now how could we take this concept of power laws and integrate it with the larger technological space is the question at hand today? Although there is much to be done, we must be cognizant of socio-digital vulnerability and ensure that the actions taken when implementing these new regulations for the digital age put power back in the hands of the people and make them more resilient in the face of ever-encroaching technology.