What a year of living safely has (not) taught us about algorithmic bias
Increasingly "AI" has come to mean augmenting inequality, unless we reverse the tide..
The pandemic and digital transformation
We have all heard the often-repeated quote that the pandemic ushered in two years of digital transformation in two months.
Along with my doctoral student Jiaoping Chen and a colleague Uttara Ananthakrishnan at another university, I wondered whether this was true across the board or only for smaller organizations. We examined how fitness companies adopted digital transformational experiences – such as virtual live classes and on-demand libraries of content – to be resilient throughout the pandemic where other industries faltered.
The pandemic has heightened the anxiety associated with physical exercise in densely populated indoor facilities. In my opinion, the fitness industry – spanning from national chains like Planet Fitness to boutique studios – faced a perfect storm of cataclysmic events with the downturn of foot traffic, as well as uncertainty about future growth and operational costs.
We used SafeGraph and Google Places to gather and analyze visits and foot traffic from nearly 3.6 million commercial and social locations based on approximately 45 million mobile devices and applications. Across the US, COVID-19 related lockdowns and re-openings gives us a natural experiment to study the impact on consumer demand. We found that gyms with virtual services see about 37% decrease in footfalls post-pandemic, in comparison to dentists. With digital substitution becoming widely available and acceptable, it is likely that those businesses that are not digitally resilient will not survive.
In the past decade, we’ve seen several aspects of human behavior digitized and while the pandemic-led short-term changes in behavior, it can be argued that shifts in business models and consumer preferences will persist long term. In a sense, being digitally resilient in operations could be a way for fitness centers to self-cannibalize their own foot traffic by shifting to digital modes of services.
A history of virtualization
The Jim Carrey movie The Truman Show depicts a protagonist whose life is a live television show. Like it or not, the pandemic made us all inadvertent participants in a large-scale experiment in everything digital. Kindergartners are learning on zoom, and artists offering live concerts or lessons on Instagram and Facebook Live. Many sectors of the economy adopting remote work. Media and entertainment rushing to stream new movies online while higher education and healthcare are embracing virtual services.
The digitization and virtualization of everyday experiences has been occurring before the pandemic. Virtual reality has already made inroads into live concert experiences. Theme parks have interactive experiences between merchandise items and tourists. Toymakers have blended interactive games and animation on smart phone or tablet devices into physical toys.
The pandemic has accelerated this process and we could expect more of experimentation in infusing virtual elements into real experiences, or even using virtual experiences to transcend the limitations of the physical. A Fortnite game called pleasant park enabled a concert hosted by music producer Marshmello that enabled nearly 11 million people to participate. This is notable because it was an entirely virtual event. Such developments in turn affect our daily life, such as how we shop, socialize and learn where we are increasingly turning to digital spaces, be it apps, social media, online communities or digital platforms such as Amazon. Online and hyper-local communities are enabling individuals to discover local resources. For instance, 'Dental Cupid' creates matches between emergency patients and dentists.
So what changes now?
In an earlier study, I looked at how participation in virtual public goods, such as Waze, or online reviews, changes when we observe others behavior. While people tend to contribute more to a virtual public good if they see others doing the same, this effect reverses if they become aware too many people are participating. Combining methods from geography, urban planning and big-data analysis, my co-authors and I studied millions of postings by users of a mobile navigation app called Waze, in which users voluntarily post traffic-related updates and road conditions in real time. Displaying user’s “virtual activities” can both encourage or discourage prosocial behavior of the whole community depending on the perceived degree of other individuals’ participation. That is, the motivation for individuals to participate and the value they create to others depends not only on physical proximity but also from virtual proximity caused by how users experience their virtual interactions.
As many sectors of the economy from healthcare to education embrace virtualization, it is important to consider how to design digital spaces (be it mobile apps, online communities or digital platforms) that foster a sense of virtual community and virtual co-presence. We need to richer virtualized experiences that parallel physical experiences. In parallel, we need more awareness of algorithmic biases and algorithmic interaction effects. The matching of consumers to services, and interactions between consumers occur through an algorithmic space. This can change, say shopping experiences, where, instead of spatial proximity, we may now witness virtual proximity through algorithmically engineered interactions between consumers.
Understanding how algorithms shape our experience with virtualized experiences
The challenge in online settings is that of information overload, wherein we are deluged with the enormity of content. It has been estimated that 400 hours of video content are uploaded to YouTube every hour. Platforms use algorithms to curate this enormity of information, by personalizing and contextualizing our experiences in providing relevant content. Social media platforms, for instance, tailor their news feed by prioritizing stories based on their perception of what is relevant and engaging to users. While digital platforms offer us a curated and personalized profile, in reality they may be hiding other choices or even dissembling to us. For instance, dating apps show profiles of individuals based on revealed preferences and not stated preferences. Online education providers, for instance, are using AI and their enormous trove of analytics to understand what engages students, and create personalized profiles for students powered by algorithms. As our social and economic transactions take place in digital spaces, it is likely that algorithms will have greater power in dictating our choices.
The sociologist Erving Goffman coined the term “interaction order” wherein individuals achieve their social selves through their interaction with others. With many observers positing that we will continue with the digital transformations unleashed by the pandemic, algorithms are not only upending our patterns of interaction, but also playing a vital role in how we see our social selves (or rather, how we see our virtual selves) including how we interact with each in digital spaces. As we navigate these digital spaces where most of our interactions take place, algorithms nudge or steer us in ways that our virtual experiences could mirror our physical experiences or expand the possibilities offered in a physical setting. The key to understanding the lasting changes unleashed by the pandemic is in understanding how we interact with each other in digital spaces.
The virtualization of everything means that old ways of socialization cannot now be transported to the Internet. Teaching online is not a substitute for actual face to face interactions. While it could expand the range of possibilities that are open to universities and students, such as gamification of learning and interactive experiences, the dark side of digitization and virtualization is algorithmic bias, algorithmic amplification and algorithmic harms. Educators have raised concerns about algorithmic bias in online learning platforms and tools. A serious harm is that learning management systems used by colleges could misidentify students as low-performing, which could lead their professors to treat them differently or otherwise disadvantage them.
Algorithms caused a grading crisis in Britain, when algorithmic scoring resulting in the lowering of grades of nearly 40% of students who could not take the exams due to Covid. It turned out that, among other things, the algorithm looked at the school’s track record, which ends up favoring students from private schools and affluent areas, while high-achieving students from public schools are disproportionately affected. A Brookings report from 2019 presciently warned “AI is coming to schools, and if we are not careful, so will its biases.”
The British Medical Journal recently published a piece that asked, “Does “AI” stand for augmenting inequality in the era of covid-19 healthcare?”