AI Reflections: Perpetuation of Bias 

AI uses data to generate content, identify patterns, summarize, and make predictions. This data exists as information firmly located in our past and present, even as it tries to move us into the future. When we consider the systems that produce the most data: education, healthcare, and legal systems, to name a few, we can see their outputs are shaped by the ongoing impacts of the colonization of Indigenous territories. We know that GenAI tools “crawl” and “fetch” information across the Internet to train their algorithms and generate responses to queries, which means any reports, news articles, videos, or publications that share this data, and the biases reflected in the data, are now incorporated into the AI’s modelling. 

The Internet, while a more “open” forum than information-sharing avenues of the past, is built on a long legacy of publishing books, articles, surveys, and reports from Indigenous peoples’ data without their consent or participation. Additionally, information by and for Indigenous peoples is not always published widely, and may not reach the Internet at all. When we prompt a GenAI tool with a question about an Indigenous community, their language, history, or territory, we must understand that there are limited reliable data points the AI can pull the responses from. Many Indigenous knowledge systems are place-based and relational, so while community-approved information exists online, when providing responses to prompts, GenAI decontextualizes this information from its source in a way that is opaque for users. The process of GenAI taking and using Indigenous data without consent has been referred to as technological or digital colonialism and has the real potential to replicate harm. 

Below, we share examples of how colonial bias and racism are embedded into AI algorithms, in applications as wide ranging as image generators to our legal system. We round out this post by sharing some interventions Indigenous peoples are creating to resist the embedding of ongoing colonial harm into our technological systems. We encourage you to engage with these tools in your own practice. 

Questions for Discussion/Reflection 

• What are some examples of bias you have encountered while engaging with AI?  
• How might bias in GenAI impact a student’s experience using the technology for an assignment? 
• Is it possible to eliminate or decrease bias in AI? Whose responsibility is it to do that work?  
• If AI is being used to make decisions based on flawed data, what rights should people have to appeal those decisions?  

Perpetuating Settler Colonial Narratives 

A recent paper analyzed the output of OpenAI’s image generation tool Sora, finding that the images it generates closely mirror dominant settler colonial narratives. The Aotearoa New Zealand based researcher Olli Hellmann prompted Sora to generate images of the country’s past. Sora’s output included images which showed landscapes of empty and untouched wilderness, colonial rule as orderly and benign, and first encounters as ‘civilizing’. The images also portrayed Māori as timeless and passive, with incorrect representations of gender roles. The author goes on to argue that the biases and stereotypes being perpetuated by GenAI matter because how people view history affects their political decisions in the present:  

“In Aotearoa New Zealand today, issues such as the role of te reo Māori in public life, co-governance, Treaty settlements, and constitutional reform are all animated by contested historical narratives. Imagery that portrays the colonial past as peaceful and consensual can diminish the perceived urgency or legitimacy of Māori claims to political sovereignty and redress through institutions such as the Waitangi Tribunal, as well as calls for cultural revitalisation.” – Olli Hellmann 

Legal System 

Around the world it is becoming increasingly common for governments to use AI at various stages in their legal system from algorithmic policing to sentencing and parole decisions. One example is the use of facial recognition in investigations, a study from the National Institute of Standards and Technology found that in the databases used by US law enforcement the highest rate of error was found when identifying Indigenous people.  

Other examples include AI that is being trained on historical incarceration data, data created as part of a deeply flawed justice system. According to a 2020 report from the Congress of Aboriginal Peoples, Indigenous people represent 5% of Canada’s population but 30% of people in federal custody. The Department of Justice’s website itself acknowledges the impact of historical and ongoing colonial policies, systems, laws and processes on the overrepresentation of Indigenous people in the justice system. Criminal risk assessment algorithms use a number of different factors to create a score which is meant to represent the likelihood that a person will reoffend. Unfortunately, the statistical patterns that machine learning algorithms are identifying are based on correlations, not causations. Now populations who have been disproportionally targeted by policing in the past are facing a future controlled by algorithms perpetuating biases, and in the process creating even more biased data to feed back into algorithms.  

Although the Canadian legal system is not yet as invested in AI as its southern neighbor, there are people and committees considering potential future uses here. More must be done to include Indigenous communities in the decision-making process, as stated in a recent report on AI from the Chiefs of Ontario “It is crucial that First Nations individuals and First Nations communities have avenues to object to AI decision making that could be the result of algorithmic bias.” As a country we are responsible for responding to the 94 calls for action in the TRC, including the 18 related to Justice, it is essential that moving forwards decisions around AI in our Justice system are in alignment with all of the work that went into the TRC.  

Tools to Counteract Bias 

While there are many concerns and issues with dominant AI systems, we would be remiss to not mention some of the work Indigenous peoples are already doing to counteract some of the biases inherent in these technologies. One such tool is called wâsikan kisewâtisiwin, which translates to “kind energy” in Cree, created by pipikwan pêhtâkwan. wâsikan kisewâtisiwin has two components; the first is to help moderate online spaces, such as comment sections, by flagging hateful comments and providing suggested responses. The second component works similarly to Grammarly, providing support for writing by flagging biased content, explaining why and how it’s biased, and providing suggestions for revisions. 

Another tool we’ve found, Octavia, is designed mostly for settlers, as part of a project called Settler Responsibilities. While you engage with Octavia Cayenne Pepper similarly to how you would engage with a GenAI chatbot, Octavia was developed “to support Indigenous-led movements and communities in the laborious work of educating settlers—without demanding yet more emotional labour from Indigenous individuals.” Octavia responds to text prompts with suggestions to reframe, reflect, and restructure the user’s approach. 

One thing we want to highlight is the way both of these tools encourage us to slow down and create space for education and reflection as we engage with AI and the world around us. In the absence of having a specific tool, we can apply these same ideas to support engagement with information about Indigenous peoples (GenAI produced or not). It is our responsibility, as human beings, to ensure we are not replicating patterns of colonial violence. We encourage you to engage with these, and other, tools being developed by Indigenous peoples, and to consistently engage in reflective practice when using AI. 

Do you have any other examples you’d like to share? Questions or comments on this post? Please don’t hesitate to get in touch; we’d love to hear your thoughts. (emily.bridge@ubc.ca, carissa.block@ubc.ca)