Bias in AI Systems

AI models carry significant risks to produce biased outputs. Bias, when applied to AI models, generally manifests as either cognitive bias or computational bias. Below, we describe both types of bias, their different forms, and their potential harms.

Cognitive Bias

Cognitive bias primarily comes in three forms: (1) implicit bias; (2) sampling bias; (3) temporal bias.

Implicit bias leads to outputs that discriminate or prejudice a specific group. This is a common problem when AI models produce outputs that either foster discrimination or bias, or deepen existing racial and socio-economic divides. Implicit bias may also lead to a chilling effect on free speech and participation in activism and protests by certain groups (who have been targeted as a result of the implicit bias) and who may be exposed to greater threats of tracking and profiling. 

Sampling bias skews the data to a subset of a larger group (meaning it is biased toward the “sample” it was trained on). For example, if an important subset of a group is left out of the training data, the output is likely to be subject to sampling bias. AI models trained on biased or non-representative datasets can produce discriminatory or flawed outputs. These risks are especially pronounced in customer-facing applications or when AI tools influence hiring, lending, or healthcare decisions. Unchecked, this can even go so far as to cause the spread of ideological bubbles and foster the spread of disinformation. 

Temporal bias, which occurs when the training data is outdated, produces models that do not perform well in the future. Temporal bias is common when the training data is based on “trending” topics or patterns, which can occur frequently among consumers whose reading and purchasing behaviors are driven by social media. 

Cognitive bias, overall, produces one of the greatest societal risks generative AI poses to individuals and groups. Cognitive bias threatens to show users content and outputs that align with existing harmful beliefs, without exposing them to contracting perspectives. This poses a serious threat to truth and authenticity, while also diminishing shared experience and understanding between and among communities. 

Computational Bias

Computational bias typically results when there is an error with the training data set. This should not come as a surprise: put bad data in, get bad data out. 

Computational bias typically manifests in one of three ways: (1) outputs that are overfitted to the training data; (2) outputs that are underfitted to the training data; and (3) outputs that are skewed by edgecases, outliers, and noisy data. 

When data is overfitted, the model is less effective with new data. Underfitting occurs when the training data does not encompass the complexity of the training data (which leads to bad predictions and/or inaccuracies). Additionally, the presence of edge cases, outliers, or noisy data may adversely impact the model’s learning and effectiveness, which could lead to bias (or even model drift) over time. 

Computational bias can lead to a number of harms, notably institutional harms (think: reputational damage, legal challenges, and economic losses - all resulting from the inaccuracies in the training data set). As a result, it is vital for companies to identify risks early in the development process and engage a diverse group of stakeholders in comprehensive settings. 

How to Mitigate Bias in AI Systems

AI models carry significant risks to produce biased outputs, and in nearly all scenarios (particularly hiring and firing), they should not be used or relied on for sole decision-making. Enterprises using AI vendors should seek out vendors that prioritize transparency, explainability, and interpretability in the design of their models to minimize risks of bias.

Transparency in AI systems involves both the information regarding the systems (how they function and work) as well as the actual technical and nontechnical documentation maintained across the AI system lifecycle. 

Explainability in AI systems means a system is supported by documentation that demonstrates how the system came to a specific output (conclusion or decision). Importantly, the term “explainable” refers to an explanation produced after the decision has been made. 

Interpretable AI systems is a concept that emphasizes building models that can be understood by humans.

In addition to working with vendors to ensure bias mitigation measures are part of their development process, it is important to establish internal audits to monitor AI outputs for discriminatory or harmful patterns, before they are deployed within or by your enterprise. If you have questions or need help, contact us at info@ambartlaw.com, to see how one of our attorneys can assist you.