Algorithmic Bias: When Systems Treat People Unequally
Byte the friendly robot stands at an illuminated wall of prediction cards, pointing to cards sorted into unequal stacks while a diverse group of illustrated figures watches from below
- Define algorithmic bias as a system that produces systematically unfair outcomes for certain groups
- Identify at least three distinct causes of algorithmic bias beyond skewed training data
- Explain how algorithmic bias creates social, ethical, economic, and cultural impacts on real communities
- Evaluate a described computing system and identify which cause of bias is most likely at work
- Propose one concrete design change that would reduce a named bias risk
Key terms
- Algorithmic bias
- Systematically unfair outcomes a computing system produces for certain groups of people.
- Feature selection
- Choosing which data attributes a model uses, a choice that can encode inequality.
- Feedback loop
- When a system's outputs reshape future data that then reinforces the original prediction.
- Proxy variable
- A feature that correlates with a protected attribute and reintroduces bias indirectly.
- Problem framing
- How engineers define success, which can favor one group from the start.
Causes Beyond Skewed Data
Skewed training data is the most cited source of bias, but it is far from the only one. Feature selection can quietly bake in inequality when a chosen attribute correlates with a protected group. Label bias enters when annotators apply prejudiced judgments. Feedback loops let a system's own outputs become tomorrow's training signal, amplifying early errors. Problem framing sets the optimization target, so a poorly chosen success metric guarantees a biased optimizer. Auditing for fairness means inspecting each of these decisions, because a system can be unbiased in its code yet biased in its outcomes.
Impacts Across Four Dimensions
Biased systems cause harm that spreads well beyond a single wrong prediction. Socially, biased admissions or hiring tools reshape who gains opportunity across generations. Ethically, treating a person as less trustworthy due to group membership violates basic fairness. Economically, communities flagged as high-risk by credit or insurance models pay more for loans and coverage, a direct financial penalty. Culturally, recommendation systems that amplify only some voices narrow whose stories get heard. Evaluating impact, not intent, is the standard, because well-meaning engineers can still build systems that compound existing injustice.
Worked examples
Diagnose the bias mechanism in a predictive-policing system whose predictions keep coming true.
- The model flags certain neighborhoods as high-risk, directing more police patrols there.
- More patrols produce more recorded arrests in those neighborhoods, independent of the true crime rate.
- The new arrest data is fed back as evidence of risk, which the model reads as confirmation.
- The cycle reinforces the original prediction, so 'predictions coming true' is self-fulfilling, not validation.
Answer: A feedback loop: the system's deployment shapes the very data later used to validate it.
Activity
Sort each scenario into the cause of algorithmic bias it best illustrates — Skewed Data, Proxy Feature, Feedback Loop, or Label Bias
Practice
A resume model excludes race yet still disadvantages a group; identify the cause and a fix.
Suggest one concrete design change to reduce feedback-loop bias in a content-moderation system.
Common mistakes to avoid
- Identical code means no biasBias is judged by unequal outcomes, so code that treats inputs identically can still be biased.
- Excluding protected attributes ensures fairnessProxy variables can re-encode the excluded attribute, letting bias persist despite exclusion.
Check your understanding
A face-recognition system is tested and found to misidentify people with darker skin tones far more often than people with lighter skin tones. An engineer claims the system is unbiased because 'the algorithm treats everyone the same.' Why is this reasoning flawed?
A predictive-policing model is deployed in a city. Over two years, crime statistics show that arrests rise sharply in the neighborhoods the model predicted as high-risk, while other neighborhoods show fewer arrests. A city council member concludes the model is working perfectly because its predictions are 'coming true.' What bias mechanism best explains what is actually happening?
A company builds a resume-screening model. Engineers deliberately exclude race, gender, and national origin from the feature set to keep the system fair. A year later, an audit finds the model still produces outcomes that disadvantage applicants from certain demographic groups. Which cause best explains this?
A municipality's automated benefits-eligibility system is found to approve far fewer applications from non-English-speaking households. An engineer argues the system cannot be biased because it makes the same binary yes/no decision for every application. A city official argues this is still a serious harm. Which of the following best supports the city official's position?
Recap
Algorithmic bias means systematically unfair outcomes for groups, caused by skewed data, feature selection, feedback loops, label bias, and problem framing. Its impacts span social, ethical, economic, and cultural dimensions and are judged by outcome rather than intent.
Reflect
Which cause of bias do you think is hardest for engineers to detect, and why?