Lesson 2 of 7

Bias & fairness

6 min read

An AI screening tool quietly favours one kind of applicant. Nobody programmed it to. So where did the bias come from — and can it be fixed?

It learns the data — warts and all

A model has no opinions; it learns the patterns in the examples it's trained on. If those examples carry human bias — who got hired before, whose faces were labelled, which voices were transcribed well — the model soaks it up and repeats it. Skewed data in, skewed calls out. It can even amplify a small imbalance into a strong one.

A model mirrors its data. Feed it an unfair pattern and it learns to repeat it.

Fairness is work, not a switch

Because the bias lives in the data — and the world the data came from — there's no single toggle that removes it. Teams mitigate: balance the training data, test outcomes across different groups, and keep a human in the loop for high-stakes calls. It reduces harm, but 'unbiased' is a direction, not a finished state, and subtle bias hides in signals no slider shows.

Bias is mitigated, not deleted — measure outcomes across groups, and keep a human on the big calls.

If an AI tool is deciding something that affects people — hiring, lending, grading — ask who checked it for bias, and how. 'The algorithm did it' is not an answer.