Why does a model repeat a mistake that was in its examples?

It learns from the data, mistakes included. Garbage in, garbage out: a wrong label in the examples turns into a wrong answer from the model.

A comment filter was trained almost only on English examples. What is most likely?

It will catch rudeness in other languages worse. That's data bias: whatever is rare in the examples, the model knows worse.

What is a label in training data?

The right answer a person attached to an example. A label tells the model which answer is right for that example, like “toxic” or “normal”.

Data decides everything

Lesson 6 of 23 in the course “AI basics”. Free, with animations and a quiz.

A model is its data

Data means the examples a model learns from: texts, photos, records. The model gets no ready-made rules. It finds patterns in the examples by itself. For example, you want a filter for the comments under your videos. You give it 10,000 old comments, and it learns to tell toxic ones from normal ones. Whatever was in the examples, it can do. Whatever wasn't, it doesn't know.

Labels: the right answers

For the filter to learn what a toxic comment is, every example needs a label. A label is the right answer a person attached to it, like “toxic” or “normal”. The model learns to repeat these labels. The catch: people make the labels, and people make mistakes. If a nasty comment is labeled “normal”, the model learns that mistake too and repeats it later.

Garbage in, garbage out

Programmers have a rule: garbage in, garbage out. A model is no smarter than its data. Wrong labels, duplicates, random spam mixed into the examples: it soaks all of that up and repeats it in its answers. Just adding more data won't help if that data is messy too. In short: a small but clean set of examples often beats a huge, dirty one.

Skewed data, skewed answers

If the data has far more examples of one kind than another, the model repeats that imbalance. This skew is called bias. For example, your filter learned mostly from English comments, so it misses rudeness in Russian slang. Chatbots are the same: the internet has more English text than any other language, so models often answer better in English and tend to see the world through the eyes of English-speaking countries.

Old data, old model

Data is collected at some point, and after that the world keeps changing without the model. A year later there's new slang, and your filter won't get it: a fresh insult slips right through. For chatbots this is the knowledge cutoff from the first lesson: the model may know nothing about what happened after it. Takeaway: check fresh trends, prices and news in up-to-date sources.

Data is someone's words

Training data can include personal stuff: names, posts, conversations that someone made public. A model usually doesn't store them word for word, but sometimes it can repeat a piece. Your own chats may also end up in training. Many chat apps let you turn this off in the settings, so check yours. Bottom line: don't send a chat anything you aren't ready to hand to strangers.

Check yourself

Why does a model repeat a mistake that was in its examples?
Correct answer: It learns from the data, mistakes included. Garbage in, garbage out: a wrong label in the examples turns into a wrong answer from the model.
A comment filter was trained almost only on English examples. What is most likely?
Correct answer: It will catch rudeness in other languages worse. That's data bias: whatever is rare in the examples, the model knows worse.
What is a label in training data?
Correct answer: The right answer a person attached to an example. A label tells the model which answer is right for that example, like “toxic” or “normal”.

← Why the answers differ every time How a chatbot is trained: 3 stages →