Inside a chat: system, user, history
Lesson 11 of 23 in the course “AI basics”. Free, with animations and a quiz.
A chat is one long document
It feels like a back-and-forth conversation. Under the hood, the app glues everything into one document with roles. System: the app's hidden instructions. User: your messages. Assistant: the model's replies. The model receives the whole document and writes the next assistant line. Sounds weird, but it's simple: a chat is a script the model keeps continuing.
System: the hidden settings
The system message sits at the very top, before your first word, and you usually don't see it. The app's makers write it: what tone to use, what not to answer, which tools are available. Custom instructions, meaning the settings field where you write something like “keep it short, I'm a student”, also go into this part. That's why they apply to every new chat.
It rereads everything every time
Between messages the model remembers nothing. When you send a new one, the app sends it the entire chat history again, top to bottom. But it can't see an endless amount. The context window is how much text the model can read at once, counted in tokens, meaning pieces of text. In a very long chat, old messages can fall out of the window or get less attention.
When a chat goes off track
History helps as long as it's useful. Example: you're writing an essay, you rejected the first outline, but the model keeps going back to it. Why: the old outline is still in the history, and the model rereads it every time. Fix: start a new chat. Paste a short summary of what matters: the topic, what's decided, what doesn't work. A clean history often beats ten arguments in the old one.
Does AI remember other chats?
The model itself doesn't: it isn't retrained on your chats on the fly, and every new chat starts as a blank page. But many apps have a memory feature. It saves notes about you, like “studying for a chemistry exam”, and inserts them at the start of the chat, like a system message. You can usually view and delete these notes in settings. Worth checking what's in there.
Check yourself
- You send a new message in a long chat. What does the model get?
Correct answer: The whole chat history again, top to bottom. Between messages the model remembers nothing, so the app sends the whole history every time, as long as it fits in the context window. - The model keeps returning to an idea you rejected. What's best?
Correct answer: A new chat with a short summary of what matters. The rejected idea sits in the history, and the model rereads it. A new chat doesn't have it, and you bring over what matters yourself. - Where do custom instructions from settings go?
Correct answer: Into the hidden system part of every new chat. The app inserts them at the start of every chat, like a system message. They don't retrain the model itself.