AI + your documents (RAG)
Lesson 15 of 23 in the course “AI basics”. Free, with animations and a quiz.
The model never read your files
A model only knows what was in its training texts. It has never seen your class notes, your job's rules or that PDF from uni. Ask it about a course deadline and it will simply guess. RAG (in plain words: find first, then answer) fixes this: a program searches your files for the right piece, and the model answers from that piece. The diagram shows it: question, search, found piece, answer with a source tag.
Search by meaning, not by words
First, the documents are cut into small pieces, for example one paragraph each. Each piece is turned into an embedding: a list of numbers that describes its meaning, like the coordinates of a point on a map. Your question becomes a point too. The search picks the pieces whose points sit closest to the question. That's why “when is the essay due?” finds the paragraph “paper submission deadline: end of May”, even with different words.
What it finds goes into the prompt
Next, the app pastes the found pieces straight into the prompt, meaning the text request sent to the model, right next to your question. It also adds an instruction: “answer only from these texts and name the source”. The model writes the answer and adds a tag like [1]: the number of the piece the idea came from. All of this happens behind the scenes. You only see the answer with its sources.
Fewer made-up answers, but not zero
Why it makes up less: the model retells the text in front of it instead of recalling from memory. Why not zero: the search can bring the wrong piece, like an old version of the rules. The right piece may not exist at all, and then the model fills in the gap itself. It can also mix up two pieces or misread a table. Pro tip: add “if the documents don't have the answer, say so”.
Where you'll run into it
Chat with a PDF: you upload a textbook or a rental contract and ask questions. A company support bot: it answers from the company's internal rules, not from memory. AI search: the model finds web pages and writes an answer with links. It's the same recipe every time: find pieces, paste them into the prompt, answer with a source. In short, RAG is a cheat sheet slipped to the model right before it answers.
There's a source? Open it
A [1] tag is not a guarantee. Sometimes the link leads to a piece that says something else, or the model retold it wrong. Before an important step (handing in work, signing a contract, spending money), open the source and find that exact sentence. It matches? Great. Can't find it? Treat the answer as a guess and ask the model where exactly that is written.
Check yourself
- What does RAG do before the model answers?
Correct answer: Finds matching pieces of your files and puts them in the prompt. Nobody retrains the model. A program finds pieces close in meaning and slips them into the prompt, and the model answers from them. - You asked “when is the essay due?” and it found “paper submission deadline”. Why?
Correct answer: Texts with similar meaning get nearby embedding points. An embedding describes meaning with numbers. A question and a paragraph about the same thing end up close on the map, even with different words. - A documents bot answered with a [1] tag. What should you do before an important decision?
Correct answer: Open source [1] and find that sentence there. The search can bring the wrong piece, and the model can retell it wrong. Checking the source takes a minute.