What is RAG? The technique that makes bots answer from your data instead of inventing

What is RAG in AI?

RAG stands for Retrieval-Augmented Generation. It combines two steps: retrieving relevant information from sources you own, then generating the answer with a large language model based on it. In other words, instead of the model answering from its general memory, we hand it a “reference sheet” from your documents with every question and ask it to answer from that.

Why is it needed? A model on its own doesn’t know your store’s prices or your company’s return policy, and its knowledge stops at its training date. Asked about what it doesn’t know, it may write a convincing but wrong answer, which is called “hallucination”. RAG connects the model to an up-to-date knowledge source of your own, so the answer rests on existing text that can be checked.

How RAG works step by step

The first three steps happen in advance and repeat whenever your data changes; retrieval and generation happen the moment each question arrives:

• Ingestion: documents such as PDF and Word files, Excel sheets and the product database are collected and their text extracted, with optical character recognition (OCR) for scanned files.
• Chunking: documents are split into small passages that hold together in meaning, keeping the document title and update date with each one.
• Embedding and indexing: each passage becomes a numeric representation of its meaning (an embedding) stored in a vector database, often with a keyword index so search is hybrid.
• Retrieval: the question is embedded the same way and the closest passages are found, with the user’s permissions applied so only what they may see is retrieved.
• Generation with citations: the passages go to the model with clear instructions: answer only from these, cite your source, and say you don’t know if the answer isn’t there.
• Logging: the question, the passages used and the answer are kept for quality review.

A worked example: a customer question on WhatsApp

A Baghdad home appliances store has a dinar price list in Excel and a warranty policy in a PDF. A customer asks on WhatsApp, in Iraqi dialect, how much a certain air cooler costs and whether it has a warranty. The system retrieves the product’s row and the warranty paragraph, and the model replies with the price and period exactly as the documents state. If the model isn’t on the list, it doesn’t guess a price; it hands the chat to a staff member.

RAG or fine-tuning? Why RAG usually beats “training” the model on your data

Because your data changes: prices, policies, stock. With RAG, updating and re-indexing the document or database changes the answer, with no costly retraining. You can also see which source an answer used, and remove information by removing its document.

Fine-tuning, by contrast, adjusts the model itself; it’s useful for fixing a style or answer format, not for teaching it changing facts. The two can be combined: fine-tuning for tone, RAG for facts.

What data does an Iraqi business feed into RAG?

Before indexing, ask of every source: who owns it? when does it change? is it the approved version? A forgotten old price list can make the bot quote a wrong price. Stock and live prices are better fetched directly from the system.

That’s what an AI knowledge assistant that answers from your company documents is built on, for customers on WhatsApp or staff internally. The most common sources:

• Price lists in Excel, in Iraqi dinars or dollars, with units and offers.
• Product catalogues and specifications, often PDF files from suppliers.
• Delivery, returns, warranty and payment policies.
• FAQs drawn from past WhatsApp chats, with their approved answers.
• Staff handbooks, internal procedures and HR policies for an internal assistant.
• Contracts, forms and circulars, some of them scanned Arabic PDFs.

Arabic and Iraqi dialect challenges in RAG

In Iraq there are extra challenges that must be tested before launch, and we cover the colloquial-writing side in building a bot that understands Iraqi dialect:

• Dialect question, formal document: the customer writes Iraqi colloquial words for “how much” and “is there” while the document uses formal Arabic. The fix is an embedding model that handles Arabic well, sometimes rewriting the question before searching.
• Mixed product names: the same name may be written in Arabic or Latin letters and spelled several ways, so hybrid search plus a synonym glossary for your products helps.
• Non-standard spelling: hamzas, taa marbuta and alif maqsura are normalised in text and question before searching.
• Scanned PDFs: Arabic OCR can confuse similar letters and tables, so review a sample of the extracted text.
• Tables and numbers: a price list is chunked so product name, price and currency stay together, with Arabic and Western digits normalised.

RAG’s limits

Answers are only as good as your documents: contradictory or outdated documents produce contradictory answers. That’s why serious RAG projects start by cleaning the knowledge base and naming an owner to update each document.

Most importantly, RAG reduces hallucination but doesn’t eliminate it, so sensitive answers cite their source and the question goes to staff when there’s no clear answer. The main things that can go wrong:

• Wrong retrieval: a passage close in wording but far in meaning, giving a well-phrased but wrong answer.
• Wrong merging: the model mixes up two facts, or adds a detail that isn’t in the source.
• Stale index: a document changes but isn’t re-indexed, so the bot keeps the old information.
• Permission leaks: if permissions aren’t enforced at retrieval, a user may see what they shouldn’t.
• Instructions injected inside a document: which is why documents are treated as information, not commands.

How to evaluate a RAG system before and after launch

Don’t judge the system on a few test questions you type yourself. Serious evaluation separates retrieval quality from answer quality, and is repeated whenever documents or the model change:

• Build a test set of real questions from your customers and staff, in their dialect and with their typos, with the correct answer and its source.
• Measure retrieval on its own: is the right passage among the top results?
• Measure the answer: is it correct? Is everything in it found in the source? Did it cite the source?
• Test questions your documents can’t answer: the system should say it doesn’t know instead of guessing.
• Test permissions: staff in one department don’t see another’s documents, and customers don’t see internal documents.
• After launch, review a sample of real conversations regularly, and add the questions the system failed on to the test set.

When is RAG not the right tool?

RAG is excellent for answering from text, but it isn’t a fix for everything, and it’s often combined with other tools inside an AI agent that both answers and acts. Look for an alternative or a complement in these cases:

• The answer lives in a structured database, such as order status or live stock: a direct query through a tool is more precise.
• Calculations and aggregation, like “how much did we sell last month?”: that’s reporting, not passage retrieval.
• A few fixed questions: an FAQ list or a menu bot is enough.
• Contradictory documents: clean them up first, or RAG will carry the mess into its answers.
• Changing the model’s style rather than its knowledge: good instructions or fine-tuning will do.

Want a bot that answers from your company’s documents?

If you have price lists, policies and repeat questions your team answers every day on WhatsApp, start with a small set of approved documents and real test questions. Explore Al-Arjan’s AI agents and WhatsApp bots, or request a quote so we can review your data together and propose the scope of a first pilot.