RAG vs Agentic RAG: What Is the Difference?

Classic RAG follows a fixed recipe: take the user’s question, search a knowledge base once, put the top results in the prompt, and let the model answer. Agentic RAG puts a model in charge of the retrieval itself: it decides whether to search, what to search for, which source to use, whether the results are good enough, and when to search again before answering.

In short, classic RAG is a pipeline, and agentic RAG is an agent that uses retrieval as a tool. The difference matches the one between workflows and agents.

Classic RAG in one paragraph

A classic RAG system splits documents into chunks, creates embeddings and builds a search index in advance. When a question arrives, it runs one search (often hybrid search with reranking), adds the best chunks to the prompt and asks the model to answer only from them, with citations. It is predictable, fast and cheap, and works well when one search can find the answer. See what RAG is.

Agentic RAG in one paragraph

An agentic RAG system gives the model retrieval tools (search the policy documents, query the orders database, search the web) and lets it work in a loop. It might rewrite a vague question, split it into sub-questions, search several sources, read the results, notice that something is missing, search again, and only then answer. Some designs add a separate checking step that verifies the answer against the retrieved text.

Side by side

	Classic RAG	Agentic RAG
Who decides the steps	The developer, in code	The model, at run time
Searches per question	Usually one	Zero to many
Sources	Usually one index	Several tools: indexes, databases, APIs, web
Handles	Direct questions answered by one passage	Multi-part, comparative or vague questions
Latency and cost	Low and predictable	Higher and variable
Failure modes	Bad retrieval, so a wrong or “not found” answer	Loops, unnecessary searches, wrong tool choice, plus everything classic RAG can get wrong
Testing	Easier: fixed steps	Harder: paths differ between runs

An example

Question: “Is our refund policy for business customers different from what it was last year, and how many refunds did we issue to business customers last month?”

• Classic RAG runs one search with the whole question. It may find the current policy, but not last year’s version, and it cannot count refunds, because that number lives in a database, not in documents.
• Agentic RAG splits the question: search the policy documents for the current business policy, search again filtered to last year’s version, compare the two, then call a SQL tool to count last month’s business refunds. It answers with citations for the policy and the query result for the number.

When to use which

Start with classic RAG when:

• Most questions are answered by one or two passages.
• Latency and cost need to be predictable, for example in a customer-facing chat.
• You need simple, repeatable evaluation.

Move to agentic RAG when:

• Questions often need several sources or several steps.
• You see many failures where the first search was simply the wrong search.
• Users accept a slower answer in exchange for a more complete one, as in research or analyst assistants.

A useful middle step is “RAG with a few smart steps”: add query rewriting, or a check that triggers one extra search when the retrieved results score poorly. This captures much of the benefit while keeping the pipeline predictable.

Common mistakes

1. Going agentic before fixing retrieval. If chunking and search are poor, an agent just searches badly more times. Fix the basics first.
2. No limit on steps. Without a maximum number of tool calls and a time budget, agents can loop and run up costs.
3. Measuring only the final answer. Also check which tools were called and whether each search was needed. See evaluating RAG and evaluating agents.
4. Ignoring permissions. Every retrieval tool must respect what the user is allowed to see, in both designs.

Interview angle

For “RAG vs agentic RAG”:

• One sentence each: a fixed retrieve-then-answer pipeline vs a model that plans and repeats retrieval with tools.
• An example question that needs agentic RAG, such as one that combines documents and a database.
• The trade-offs: quality on complex questions vs latency, cost and harder testing.
• How you would keep an agentic system safe: step limits, tool permissions and evaluation of the paths it takes.

Practise with the AI engineer interview questions.

Key takeaway

Classic RAG retrieves once with fixed code and is fast, cheap and easy to test. Agentic RAG lets a model decide what, where and how often to retrieve, which handles multi-step and multi-source questions at the cost of latency, money and predictability. Start simple, add agentic steps where measurements show that one search is not enough.