Agentic RAG: definition, how it works and use cases

Agentic RAG is one of the most talked-about topics in applied AI. Behind the jargon, the idea is simple: give an AI assistant access to your documents, and the ability to carry out its own search. I build this kind of system with n8n for SMEs; here is what you need to understand before getting started.

Quiz · 1 minute

Classic RAG or agentic RAG: what do you need?

1. 1. Where does the information you need live?

   In a single document baseIn two or three sourcesAcross several tools (CRM, accounting, documents)
2. 2. What do the questions look like?

   Direct: “what is the procedure for…”Ones that require cross-checking information
3. 3. What matters most?

   Fast answers and controlled costDepth of analysis
4. 4. Is your documentation up to date and well organised?

   YesNot really

Answer the 4 questions to get a recommendation.

What is RAG?

RAG stands for Retrieval-Augmented Generation. The approach was formalised in 2020 by researchers at Facebook AI Research (Lewis et al.).

A language model only knows what it learned during training. It knows nothing about your procedures, contracts or prices. RAG fills that gap in three steps:

1. Retrieval: the system searches your document base for the passages most relevant to the question.
2. Augmentation: those passages are added to the question sent to the model.
3. Generation: the model answers based on those passages, ideally citing its sources.

For search to work on meaning rather than keywords, documents are split into chunks and converted into vectors, stored in a vector database. A question about "time off" will then find a passage that talks about "holidays".

What is agentic RAG?

In classic RAG, the path is fixed: one question, one search, one answer. If the search does not bring back the right information, the answer will be wrong, and the system will not notice.

Agentic RAG hands the process to an AI agent, meaning a model that can decide which actions to take and use tools. According to the survey by Singh et al. (2025), it relies on four patterns:

• planning: breaking a complex question into sub-questions;
• tool use: choosing between several sources (documentation, database, CRM, web search);
• reflection: assessing whether the information found is sufficient, and searching again if not;
• multi-agent collaboration: splitting the work between several specialised agents.

A widespread approach is the ReAct pattern (reason, then act), described by Yao et al. in 2022: the agent alternates a reasoning step and an action, observes the result, then decides what to do next.

Classic RAG or agentic RAG: comparison

Criterion	Classic RAG	Agentic RAG
Flow	Fixed: one search, one answer	Adaptive: several searches if needed
Sources	Usually a single base	Several sources and tools
Complex questions	Limited	Good
Response time	Fast	Slower (several model calls)
Cost per question	Low	Higher
Predictability	High	Lower, needs guardrails
Implementation	Simple	More demanding

Key point: agentic RAG is not a "better" version of RAG, it is a more powerful and more expensive one. It is justified when the questions call for it, not as a matter of principle.

Concrete SME use cases

Querying internal documentation

"What is the procedure for an expense claim abroad?" Classic RAG answers this kind of question very well, from the internal policy, citing the relevant passage. It is often the first project I recommend.

Preparing customer replies

The system retrieves the useful elements from the answer base, the terms and conditions and the history, and prepares a draft that the team approves before sending. Search time disappears; the decision stays human.

Cross-checking several sources

"Does this client have overdue invoices, and what does their contract say about late payment?" This is where agentic RAG comes into its own: the agent queries the accounting system, then the contract, then composes an answer from both.

Analysing a file

For a fiduciary firm or a practice, an agent can go through a complete file, extract the points that need attention and flag missing documents. It is a demanding use case that requires systematic review by a professional.

How to set up RAG with n8n

n8n lets you build RAG without heavy development (if you are still undecided about the tool, see my Make vs n8n vs Zapier comparison). Its official documentation describes the approach, and workflow templates are available. The main building blocks:

1. An ingestion workflow: it fetches the documents (shared folder, Google Drive, SharePoint), splits them into chunks and stores them in a vector database such as Qdrant, Pinecone or PGVector.
2. An AI agent node connected to a language model.
3. The vector store plugged in as a tool of the agent: the agent decides when to query it.
4. Other tools if needed: a query to the database, the CRM or an API.

Hosted in Switzerland, n8n keeps control of the orchestration. But documents also pass through the language model and, depending on the configuration, through the vector database service: these choices are part of the design, especially with data subject to the Swiss FADP.

Limits and risks to be aware of

• Poor-quality documents produce poor answers. Outdated versions, duplicates, contradictions: RAG doesn't fix messy documentation, it exposes it.
• Hallucinations decrease but don't disappear. The answer must cite its sources, so the user can check.
• Cost and latency grow with autonomy. An agent that chains five searches costs roughly five times as much and answers more slowly.
• Access rights must be respected. An employee must not be able to obtain, through the assistant, a document they would not be allowed to open.
• Autonomy must be supervised. An agent that can act (send an email, change data) must have limited permissions and human approval steps.

My view

RAG is currently the most useful application of AI for an SME, because it makes the most of an asset the business already has: its documentation, its procedures, its standard answers.

On the other hand, I advise against starting with agentic RAG. A classic RAG, well fed and citing its sources, covers the vast majority of needs, at a much lower cost and complexity. I only add an agent when the questions genuinely require cross-checking several sources.

The real work is almost always the same: putting documents in order before handing them to an AI. It is less spectacular than the architecture, but it is what determines the quality of the answers.

An AI agent is only valuable when it is connected to your own data and tools. That is what I set up in my AI automation and AI agent projects in Geneva. For an overview, also read my guide to business automation with AI, and the AI and automation glossary (in French) for technical terms.

Useful sources

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, 2020
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, 2022
• Singh et al., Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG, 2025
• n8n Docs, RAG in n8n
• n8n Docs, AI Agent node