Best RAG Agent Frameworks in 2026

Quick picks

Every framework here is free to use under a permissive licence, so the table compares fit, not price. Facts were checked against each project's official documentation and repository in November 2026.

What a RAG agent framework actually does

A RAG agent is a model in a loop that decides when to search, what to search for and whether the results are good enough to answer. A framework can help in two places, and most are much stronger in one than the other.

• Orchestration. The loop itself, state that survives a crash or a pause, streaming, human approval, handoffs between agents. Frameworks model this as a graph, as event-driven workflows, as roles in a team, or as a plain loop.
• Retrieval. Loaders, chunking, embeddings, vector stores, keyword and hybrid search, reranking. Some frameworks ship all of it. Others ship none and expect a function called something like search_docs.

The LangChain documentation describes the trade-off you are buying into. A fixed two-step pipeline (always retrieve, then generate) is simple and predictable, with high control and fast responses. Agentic RAG, where the model decides when and how to retrieve, gives high flexibility, lower control and variable latency. Every framework below can build both. They differ in how much of the loop you see and can change.

Comparison table

Two names are missing on purpose. As of November 2026 the AutoGen repository states that the project has been archived, is no longer actively maintained by Microsoft, and that new users should start with Microsoft Agent Framework. The Semantic Kernel repository opens with a notice that Semantic Kernel is now Microsoft Agent Framework. Both are covered under that entry.

The frameworks in depth

LangGraph: best for full control over the retrieval loop

How it models agents: the LangGraph documentation calls it a low-level orchestration framework and runtime for long-running, stateful agents. You define a graph of shared state, nodes and edges, and can mix fixed steps with model-driven ones.

For retrieval: LangGraph has no retrieval layer of its own. That comes from LangChain. The official agentic RAG tutorial shows why the graph matters. The model decides whether to call a retriever tool, a grading step checks whether the documents are relevant, and a rewrite step reformulates the question and loops back when they are not. For the LangChain side in detail, see RAG with LangChain.

Status signals: MIT licensed. The docs describe LangGraph v1 as a stability-focused release that keeps the core graph APIs unchanged. The runtime offers checkpointing with resume after failures or pauses, human-in-the-loop, memory and streaming. The repository README names Klarna, Replit and Elastic among its users. One thing to know: the older create_react_agent prebuilt is deprecated in favor of LangChain's create_agent.

Choose it when you already know the loop you want and need to own every step.

LlamaIndex: best when retrieval quality is the hard part

How it models agents: the documentation describes the framework as a Python toolkit for building LLM-powered agents over your data. Agents come as FunctionAgent (native tool calling), ReActAgent and CodeActAgent. Workflows are event-driven, multi-step processes with branching, retries and human review. For multi-agent work the docs give three patterns: AgentWorkflow, where agents declare who they can hand off to; an orchestrator agent that calls sub-agents as tools; and a custom planner you write as a workflow.

For retrieval: this is where it started, and the docs still call it their original open-source toolkit for RAG and agents. You get the full chain as separate parts: nodes (chunks of your source documents), retrievers, routers that pick a retriever per query, node postprocessors for filtering and reranking, and response synthesizers. The repository lists over 300 integration packages.

Status signals: MIT licensed. The same team offers LlamaParse, an enterprise platform for document OCR, parsing and extraction, and the repository README now opens with it.

Choose it when your documents are messy, your sources are many, and you expect to spend most of your effort on indexing and retrieval strategy.

Haystack: best for inspectable pipelines plus agents

How it models agents: Haystack separates two things clearly. A pipeline, according to its docs, is a graph you wire yourself, so it runs the same way every time. An agent hands step-by-step control to the model. The Agent component loops over tool calls until an exit condition is met, keeps typed state that tools can read and write, and accepts components, whole pipelines and MCP servers as tools. Wrapping an agent as a tool gives you multi-agent systems.

For retrieval: strong. Retrievers are tied to document stores and come in keyword (BM25), dense embedding and sparse embedding types, and the docs describe combining sparse and dense retrievers in one pipeline to use the strengths of both, which is how you get hybrid search. A useful pattern: test a retrieval pipeline on its own, then hand it to the agent as a tool.

Status signals: Apache 2.0 licensed, Python. The README says Haystack is used by thousands of teams and lists organizations including Airbus, Netflix and the European Commission. The README also offers a Haystack Enterprise Platform and expert support from the Haystack team.

Choose it when you want deterministic retrieval you can inspect, with agent behavior only where it pays.

CrewAI: best for role-based multi-agent work, fast

How it models agents: two modes. Crews are teams of role-based agents working through tasks with a high degree of autonomy. Flows are event-driven workflows built with @start, @listen and @router decorators, with shared state, persistence (SQLite by default) and human feedback steps. A Flow can run several Crews and pass results between them.

For retrieval: the Knowledge feature loads sources (text, PDF, CSV, Excel, JSON) into a vector store and makes them available to one agent or to the whole crew. According to the docs, the default store is ChromaDB with Qdrant also supported, and the agent's task prompt is rewritten into a search query automatically. Check one default before you ship: knowledge storage uses OpenAI's text-embedding-3-small unless you configure another embedder, even when your LLM is from a different provider.

Status signals: MIT licensed, requires Python 3.10 to 3.13. Besides Knowledge, the docs describe a separate RAG client for direct vector store control or custom retrieval pipelines.

Choose it when the problem reads naturally as roles and tasks, and you want a working prototype this week. Skip it when you need fine control over chunking, hybrid search and reranking: Knowledge is convenient, not deep. Our guide to multi-agent RAG covers when several agents are worth their cost at all.

Microsoft Agent Framework: best for .NET teams

How it models agents: Microsoft's documentation calls it the direct successor to Semantic Kernel and AutoGen, built by the same teams. It combines AutoGen's agent abstractions with Semantic Kernel's session state, middleware and telemetry, and adds functional and graph-based workflows for explicit multi-agent paths. Workflows support checkpointing, streaming and human-in-the-loop.

For retrieval: the RAG documentation describes two routes: context providers that add retrieved content before the model is called, and search tools the model calls on demand. In .NET, TextSearchProvider does either, over any search function you supply. In Python, create_vector_search_tool() turns a vector store collection into a tool with vector or keyword-hybrid search and filters the model can set. Native Python stores are listed for in-memory, Azure AI Search, PostgreSQL with pgvector, Qdrant and Redis.

Status signals: MIT licensed. The Semantic Kernel repository states that Agent Framework is available at version 1.0 as a production-ready release with stable APIs. .NET and Python are the core SDKs; the Go SDK is in public preview. OpenTelemetry observability is built in.

Choose it when your product is .NET, you run on Azure, or you are migrating from AutoGen or Semantic Kernel (Microsoft publishes guides for both).

Pydantic AI: best for typed Python with minimal abstraction

How it models agents: the docs call it a typed, extensible agent loop, typed end to end across structured outputs, dependency injection and tools. When one loop is not enough, Pydantic Graph adds typed control flow. Durable execution and an evaluation library, Pydantic Evals, are part of the family.

For retrieval: almost nothing, by design. The official RAG example registers a plain function as a tool; inside, it embeds the query and runs a pgvector SQL query written by hand.

Status signals: MIT licensed, Python, maintained by the team behind Pydantic. For tracing it pairs with Logfire, the same company's observability platform.

Choose it when your team values type safety and already has, or wants to write, its own retrieval code.

OpenAI Agents SDK: best for teams already on OpenAI's platform

How it models agents: deliberately few primitives. Agents are models with instructions and tools; handoffs and agents-as-tools let one agent delegate to another; guardrails validate inputs and outputs; sessions manage conversation history across runs. Tracing is built in.

For retrieval: FileSearchTool retrieves from OpenAI Vector Stores. It is a hosted tool, and the docs tie hosted tools to OpenAI's Responses models. Anything else is a function tool you write.

Status signals: MIT licensed, with Python and JavaScript/TypeScript SDKs. The README says the SDK supports OpenAI's APIs and more than 100 other LLMs, so the loop is portable. Hosted file search is not: your index lives with OpenAI.

Choose it when you are committed to OpenAI models and want retrieval working in an afternoon.

Google ADK: best for Google Cloud and for Java, Go or Kotlin

How it models agents: agents built from prompts and tools, template workflows (sequential, loop, parallel) and a graph-based workflow runtime with routing, retries and timeouts.

For retrieval: the ADK documentation groups it under grounding: Google Search for live web information, grounding on your organization's private documents and enterprise data through search data stores, and agentic RAG where the agent builds queries and filters itself.

Status signals: Apache 2.0 licensed. Available in Python, TypeScript, Go, Java and Kotlin, the widest language coverage here. The README describes ADK as model-agnostic but optimized for Gemini.

Choose it when you build on Google Cloud and Gemini, or your backend is on the JVM or Go.

Built by Gilzor

Results we’ve shipped

70+products launched

98%delivered on time

85%clients come back

5.0 ★on Clutch

Aviation33%faster baggage weigh-inWeb and Mobile application for Airlines→E-commerce · AITaking an AI-built e-commerce SEO platform to production→Healthcare+30%lab test ordersMedical Lab Web and Native Mobile Applications→

Talk to the people who build it. Tell us about your project and get a free estimate of scope, timeline and cost.

See how we’d approach yours →

How to choose

Control vs speed

Role-based and prebuilt agents get you a demo fastest. Graphs and workflows take longer and pay back when something goes wrong, because each step can be tested and replaced. If you can already draw the loop on a whiteboard, pick a framework that lets you encode that drawing. If you cannot, prototype with a prebuilt agent and watch what it does.

Single agent vs multi-agent

Every framework here supports multiple agents. That is not a reason to use them: one agent with a tool per source costs less and is easier to debug.

Observability

You will debug by reading traces: which query the agent wrote, what came back, what it did next. Microsoft Agent Framework emits OpenTelemetry. LangGraph pairs with LangSmith, Pydantic AI with Logfire, and the OpenAI Agents SDK has its own tracing.

Lock-in

With permissive licences everywhere, the lock-in is technical. It sits in the state and checkpoint format, the retrieval abstractions, and hosted services such as vendor vector stores. The cheapest protection is in the diagram above: keep retrieval behind one function you own, returning passages with source ids. Then swapping the framework does not mean rebuilding the index.

A one-week bake-off

1. Write 20 to 30 real questions with expected answersInclude some that need two sources and some with no answer in your data.
2. Build retrieval once, outside any frameworkQuery and filters in, passages with source ids out. Measure it on its own first.
3. Shortlist two frameworks by team fitLanguage, cloud and how much control you need.
4. Build the same single agent in bothSame model, same retrieval function, same instructions.
5. Compare on what you will live withAnswer quality, tokens and seconds per question, how readable the traces are, and how much code you wrote to work around the framework.

When to build custom instead

Microsoft's own Agent Framework documentation says it plainly: if you can write a function to handle the task, do that instead of using an AI agent. The same logic applies one level up: if plain code can run the loop, you may not need a framework.

A single RAG agent is small. Model APIs handle tool calling directly, so the loop is: send the messages and tool definitions, run the tool calls the model asks for, append the results, repeat until it answers or hits a step limit. Add a retrieval library for the tool and structured logging for traces. As a rough guide, that is a few hundred lines you fully understand.

A middle path is common: write retrieval yourself, because it is where quality is won, and take a thin loop or a graph runtime for orchestration. If you would rather not staff that work in-house, a team that offers RAG development services can build either version; ask them to justify the framework choice against a plain-code baseline.

FAQ

Where Gilzor fits

A framework choice is a small decision that gets expensive when it is made before anyone has measured retrieval. We build custom RAG systems and RAG applications, add RAG to existing products, and consult on and evaluate RAG systems. If you are weighing the frameworks above against plain code, that is a question we can help you settle on your own data. Our AI work includes a payroll platform that extracts data from PDFs and cut salary calculation errors by 17%. Gilzor has launched 70+ products and works from Poland with partial overlap with the US working day. You can start with a 2-week fixed-price Discovery Sprint.