GraphRAG: When Knowledge Graphs Beat Vector Search

What is GraphRAG?

GraphRAG is retrieval-augmented generation built around entities and relationships rather than only text similarity. A GraphRAG pipeline extracts a knowledge graph from source documents, groups related entities into communities, produces summaries, and retrieves graph context alongside supporting text.

This matters when the answer depends on connections spread across many documents. Standard vector search can retrieve passages that look similar to the question. It does not inherently preserve that one supplier owns another company, that both are named in separate incidents, and that those incidents form a broader operational pattern.

GraphRAG is not a replacement for every vector database. It is an additional retrieval strategy for relationship-heavy questions.

Where does vector RAG struggle?

Vector retrieval works well when a small number of chunks contains the answer:

• “What is the cancellation clause in this contract?”
• “How do I reset the device?”
• “What did the customer say about onboarding?”

It becomes less reliable when the question requires collection-wide synthesis:

• “Which organizations influence this market, and how?”
• “What risks recur across all regional reports?”
• “How are these people, projects, and decisions connected?”
• “Which themes appear across documents that use different terminology?”

Embedding similarity is local: it ranks candidate chunks. Global questions require aggregation, relationship traversal, or precomputed summaries that represent more than a single chunk.

How does a GraphRAG index work?

Microsoft’s reference pipeline describes a sequence that extracts entities, relationships, and optional claims; detects communities; generates community reports; and embeds selected artifacts.

What are local and global search?

The Microsoft GraphRAG query engine distinguishes several modes.

Local search begins with specific entities and combines graph relationships with relevant source text. It fits questions such as “What contracts involve this vendor, and what obligations connect them?”

Global search works over community reports, often with a map-reduce style of aggregation. It fits questions such as “What are the main systemic risks across the complete collection?”

Basic vector search remains useful as a baseline and for direct fact retrieval. A mature system can route among these strategies rather than force every question through the most expensive path.

When is GraphRAG worth the complexity?

Use it when the information domain has durable entities and meaningful relationships: investigations, supply chains, scientific literature, corporate intelligence, legal matters, cyber incidents, or complex project histories.

It is less compelling when:

• the corpus is small;
• most questions are direct lookups;
• entities change too rapidly to normalize;
• source documents lack reliable identifiers;
• the team cannot maintain provenance and access controls;
• a reranker or better chunking already solves the measured failure.

Start from failed questions. If the failure is caused by missing relationships or global aggregation, GraphRAG may fit. If retrieval simply returned the wrong passage, improve the conventional pipeline first.

How do you preserve provenance and security?

Every entity, relationship, claim, and community summary should point back to source evidence. A graph edge without provenance can turn extraction error into authoritative-looking structure.

Carry document permissions into graph artifacts as well. If two departments can see different source documents, they must not receive a shared community summary that leaks the restricted content. Apply access filtering before aggregation or maintain policy-specific indexes.

How should GraphRAG be evaluated?

Build an evaluation set with at least three categories:

1. direct fact questions that vector retrieval should already answer;
2. local relationship questions requiring multiple entities or hops;
3. global questions requiring themes or aggregation across the collection.

Compare answer correctness, source support, retrieval coverage, latency, and index cost. Include abstention cases where the corpus does not support an answer. Evaluate graph extraction separately from final generation; otherwise a polished answer can hide a broken entity or relationship.

The baseline matters. Microsoft’s repository explicitly warns that GraphRAG indexing can consume significant model resources. A graph approach should earn its cost by improving a measured class of questions, not by sounding architecturally advanced.

A practical adoption path

Begin with a representative subset of the corpus. Normalize a small number of high-value entity types. Keep the original text index. Add local graph search for a known multi-hop failure, then test global search on a defined synthesis task. Route simple questions to vector retrieval and reserve graph-heavy paths for queries that need them.

GraphRAG succeeds when it makes hidden relationships inspectable and supportable. It fails when the graph becomes another opaque layer between the user and the evidence.

Sources and further reading

• Microsoft Research: GraphRAG publications
• Microsoft GraphRAG repository
• GraphRAG indexing overview
• GraphRAG query engine overview