Skip to content

All lessons

Area 02 // RAGLesson 42 / 14014 min read

GraphRAG

GraphRAG builds a knowledge graph of the entities in your documents (people, teams, products) and how they relate, so the system can answer questions that need connecting facts spread across many documents, or summarising a whole collection, which plain chunk search struggles with.

Think of it like this

Analogy

Regular RAG is like searching a pile of index cards for the ones that look most like your question. GraphRAG is like a detective's wall: photos of people, places and companies, connected with string labelled "works for", "owns", "depends on". To answer "how is the suspect connected to the bank?", you follow the strings, even if no single card mentions both.

The problem it solves

Some questions can't be answered by any one chunk:

  • Multi-hop: "Who should Priya contact if the Fraud API breaks checkout?" needs: Priya manages Payments; Payments owns Checkout; Checkout depends on the Fraud API; the Fraud API is owned by Risk; Tom manages Risk. Five facts in five documents.
  • Global: "What are the main complaints across all 10,000 support tickets this quarter?" No chunk contains the answer; it's a property of the whole collection.

Similarity search retrieves chunks that look like the question. For these questions, that's not enough.

How it works, step by step

Building the graph (ingestion)

  1. Chunk documents as usual.
  2. Extract entities and relationships from each chunk with an LLM (structured output): (Priya, manages, Payments team), with a short description and the source chunk ID.
  3. Merge duplicates ("Payments", "Payments team", "the payments squad") into one node. Entity resolution is the hard part.
  4. Store the knowledge graph in a graph database (e.g. Neo4j) or simple tables, keeping links back to source chunks for citations.
  5. Find communities: cluster tightly connected nodes (e.g. with the Leiden algorithm) and have an LLM write a community summary of each.

Answering (query time)

  1. Local search for multi-hop questions: find entities mentioned in the question, walk their relationships a few hops, collect connected facts and their source chunks, and give them to the LLM.
  2. Global search for big-picture questions: map the question over community summaries (each produces a partial answer), then reduce them into one answer.
  3. Combine with normal RAG: many systems use vector search for ordinary questions and graph traversal only when needed (routing).
A multi-hop answer is a path through the graph, assembled from facts found in different documents.

See it in code

Needs
Nothing: plain Python 3.

A tiny knowledge graph from extracted triples, and a walk that answers a question no single document answers.

tiny_graphrag.py
from collections import defaultdict
 
# Facts an LLM extracted from many separate documents, as (subject, relation, object) triples.
triples = [
    ("Priya", "manages", "Payments team"),
    ("Payments team", "owns", "Checkout service"),
    ("Checkout service", "depends on", "Fraud API"),
    ("Fraud API", "is owned by", "Risk team"),
    ("Tom", "manages", "Risk team"),
]
graph = defaultdict(list)
for s, rel, o in triples:
    graph[s].append((rel, o))
 
def paths(start, depth=4, path=None):
    """Walk the graph from one entity, collecting every chain of relationships."""
    path = path or [start]
    for rel, nxt in graph[path[-1]]:
        new = path + [rel, nxt]
        yield new
        if depth > 1:
            yield from paths(start, depth - 1, new)
 
# Question: "Who should Priya talk to if the Fraud API breaks checkout?"
# No single document says this. The answer needs 4 hops across documents.
for p in paths("Priya"):
    if p[-1] == "Risk team":
        print(" -> ".join(p))
print("answer: talk to", [o for _, o in graph["Tom"]][0], "manager", "Tom")
Output
Priya -> manages -> Payments team -> owns -> Checkout service -> depends on -> Fraud API -> is owned by -> Risk team
answer: talk to Risk team manager Tom

In a real system the path and the source chunks of each fact are passed to the LLM, which writes the answer with citations. Libraries such as Microsoft's GraphRAG, LlamaIndex's property graph tools and Neo4j's GraphRAG package automate extraction, storage and querying.

A real example

Example

An engineering knowledge assistant

A large company's internal assistant answered "how do I deploy service X?" well with normal RAG, but failed at "if the auth service goes down, which customer-facing products break and who owns them?". That answer was spread across service catalogues, architecture docs and team pages.

They built a graph of services, teams, owners and dependencies, extracted mostly from structured sources (the service catalogue) plus LLM extraction from docs, and routed dependency and ownership questions to graph traversal. Incident responders got correct blast-radius answers with links to every source. They kept normal RAG for everything else, because the graph took real effort to keep accurate.

The easy way

Start here

Before building GraphRAG, check whether your questions really need it: collect 50 real questions and mark which need multiple documents connected. If relationships already exist in a structured system (HR directory, service catalogue, CRM), query that system with a tool instead of extracting a graph from text.

When you grow: the production setup

Production GraphRAG needs an extraction schema (allowed entity and relation types), entity resolution with human review for important entities, incremental graph updates as documents change, provenance on every edge for citations, routing between graph and vector retrieval, and evals for multi-hop accuracy. Budget for the LLM cost of extraction across the whole corpus.

Trade-offs

What you gain

  • Answers multi-hop questions that chunk search can't.
  • Global summaries answer "what are the themes?" questions over huge collections.
  • Explicit relationships are explainable and easy to cite.

What it costs you

  • Expensive extraction (LLM calls over the whole corpus) and harder updates.
  • Entity resolution errors create wrong connections.
  • More infrastructure (graph store, community detection, routing).

When not to use it

  • Most questions are answered by one or two chunks: hybrid search plus reranking is cheaper and simpler.
  • The relationships already live in a database you can query directly.

Common mistakes

  • Building a graph before confirming you have multi-hop or global questions.
  • Skipping entity resolution, so "ACME", "Acme Corp" and "ACME Ltd" become three nodes.
  • Losing links from graph facts back to source documents.
  • Rebuilding the whole graph on every document change.
  • Using GraphRAG for every question instead of routing.

Interview questions

Try answering out loud first, then open the model answer.

What is GraphRAG and when would you use it?

Model answer

GraphRAG builds a knowledge graph from documents by LLM-extracting entities and relations, optionally clustering it into communities with LLM summaries. Local search traverses from query entities to gather connected facts for multi-hop questions; global search answers corpus-wide questions from community summaries. I'd use it when a meaningful share of questions require connecting information across documents or summarising whole collections, and the extraction cost is justified.

What are the main engineering challenges of GraphRAG?

Model answer

Extraction quality and cost over large corpora, entity resolution and deduplication, schema design for entity and relation types, incremental updates when documents change, maintaining provenance for citations, scaling graph storage and traversal, routing between graph and vector retrieval, and evaluating multi-hop correctness.

How does GraphRAG differ from agentic RAG for multi-hop questions?

Model answer

GraphRAG precomputes relationships at ingestion so multi-hop queries become fast traversals, at a high upfront cost. Agentic RAG discovers the chain at query time by iteratively searching and reading, costing more per question but needing no graph. Agentic RAG is more flexible; GraphRAG is faster and more consistent for frequent relationship-heavy queries.

Quiz

Pick an answer to see whether it’s right and why.

  1. 01What kind of question is GraphRAG best at?
  2. 02How is the knowledge graph usually built?
  3. 03Why does plain vector search fail on "Who should Priya talk to if the Fraud API breaks checkout?"
  4. 04What is a community summary in GraphRAG?
  5. 05What's the biggest downside of GraphRAG?

0 of 5 answered

Key terms

GraphRAG
RAG over a knowledge graph of entities and relationships extracted from documents, for multi-hop and corpus-wide questions.
Knowledge graph
A network of entities (people, products, teams) connected by labelled relationships.
Entity
A specific thing named in text, such as a person, company, product or place.
Multi-hop question
A question whose answer needs several facts chained together, often from different documents.
Community summary
In GraphRAG, an LLM-written summary of a cluster of closely connected entities, used for big-picture questions.

Before and after

Read first

Read next

Progress is saved in this browser only.