GraphRAG: Knowledge-Graph Augmented Retrieval

Standard RAG works by embedding chunks of text, storing them in a vector index, and retrieving the top-k closest chunks at query time. For narrow, factual look-ups — "what is the capital of France?" — this is fine. It breaks down for global, sensemaking queries like "what are the main themes across this entire corpus?" or "how are these companies connected?"

The problem is structural: embedding similarity finds locally relevant passages but cannot synthesize information scattered across thousands of documents. A chunk about Company A's CEO and a chunk about Company B's acquisition sit at different addresses in embedding space; no single query pulls both into context simultaneously.

Microsoft Research's GraphRAG (Edge et al., 2024) replaces the flat chunk index with a knowledge graph + community hierarchy, enabling genuine cross-document reasoning without asking the LLM to read the whole corpus.

The pipeline has two distinct phases: indexing (top half) builds the graph and community summaries offline; querying (bottom half) routes to global or local search depending on query type.

The indexer passes each text chunk to an LLM with a prompt that asks it to identify entities (people, organizations, events, concepts) and relationships between them, returning structured JSON.

A simplified extraction prompt result looks like:

{
  "entities": [
    {"name": "OpenAI", "type": "ORGANIZATION", "description": "AI research lab"},
    {"name": "Sam Altman", "type": "PERSON", "description": "CEO of OpenAI"}
  ],
  "relationships": [
    {"source": "Sam Altman", "target": "OpenAI", "description": "CEO of", "weight": 9}
  ]
}

The same entity can appear in hundreds of chunks. GraphRAG merges duplicate mentions by name-normalisation and co-reference, accumulating a richer description over occurrences. Relationship weights are summed across chunks, so edges with many supporting passages end up with higher weight — a signal of evidential strength, not just semantic similarity.

After extraction the merged entities become nodes and the relationships become weighted, directed edges. The result is a heterogeneous property graph stored (in the reference implementation) as a set of Parquet files — nodes.parquet, edges.parquet — alongside the raw text units.

Graph stats for a 1 M-token corpus are typically:

• ~5 000–20 000 entity nodes
• ~15 000–60 000 relationship edges
• Average degree: 6–10 edges per node

This scale is still tractable for graph algorithms, but far too large to summarise naively. That's where clustering enters.

Heads up: Entity extraction quality is the biggest quality lever in the whole pipeline. Prompt-engineering the extraction step (adding domain-specific entity types, giving few-shot examples) yields more improvement than tuning downstream parameters.

GraphRAG partitions the knowledge graph using the Leiden algorithm (Traag et al., 2019), a community-detection method that optimises modularity faster and with better quality guarantees than the older Louvain algorithm.

Leiden is run at multiple resolution levels, producing a hierarchy:

Each node belongs to exactly one community per level. The hierarchy lets the system answer queries at the right granularity — broad questions get routed to level-0 summaries; specific questions can go deeper.

In code, the reference implementation calls graspologic under the hood:

from graspologic.partition import hierarchical_leiden

communities = hierarchical_leiden(
    graph,          # networkx Graph with edge weights
    max_cluster_size=10,
    random_seed=42
)

For each community at each level, GraphRAG collects all the entities and relationships in that community, formats them as a structured context block, and asks an LLM to write a community report: a multi-paragraph summary covering key entities, their roles, notable relationships, and any claims or events.

The prompt structure (abridged) is:

You are given a knowledge graph community.
Entities: OpenAI (AI research lab), Sam Altman (CEO), ...
Relationships: Sam Altman --CEO of--> OpenAI (weight: 9), ...

Write a concise report covering the main themes,
key entities, and notable findings. Max 200 words.

With a 1 M-token corpus and ~500 level-2 communities, this phase makes ~500 LLM calls. That's the main indexing cost. The payoff: you now have a compressed, human-readable summary for every sub-graph — a semantic map of the corpus that no vector index can provide.

Global search targets queries that need synthesis across the whole corpus: "What are the recurring concerns about AI safety in these documents?"

The process:

1. Collect all community reports at a chosen level (usually level 1 or 2).
2. Shuffle and batch them into context windows.
3. Send each batch to the LLM asking: "Here are community reports. What is relevant to the query? Return a rated partial answer."
4. Aggregate rated partial answers, weighted by relevance score, into a final answer.

This is a map-reduce pattern over community summaries. It can process millions of tokens of source material because the summaries are already compressed. The critical parameter is response_type — setting it to "multiple paragraphs" or "single paragraph" shapes the aggregation.

Heads up: Global search is slow (5–30 s, many LLM calls) and expensive. Use it only for questions that genuinely require cross-document synthesis. Routing matters.

Local search targets questions about specific entities or events: "What did Sam Altman say about AGI timelines?"

It works more like augmented vector RAG:

1. Embed the query and retrieve the top-k most relevant entities from the entity store.
2. Follow graph edges to collect related entities, relationships, and co-membered community reports.
3. Retrieve the original text units (chunks) linked to those entities.
4. Pack entities + relationships + community summaries + raw text into the LLM context.

The graph traversal is what distinguishes this from vanilla RAG. If the query mentions Company A, local search also surfaces Company A's partners, subsidiaries, and the events they co-appear in — context that embedding search would miss unless those chunks happened to score high similarity.

Typical latency: 2–8 s, ~3–6 LLM calls.

Routing can be rule-based (keywords like "overall", "across all", "themes" → global) or LLM-classified. Microsoft's reference implementation defaults to local search and requires explicit configuration for global.

The Microsoft reference implementation is a Python package:

pip install graphrag

# 1. Initialise a project
graphrag init --root ./my-project

# 2. Drop your .txt/.pdf files into ./my-project/input/

# 3. Run the indexing pipeline (builds graph + summaries)
graphrag index --root ./my-project

# 4. Query
graphrag query \
  --root ./my-project \
  --method global \
  --query "What are the main themes in these documents?"

Key config levers in settings.yaml:

• entity_extraction.max_gleanings — how many extraction passes per chunk (default 1; raise to 2–3 for dense technical text)
• community_reports.max_length — token budget per community report
• leiden.max_cluster_size — trades community granularity vs. summary cost
• local_search.text_unit_prop — fraction of context window reserved for raw text chunks vs. graph context

GraphRAG is not a drop-in replacement for every RAG workload. The trade-offs are stark:

GraphRAG earns its cost when:

• The corpus is stable (reports, research papers, legal documents) not a live feed.
• Queries are thematic or relational rather than point look-ups.
• You need explainable provenance — the community report links back to source entities and text units.

For most production chatbots over a knowledge base, a hybrid is optimal: use local GraphRAG search for entity questions, fall back to vector search for narrow factual queries.

Things that bite teams in production:

• Extraction hallucinations: LLMs sometimes invent entity names or misattribute relationships. Validate with a stricter extraction prompt or post-process with a second pass that checks entity names against a dictionary.
• Entity deduplication failures: "OpenAI", "Open AI", and "OPENAI" end up as three separate nodes unless you normalise before merge. The reference implementation uses case-folding; you may need fuzzy matching for messy real-world text.
• Community report token bloat: A very dense community (high-degree hub node) can generate a context window overflow. Cap with max_input_tokens and filter to top-N entities by degree.
• Re-indexing cost: Even a small corpus update triggers full re-extraction for changed chunks and potentially re-clustering. Plan for incremental indexing (only re-run changed chunks) if content updates frequently.
• Prompt injection via document content: If documents are untrusted, the extraction prompt can be hijacked. Wrap document content in XML delimiters and instruct the model to treat anything inside as data, not instructions.