HippoRAG and RAPTOR: Hierarchical and Memory-Style RAG

Standard RAG — embed query, cosine-search a flat vector store, stuff top-k chunks into the prompt — fails reliably on questions like "Which paper introduced the architecture used by the model that beat GPT-4 on MATH?" The answer requires chaining through at least two facts that live in separate chunks, and neither chunk scores high enough individually against the query embedding to make the top-k.

This is the multi-hop retrieval gap: flat retrieval is great at "find the one passage that contains the answer" but poor at "find and join multiple passages whose combination contains the answer." Two architectural families address this gap in different ways:

• HippoRAG builds a biological-memory-inspired graph index so the retriever can propagate relevance across related passages like the human hippocampus does.
• RAPTOR builds a tree of progressively coarser summaries so a single embedding lookup can match at whatever level of abstraction the question needs.

Both were introduced in 2024 and are worth understanding together because they solve the same problem from opposite ends.

HippoRAG (Guo et al., Stanford, 2024) draws explicitly on the Complementary Learning Systems (CLS) theory from neuroscience. In CLS, the hippocampus stores episodic memories as sparse, pattern-separated indexes, while the neocortex stores slow-learned semantic knowledge. Crucially, hippocampal retrieval works by spreading activation: cues trigger nearby memory nodes, which trigger their neighbors, surfacing related memories even when the cue doesn't directly match them.

The paper maps this onto RAG as follows:

This mapping isn't just metaphor — it directly drives the retrieval algorithm.

During indexing, an LLM (GPT-3.5-Turbo in the paper) reads each chunk and extracts OpenIE-style triples: (subject, relation, object). Subjects and objects become entity nodes; chunks become passage nodes connected to every entity they mention.

# Pseudocode matching HippoRAG's offline indexing step
from openai import OpenAI
import networkx as nx

client = OpenAI()
G = nx.Graph()

def extract_triples(chunk: str) -> list[tuple]:
    resp = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user",
            "content": f"Extract (subject, relation, object) triples.\n{chunk}"}]
    )
    # parse structured response into list of tuples
    return parse_triples(resp.choices[0].message.content)

for chunk_id, chunk in enumerate(corpus):
    triples = extract_triples(chunk)
    G.add_node(f"passage:{chunk_id}", text=chunk)
    for subj, rel, obj in triples:
        G.add_node(subj)
        G.add_node(obj)
        G.add_edge(f"passage:{chunk_id}", subj)
        G.add_edge(f"passage:{chunk_id}", obj)
        G.add_edge(subj, obj, relation=rel)

Edge weights between entity nodes are boosted by embedding similarity so that near-synonyms ("New York City" / "NYC") end up tightly coupled in the graph.

At query time, the system embeds the query and finds the top-scoring entity nodes (not passage nodes) by cosine similarity. These become the seed set for Personalized PageRank (PPR).

PPR is a variant of standard PageRank where the random surfer teleports back to the seed nodes instead of uniformly to the whole graph. This causes relevance to diffuse outward from the seeds through shared entities — exactly the spreading-activation behavior described in CLS theory.

import networkx as nx

def hipporag_retrieve(query_embedding, G, entity_embeddings, top_k=5):
    # 1. Find seed entity nodes
    scores = cosine_similarity([query_embedding], entity_embeddings)[0]
    seed_nodes = [entities[i] for i in scores.argsort()[-5:][::-1]]
    personalization = {n: (1/len(seed_nodes) if n in seed_nodes else 0)
                       for n in G.nodes()}

    # 2. Run PPR
    ppr_scores = nx.pagerank(G, alpha=0.85,
                              personalization=personalization)

    # 3. Rank passage nodes
    passage_scores = {n: s for n, s in ppr_scores.items()
                      if n.startswith("passage:")}
    return sorted(passage_scores, key=passage_scores.get, reverse=True)[:top_k]

Key insight: PPR naturally handles multi-hop — if chunk A mentions entity X and chunk B mentions both X and Y, and the query is about Y, PPR will still surface chunk A because X serves as a bridge.

RAPTOR (Sarthi et al., Stanford, 2024) attacks the multi-hop problem differently: instead of enriching the index with a knowledge graph, it pre-computes summaries at multiple granularities and indexes all levels together.

The algorithm:

1. Embed all leaf chunks and cluster them with Gaussian Mixture Models (soft clustering, so a chunk can belong to multiple clusters).
2. Use an LLM to write a summary for each cluster.
3. Embed those summaries, cluster them again, summarize again.
4. Repeat until one root summary covers the whole corpus.

At query time, you can either:

• Tree traversal: start at root, greedily descend to the most relevant child at each level (top-down).
• Collapsed retrieval: flatten all nodes (leaf chunks + all summaries) into one vector store and do a single FAISS search (the approach that scores best in the paper).

The collapsed approach is surprisingly powerful: a vague multi-hop question often matches a mid-level summary better than any individual leaf chunk.

Below is a condensed version of RAPTOR's core clustering-and-summarization loop, close to the open-source release:

from sklearn.mixture import GaussianMixture
import numpy as np

def raptor_build_tree(chunks: list[str], embed_fn, summarize_fn,
                      max_levels=3, n_components=10):
    all_nodes = []  # will hold (text, embedding, level)
    texts = chunks

    for level in range(max_levels):
        embeddings = np.array([embed_fn(t) for t in texts])
        if len(texts) <= n_components:
            break  # stop when too few nodes to cluster

        gm = GaussianMixture(n_components=n_components,
                             covariance_type="full").fit(embeddings)
        labels = gm.predict(embeddings)  # hard assignment for simplicity

        all_nodes.extend(zip(texts, embeddings, [level]*len(texts)))

        # Summarize each cluster
        summaries = []
        for cluster_id in range(n_components):
            members = [t for t, l in zip(texts, labels) if l == cluster_id]
            if members:
                summaries.append(summarize_fn(members))
        texts = summaries  # next level's inputs

    all_nodes.extend(zip(texts, [embed_fn(t) for t in texts],
                         [max_levels]*len(texts)))
    return all_nodes  # index all nodes together

The real RAPTOR uses UMAP to reduce embedding dimensionality before GMM fitting, which dramatically improves cluster quality in high-dimensional spaces.

Consider a corpus about pharmaceutical research. A multi-hop question — "Which drug approved in 2023 targets the same receptor as the compound described in the 2019 trial?" — requires:

1. Knowing which receptor the 2019 compound targets (chunk A).
2. Knowing which 2023 drug targets that receptor (chunk B).

In flat RAG, neither chunk A nor chunk B individually matches the query embedding well. But RAPTOR's level-1 summaries likely produced one node that says "Research on [receptor] spans trials in 2019–2024, including the approved drug X" — a single node that matches the full question and points downstream to both leaf chunks.

This is the core value proposition: summaries compress disparate facts into a representation whose embedding is closer to abstractly-worded queries. The more hops required, the higher up the tree the relevant summary sits, and the better it scores.

HippoRAG achieves the same outcome differently — PPR propagation connects the receptor mention in chunk A to the drug mention in chunk B via the shared entity node — but the net effect is identical: multi-hop evidence gets surfaced in one retrieval step.

Both papers report results on multi-hop benchmarks (MuSiQue, HotpotQA, 2WikiMultiHopQA):

Numbers are approximate from the respective papers; exact splits vary.

Use HippoRAG when:

• The corpus has rich entity structure (Wikipedia, biomedical, legal).
• You can afford offline LLM calls for triple extraction.
• You need interpretable retrieval (the KG is inspectable).

Use RAPTOR when:

• The corpus is long-form narrative (reports, books, documentation).
• You want a drop-in upgrade: RAPTOR's collapsed retrieval plugs into any existing FAISS-based pipeline.
• Entity extraction is noisy or poorly defined for your domain.

Gotcha: RAPTOR's summarization quality is highly sensitive to LLM choice. GPT-4-class summaries produce meaningfully better trees than GPT-3.5-class.

Nothing stops you from layering these techniques. One production pattern:

1. Extract entities and build a HippoRAG-style KG for precise entity-level retrieval.
2. Independently build RAPTOR summaries for abstract or narrative questions.
3. Route the query to one index or the other based on a lightweight classifier (or just query both and rerank).

Practical trade-offs to keep in mind:

• Indexing cost: HippoRAG runs an LLM over every chunk for triple extraction — expensive for millions of documents. RAPTOR is cheaper per chunk but the tree doubles or triples your stored nodes.
• Freshness: both require re-indexing when the corpus changes. HippoRAG's KG is easier to surgically update (add/remove nodes); RAPTOR's tree may need partial rebuilding.
• Embedding drift: summaries and leaf chunks should be embedded with the same model. Mixing text-embedding-3-small for leaves and text-embedding-3-large for summaries breaks the collapsed index search.
• Chunk size sensitivity: RAPTOR clusters benefit from uniform chunk sizes (~250–400 tokens). Very short chunks cluster poorly; very long chunks produce summaries that don't generalize.

For most teams, RAPTOR's collapsed retrieval is the pragmatic first step — it requires no KG infrastructure and delivers measurable multi-hop gains in a weekend.

Both systems have official or community implementations you can run today:

# HippoRAG (official, Python)
git clone https://github.com/OSU-NLP-Group/HippoRAG
cd HippoRAG
pip install -r requirements.txt
# Set OPENAI_API_KEY, then:
python src/index.py --dataset musique --llm gpt-3.5-turbo
python src/retrieve.py --dataset musique --llm gpt-3.5-turbo

# RAPTOR (official, Python)
git clone https://github.com/parthsarthi03/raptor
cd raptor
pip install -r requirements.txt

# RAPTOR minimal usage
from raptor import RetrievalAugmentation, RetrievalAugmentationConfig
from raptor import TreeBuilder, TreeBuilderConfig

config = RetrievalAugmentationConfig(
    tree_builder_config=TreeBuilderConfig(
        summarization_length=200,
        num_layers=3
    )
)
ra = RetrievalAugmentation(config=config)
ra.add_documents(["path/to/your/corpus/"])
answer = ra.answer_question("Your multi-hop question here")

For LangChain users: RAPTOR has a community integration under langchain_community.retrievers.raptor. HippoRAG's PPR step is not yet in LangChain but the KG construction can be wired to LangChain's graph stores (Neo4j, NetworkX).