Skip to content
Nikhil Ghind
All work

Retrieval

Pythia — agentic RAG over 10k+ documents

A stateful LangGraph agent that routes, retrieves, reflects, and re-queries when its own context looks weak — cutting response latency 40%.

Role

Sole engineer

When

2026

Outcome

40% lower response latency

Stack

PythonLangGraphMCPPineconeFastAPIAWSLangFuse
Problem
Single-shot RAG answers confidently from whatever the first retrieval returns. When that context is thin or off-topic, the model has no mechanism to notice or recover.
Move
Modelled the pipeline as an explicit LangGraph state machine — route → retrieve → reflect → generate — with MCP tool-calling over 500k+ chunks in Pinecone and a re-query loop triggered by low-confidence context.
Result
Grounded answers with inline citations, served from FastAPI on AWS with caching and async fan-out that cut response latency 40%, and retrieval quality tracked through LangFuse evaluations.
agent state machine
LOW CONFIDENCE → RE-QUERYRouteclassify intentRetrievepinecone · 500k chunksReflectscore the contextGenerategrounded + citedAnswerinline citations
01 / 07

$ query in · router picks the retrieval strategy

  • active node
  • traversed edge
  • self-correction loop
Most RAG pipelines are a straight line: retrieve once, then generate whatever that returned. The reflect node scores its own context first, and on low confidence sends the query back to retrieve with a reformulation — so a thin first search produces a second attempt rather than a confident wrong answer.focus + ← → to step

CONTEXT

Retrieval quality, not generation quality, is usually what makes a RAG system feel unreliable. If the retrieved chunks don't contain the answer, a capable model will still produce fluent text — it just won't be grounded in anything.

The fix is to make the pipeline capable of judging its own inputs. That means an explicit state machine rather than a single chain, so there is somewhere for a reflection step to live and something for it to loop back to.

WHAT I DID

  • Built a stateful LangGraph agent with an explicit route → retrieve → reflect → generate cycle.
  • Wired MCP tool-calling over 500k+ document chunks indexed in Pinecone.
  • Added a self-correction path that re-queries when the retrieved context scores low-confidence, instead of generating from weak evidence.
  • Grounded every answer with inline citations back to source chunks.
  • Served the agent from FastAPI on AWS, adding caching and async fan-out across retrieval calls.
  • Instrumented retrieval quality and regression tracking through LangFuse evaluations.

RESULT

Caching and async fan-out cut response latency 40%, and LangFuse evaluations made retrieval regressions visible as the corpus grew rather than surfacing as user-reported bad answers.