Overview
Northstar answers questions over a small internal-document corpus while keeping each answer tied to retrieved evidence. The goal was to demonstrate the complete engineering path from document ingestion to an API response a reader can verify.
Challenge
The service must refuse questions the indexed documents do not support. It also needs citations that identify the source document, page, and chunk instead of returning unsupported prose.
My role
I independently built the project end to end: ingestion, chunking, embeddings, vector storage, retrieval filters, grounded generation, citations, refusal behavior, evaluation, automated tests, and Docker packaging.
What I built
I exposed health and question-answering endpoints with FastAPI and Uvicorn, extracted PDFs with PyMuPDF, created chunks with LangChain text splitters, generated local embeddings with all-MiniLM-L6-v2, and persisted vectors in Chroma. Docker and Docker Compose package the API and its data volumes.
Architecture
PDF / TXT / Markdown → clean text → recursive chunks → local embeddings → persistent Chroma → filtered retrieval → grounded prompt → answer + citations
The default pipeline uses 700-character chunks with 100-character overlap, retrieves the nearest candidates, and rejects chunks beyond a configurable cosine-distance threshold.
Results
The delivered system demonstrates grounded retrieval, source-level citations, deterministic refusal when evidence is insufficient, pytest coverage, and a RAGAS pipeline for faithfulness, answer relevancy, context precision, and context recall.
Lessons
The key engineering lesson was to make “I do not have enough evidence” part of the architecture. Retrieval thresholds, citations, and evaluation are product behavior, not optional additions after generation works.