Document Question Answering
Your first retrieval project: ask questions about a set of documents and get answers grounded in them, with citations.
Technologies
The brief
What the finished thing needs to do.
- Ingest a set of documents and make them searchable by meaning
- Answer a question using only content retrieved from those documents
- Cite which document and section each answer came from
- State clearly when the documents do not contain the answer
- Handle a question that is unrelated to the document set
How to structure it
Two distinct phases. Ingestion — read documents, split them into chunks, generate embeddings, store them — happens once per document. Querying happens per question: embed the question, retrieve similar chunks, and pass them to the model as context.
The model must be instructed to answer only from the supplied context and to say when it cannot. Without that constraint it will answer from general knowledge, which defeats the purpose and produces confident wrong answers.
Chunking strategy has more effect on answer quality than prompt wording. Chunks that split mid-sentence or lose their heading context retrieve poorly.
Citations are not a nice extra. They are what makes the output verifiable, and without them a user has no way to check an answer.
Why these technologies
- Embeddings plus a vector store
- Finds relevant passages by meaning rather than exact keyword match.
- Retrieval before generation
- The model answers from supplied text, so answers are grounded and checkable.
- Explicit refusal instruction
- Without it the model fills gaps from general knowledge and sounds equally confident doing so.
- Chunk-level citations
- Makes every answer traceable to its source.
Build it in this order
Each stage produces something that works. That matters — a project that only runs at the very end is a project people abandon.
Load and chunk documents
Read files and split them sensibly. Inspect the chunks before going further.
Generate and store embeddings
Embed each chunk and store it with its source reference.
Retrieve by similarity
Embed a question and return the closest chunks. Check the results are actually relevant.
Generate an answer
Pass retrieved chunks as context with an instruction to use only them.
Add citations
Return which chunk and document each part of the answer came from.
Handle no answer
Confirm it declines when the documents do not cover the question.
Improve chunking
Adjust size and overlap, and measure whether retrieval quality improves.
Done means
How to know it is finished
Check each of these before moving on. If one fails, the project is not done yet — and that is useful information rather than a setback.
- Answers are grounded in the documents, not general knowledge
- Every answer cites its sources
- Questions the documents do not cover produce an explicit acknowledgement rather than a guess
- Retrieved chunks are genuinely relevant to the question asked
If you want to go further
Extensions worth attempting
Only once the core build meets every criterion above.
- Add hybrid search combining keyword and semantic retrieval
- Add reranking of retrieved results
- Support follow-up questions with conversation context
- Add an evaluation set and measure answer quality as you change things
Other projects at this level
Employee Management Console Application
The C# course project. Build it with what you know early, then deliberately refactor it as the language gives you better tools.
Employee Management API
A production-shaped web API: validated, paged, documented, secured, backed by a real database and covered by tests.
Blog API
A smaller API focused on relationships, slugs, publishing state and search.
Have a project worth talking through?
Tell us what you're building or what's slowing your current system down. We'll give you a direct read on scope and approach.