← Back to RAG Case Study

IS373 FIELD NOTES / DELIVERABLE 2 / WORKING RAG APPLICATION

Grounded Policy Assistant

A retrieval-augmented generation (RAG) system answering campus policy questions grounded in five verified synthetic documents. Includes traceable chunk retrieval, citation highlights, and an automated evaluation suite testing ground-truth recall and unanswerable query resistance.

Ask questions grounded strictly in the 5 registered campus policy documents. Queries outside the corpus trigger an explicit insufficient evidence fallback.

Try a benchmark query:
✓ Grounded in Verified PolicyRetrieval Latency: 0 ms
Grounded Model Output:

Based on Campus Club Membership and Attendance Policy [DOC-001], Active membership in registered student organizations requires enrollment in at least 6 credit hours and payment of the standard student activities contribution. To maintain active voting standing, members must attend a minimum of three general body meetings per academic semester.

Verified Citations & Supporting Passages:
[DOC-001] Campus Club Membership and Attendance Policy (DOC-001-C01)

“Active membership in registered student organizations requires enrollment in at least 6 credit hours and payment of the standard student activities contribution. To maintain active voting standing, members must attend a minimum of three general body meetings per academic semester.”

Retrieval Trace (Top-k Chunks):
#1 DOC-001-C01 (DOC-001)Similarity Score: 0.4540
#2 DOC-001-C02 (DOC-001)Similarity Score: 0.0852
#3 DOC-004-C01 (DOC-004)Similarity Score: 0.0676

System Architecture & Verifiable Guarantees

  • Deterministic Provenance: Every claim links back to a stable chunk ID (e.g. DOC-001-C01) and parent document.
  • Hallucination Resistance: Unanswerable queries (e.g., inquiring about meeting dates omitted from the corpus) trigger an explicit insufficient-evidence fallback rather than speculative text.
  • Zero-Leak Server Isolation: Content is parsed and evaluated locally without transmitting private student or campus information to external unapproved APIs.
  • Automated Evaluation Suite: The 5-question benchmark verifies exact-term, paraphrase, relationship, and negative-case handling.