From 8d98d19e7e23a0f246022fef36f3f6672232e2f6 Mon Sep 17 00:00:00 2001 From: KS Jannette Date: Sat, 9 May 2026 18:15:00 -0400 Subject: [PATCH] update README.md --- README.md | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index d99f6e3..c507925 100644 --- a/README.md +++ b/README.md @@ -42,15 +42,24 @@ This application is a RAG (Retrieval-Augmented Generation) system that allows us When a user submits a query, the system enforces groundedness through a multi-layered strategy: -1. Retrieval constraint — The query is embedded (also via voyage-3) and compared against stored chunk vectors using cosine similarity, returning the top 20 candidates. Only user-supplied source material is searched; the system has no web search capability. - -3. Reranking for precision — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder, which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive. - -5. Prompt-level constraint — The top 5 chunks are passed to the Primary LLM (Claude claude-opus-4-6) with an explicit system instruction: "Answer the user's question using ONLY the source documents provided below." The LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are insufficient. - -7. Schema enforcement — The LLM's response is constrained to a JSON schema requiring structured fields (answer, citedSourceIndices, followUpQuestions), and any cited source indices that don't correspond to real source groups are programmatically stripped out. - -9. Post-generation groundedness scoring — After the answer is generated, it is split into individual sentences, each sentence is embedded via voyage-3, and each sentence embedding is compared (cosine similarity) against the vectors of the cited chunks. The raw similarities are calibrated to a 0–1 scale and averaged, producing a single groundedness score that is surfaced to the user as a visual indicator (green/gold/red). +1. **Retrieval constraint** — The query is embedded (via Voyage AI voyage-3) and compared against + stored chunk vectors using cosine similarity, returning the top 20 candidates. + +2. **Reranking for precision** — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder, + which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive. + +3. **Prompt-level constraint** — The top 5 chunks are passed to the "Primary LLM" (Claude opus-4-6). + The LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are + insufficient. + +4. **Schema enforcement** — The LLM's response is constrained to a JSON schema requiring structured + fields (answer, citedSourceIndices, followUpQuestions). Any cited source indices that do not + correspond to real source groups are programmatically stripped out. + +5. **Post-generation groundedness scoring** — The answer is split into individual sentences. Voyage + AI voyage-3 embeds each sentence, and compares it (using cosine similarity) against the vectors + of the cited chunks. The similarity is calibrated to a 0–1 scale and averaged, producing a single + groundedness score that is surfaced to the user with a visual indicator (green/yellow/red). ## Design