From bf89c7544f7a49a06ba618c59d668885859d5754 Mon Sep 17 00:00:00 2001 From: KS Jannette Date: Mon, 3 Aug 2026 01:16:07 -0400 Subject: [PATCH] Enhanced readme.md --- README.md | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 0f71df7..0c725c3 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,22 @@ + + + + + + # Citation Sentinel -A source-grounded research assistant that employs cosine similarity scoring for LLM-generated query responses to provide a "groundedness" score. This is a metric in Retrieval-Augmented Generation (RAG) systems that quantifies how well an AI-generated answer is supported by retrieved context. It measures "faithfulness" to source documents, ensuring the answer is not hallucinated or pulled from the model's pre-training data. +A source-grounded research assistant that empowers users to transform an otherwise-unmanageably-large corpus of data on any topic of interest into refined, easily-digested subtopics and pose focused inquiries, to instantly receive concise, quantifiably-evaluated, accurate responses. The system also provides suggested follow-up questions further refining in user inquiries by detecting the query goals. -Users upload source documents, or provide links to online sources including audio/video (i.e. links to youtube videos). Users may then ask questions and receive answers (with inline citations) verifiably grounded in the provided information sources. +# Reliability and Understandability -Built with a React/Vite frontend and a Typescript/Node/Express backend, using Anthropic Claude for generation, OpenAI Whisper for video audio track transcription, Voyage AI for embeddings and response cosine similarity scoring (the "groundedness" score). +Citation Sentinel's Retrieval-Augmented Generation (RAG) pipeline uses two-stage pass-through to Voyage AI embedding and ranking models, which uses cosine similarity methodology to scoring to evaluate query-source embeddings in multi-dimensional vector space. This yields a "groundedness" score. + +Groundedness metrics quantify how well an AI-generated answer is supported by retrieved context. It measures "faithfulness" to source documents, ensuring the answer is not hallucinated, inaccurate or stale (pulled from the model's training data). + +# Architecture - Basic Overview + +Built with a React/Vite frontend and a Typescript/Node/Express backend, using Anthropic Claude for generation, OpenAI Whisper for video audio track transcription, Voyage AI voyage-3 for embeddings and Voyage AI rerank-2 for response cosine similarity scoring, to generate the "groundedness" score, presented as an easily-understandable red/yellow/green "badge" style tooltip in the response (see above.) ## Query pipeline @@ -50,25 +62,15 @@ The embeddings are stored in an in-memory vector store. When a user submits a query, the system enforces groundedness through a multi-layered strategy: -1. **Retrieval constraint** — The query is embedded (via Voyage AI voyage-3) and compared against - stored chunk vectors using cosine similarity, returning the top 20 candidates. +1. **Retrieval constraint** — The query is embedded (via Voyage AI voyage-3) and compared against stored chunk vectors using cosine similarity, returning the top 20 candidates. -2. **Reranking for precision** — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder, - which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive. +2. **Reranking for precision** — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder, which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive. -3. **Prompt-level constraint** — The top 5 chunks are passed to the "Primary LLM" (Claude opus-4-6) - for natural language query response. - The Primary LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are - insufficient. +3. **Prompt-level constraint** — The top 5 chunks are passed to the "Primary LLM" (Claude opus-4-6) for natural language query response. The Primary LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are insufficient. -4. **Schema enforcement** — The Primary LLM's response is constrained to a JSON schema requiring structured - fields (answer, citedSourceIndices, followUpQuestions). Any cited source indices that do not - correspond to real source groups are programmatically removed. +4. **Schema enforcement** — The Primary LLM's response is constrained to a JSON schema requiring structured fields (answer, citedSourceIndices, followUpQuestions). Any cited source indices that do not correspond to real source groups are programmatically removed. -5. **Post-generation groundedness scoring** — The answer is split into individual sentences. Voyage - AI voyage-3 embeds each sentence, and compares it (using cosine similarity) against the vectors - of the cited chunks. The similarity is calibrated to a 0–1 scale and averaged, producing a single - groundedness score that is surfaced to the user with a visual indicator (green/yellow/red). +5. **Post-generation groundedness scoring** — The answer is split into individual sentences. Voyage AI voyage-3 embeds each sentence, and compares it (using cosine similarity) against the vectors of the cited chunks. The similarity is calibrated to a 0–1 scale and averaged, producing a groundedness score surfaced to the user with a visual indicator (“badge” in green/yellow/red). ## Similarity Metrics for Semantic Understanding @@ -97,3 +99,5 @@ MIT. See [LICENSE](./LICENSE). ## Author @sjdev + +