kjannette/citation_sentinel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Readme updates #1

Closed
kjannette wants to merge 1 commits from FEAT-readme-update into master
pull from: FEAT-readme-update
merge into: kjannette:master
Conversation 1 Commits 1 Files Changed 1 +30 -19
1 changed files with 30 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
README.md
View File

@@ -8,12 +8,12 @@ Built with a React/Vite frontend and a Typescript/Node/Express backend, using An
## Query pipeline ## Query pipeline
Source Ingestion -> Parsing -> Chunking -> Embedding (using Voyage AI voyage-3 model) -> Storage (vector store) -> { user query submission } -> Evaluation of User Query -> Retrieval -> Ranking -> Response Generation (Using Anthopic's claude-opus-4-6 model) -> Response Groundedness Scoring (using Voyage AI rerank-r model) Source Ingestion -> Parsing -> Chunking -> Embedding (using Voyage AI voyage-3 model) -> Storage (vector store) -> { user query submission } -> Evaluation of User Query -> Retrieval -> Ranking -> Response Generation (Using Anthopic's claude-opus-4-6 model) -> Response Groundedness Scoring (using Voyage AI rerank-r model)
## Prerequisites ## Prerequisites
- **Node.js** (v18+) - **Node.js** (v18+)
- **yt-dlp** -- required for YouTube video source support (`brew install yt-dlp`) - **yt-dlp** -- Required for YouTube video source support (`brew install yt-dlp`)
## Getting Started ## Getting Started
@@ -38,9 +38,17 @@ This app demonstrates the core source-grounded Q&A pattern with transparent retr
## Methodology ## Methodology
This application is a RAG (Retrieval-Augmented Generation) system that allows users to upload source documents — PDFs, DOCX files, plain text, audio files, web URLs, and YouTube videos — which are then parsed, split into ~2000-character overlapping chunks, and converted into vector embeddings using Voyage AI's voyage-3 model. Those embeddings are stored in an in-memory vector store. This is a RAG (Retrieval-Augmented Generation) and query-response source-groundedness assurance system allowing users to create a research knowledge corpus, including:
When a user submits a query, the system enforces groundedness through a multi-layered strategy: 1. Documents — PDFs, DOCX files, plain text audio files.
2. Internet sources: via web URLs.
2. Audio sources: i.e. YouTube videos, audio from which is transcribed to text.
These are then parsed, split into ~2000-character overlapping chunks, and converted into vector embeddings using Voyage AI's voyage-3 model.
The embeddings are stored in an in-memory vector store.
When a user submits a query, the system enforces groundedness through a multi-layered strategy:
1. **Retrieval constraint** — The query is embedded (via Voyage AI voyage-3) and compared against 1. **Retrieval constraint** — The query is embedded (via Voyage AI voyage-3) and compared against
stored chunk vectors using cosine similarity, returning the top 20 candidates. stored chunk vectors using cosine similarity, returning the top 20 candidates.
@@ -48,13 +56,14 @@ When a user submits a query, the system enforces groundedness through a multi-l
2. **Reranking for precision** — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder, 2. **Reranking for precision** — Those 20 candidates are sent to Voyage AI's rerank-2 cross-encoder,
which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive. which re-scores each query-chunk pair with deeper semantic analysis. Only the top 5 survive.
3. **Prompt-level constraint** — The top 5 chunks are passed to the "Primary LLM" (Claude opus-4-6). 3. **Prompt-level constraint** — The top 5 chunks are passed to the "Primary LLM" (Claude opus-4-6)
The LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are for natural language query response.
The Primary LLM must cite sources using bracketed indices (e.g., [1], [2]) and admit when sources are
insufficient. insufficient.
4. **Schema enforcement** — The LLM's response is constrained to a JSON schema requiring structured 4. **Schema enforcement** — The Primary LLM's response is constrained to a JSON schema requiring structured
fields (answer, citedSourceIndices, followUpQuestions). Any cited source indices that do not fields (answer, citedSourceIndices, followUpQuestions). Any cited source indices that do not
correspond to real source groups are programmatically stripped out. correspond to real source groups are programmatically removed.
5. **Post-generation groundedness scoring** — The answer is split into individual sentences. Voyage 5. **Post-generation groundedness scoring** — The answer is split into individual sentences. Voyage
AI voyage-3 embeds each sentence, and compares it (using cosine similarity) against the vectors AI voyage-3 embeds each sentence, and compares it (using cosine similarity) against the vectors
@@ -71,6 +80,8 @@ Archetypical cosine scores range from -1 to 1.
0: Vectors are at a 90-degree angle (orthogonal/unrelated). 0: Vectors are at a 90-degree angle (orthogonal/unrelated).
-1: Vectors point in opposite directions. -1: Vectors point in opposite directions.
Cosine distance between high-dimensional text embeddings is compressed: unrelated notes sit close to orthogonal, so both the within-cluster and nearest-cluster distances land near 0.8. In this application, scores are normalized to a bounded range wherein unrelated query-response pair scores are nearly orthogonal.
## Design ## Design
Two-package monorepo: Two-package monorepo:
@@ -85,4 +96,4 @@ MIT. See [LICENSE](./LICENSE).
## Author ## Author
[@sjdev](https://sjdev.co) @sjdev