Compare commits
7 Commits
fe36ea6aaa
...
FEAT-minor
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5e0189be61 | ||
| fc6ceb558b | |||
| 676c86cf43 | |||
| 6810704173 | |||
| 4a1c67c518 | |||
| 3a4ac641cd | |||
| 861983531a |
27
README.md
27
README.md
@@ -1,19 +1,21 @@
|
||||
# Citation Sentinel © 2023 @sjDev
|
||||
# Source Sentinel - 2025 @sjDev - LICENSE: MIT
|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
Image one: source-grounded research assistant for applied use of LLMs.
|
||||
|
||||
Image one: source-grounded research assistant that empowers users to transform an otherwise-unmanageably-large corpus of data on any topic of interest into refined, easily-digested subtopics and pose focused inquiries, to instantly receive concise, quantifiably-evaluated, accurate responses. The system also provides suggested follow-up questions further refining in user inquiries by detecting the query goals.
|
||||
**Source Sentinel's value proposition: analyzing and conveniently working with large data corpora, when query-result accuracy is of the highest value. Examples include technical medical data exploration (reviewing health records, test results, or studies to find hidden patterns, trends, and answers), complex litigation (i.e. Patent/IP), advanced LLM development and refinement.**
|
||||
|
||||
Source Sentinel empowers users to transform an otherwise-unmanageably-large corpus of data on any topic of interest into refined, easily-digested subtopics and pose focused inquiries, to instantly receive concise, quantifiably-evaluated, accurate responses. The system also provides suggested follow-up questions further refining in user inquiries by detecting the query goals.
|
||||
|
||||
Image Two: demo co clickable inline citations that take user to source-grounded research basis/knowledge corpus supporting query response, with rated “groundedness” score.
|
||||
ALSO NOTE IN DEMO: clickable inline citations (in blue) that take user to source-grounded research basis/knowledge corpus supporting query response, with rated “groundedness” score.
|
||||
|
||||
|
||||
# Reliability and Understandability
|
||||
|
||||
|
||||
Citation Sentinel's Retrieval-Augmented Generation (RAG) pipeline uses two-stage pass-through to Voyage AI embedding and ranking models, which uses cosine similarity methodology to scoring to evaluate query-source embeddings in multi-dimensional vector space. This yields a "groundedness" score.
|
||||
Source Sentinel's Retrieval-Augmented Generation (RAG) pipeline uses two-stage pass-through to Voyage AI embedding and ranking models, which uses cosine similarity methodology to scoring to evaluate query-source embeddings in multi-dimensional vector space. This yields a "groundedness" score.
|
||||
|
||||
|
||||
Groundedness metrics quantify how well an AI-generated answer is supported by retrieved context. It measures "faithfulness" to source documents, ensuring the answer is not hallucinated, inaccurate or stale (pulled from the model's training data).
|
||||
@@ -22,7 +24,7 @@ Groundedness metrics quantify how well an AI-generated answer is supported by re
|
||||
# Architecture - Basic Overview
|
||||
|
||||
|
||||
Built with a React/Vote frontend and a Typescript/Node/Express backend, using Anthropic Claude for generation, OpenAI Whisper for video audio track transcription, Voyage AI voyage-3 for embeddings and Voyage AI rerank-2 for response cosine similarity scoring, to generate the "groundedness" score, presented as an easily-understandable red/yellow/green "badge" style tooltip in the response (see above.)
|
||||
Built with a React/Vite frontend and a Typescript/Node/Express backend, using Anthropic Claude for NL response generation, OpenAI Whisper for video audio track transcription, Voyage AI voyage-3 for embeddings and Voyage AI rerank-2 for response cosine similarity scoring (to generate the "groundedness" score) presented as an easily-understandable red/yellow/green "badge" style tooltip in the response (see image above.)
|
||||
|
||||
|
||||
## Query pipeline
|
||||
@@ -30,6 +32,19 @@ Built with a React/Vote frontend and a Typescript/Node/Express backend, using An
|
||||
|
||||
Source Ingestion -> Parsing -> Chunking -> Embedding (using Voyage AI voyage-3 model) -> Storage (vector store) -> { user query submission } -> Evaluation of User Query -> Retrieval -> Ranking -> Response Generation (Using Anthopic's claude-opus-4-6 model) -> Response Groundedness Scoring (using Voyage AI rerank-r model)
|
||||
|
||||
## Why Database Math Can't Replace Groundedness Scoring
|
||||
|
||||
Many recent additions to the Vector database/store product space use distance metrics (Cosine, L2, Inner Product) for Bi-Encoder Similarity. They compare the vector of the user query against the vectors of chunks independently.
|
||||
|
||||
The rerank-r model at the end of the Source Sentinel pipeline performs Cross-Encoder Evaluation, taking two entirely separate text inputs:
|
||||
|
||||
1) The generated response from claude-opus-4-6 and
|
||||
|
||||
2) The raw source chunks
|
||||
|
||||
It processes them simultaneously through deep attention layers to check for hallucinations, missing context, and factual alignment.
|
||||
|
||||
A vector database cosine metric only calculates how close two embeddings are in coordinate space. It **does not read the generated text of a response to verify if it accurately reflects the source chunks**.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
||||
BIN
sample screen scaps/DEMO FOR SOURCE SENT.jpg
Normal file
BIN
sample screen scaps/DEMO FOR SOURCE SENT.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 365 KiB |
BIN
sample screen scaps/DEMO FOR SOURCE SENT_SMALL.jpg
Normal file
BIN
sample screen scaps/DEMO FOR SOURCE SENT_SMALL.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
BIN
sample screen scaps/SMALL_DEMO FOR SOURCE SENT.jpg
Normal file
BIN
sample screen scaps/SMALL_DEMO FOR SOURCE SENT.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 157 KiB |
BIN
sample screen scaps/Screenshot 2026-08-03 at 12.03.34 AM.png
Normal file
BIN
sample screen scaps/Screenshot 2026-08-03 at 12.03.34 AM.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 307 KiB |
BIN
sample screen scaps/Screenshot 2026-08-03 at 12.04.10 AM.png
Normal file
BIN
sample screen scaps/Screenshot 2026-08-03 at 12.04.10 AM.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 309 KiB |
@@ -104,11 +104,11 @@ export async function generateStudyGuide(sourceGroups: SourceGroup[]): Promise<S
|
||||
const sources = buildSourceBlock(sourceGroups);
|
||||
const start = Date.now();
|
||||
|
||||
const prompt = `You are an expert educator. Given the source documents below, produce a comprehensive study guide. Follow these rules:
|
||||
const prompt = `You are an expert reasearch assistant. Given the source documents below, produce a comprehensive guide. Follow these rules:
|
||||
|
||||
1. Create a structured outline of the main ideas organized into logical sections.
|
||||
2. For each section, provide: bullet-point key concepts, key terms with definitions, and 2-3 self-review questions.
|
||||
3. At the end, provide mnemonic devices or simplified restatements to help memorize the hardest concepts.
|
||||
3. At the end, provide mnemonic devices or simplified restatements of the hardest concepts.
|
||||
4. Be concise but thorough. Use simple, clear language.
|
||||
|
||||
--- SOURCE DOCUMENTS ---
|
||||
|
||||
Reference in New Issue
Block a user