quick edit readme

This commit is contained in:
KS Jannette
2026-08-26 22:03:38 -04:00
parent cae6edfb78
commit b1c7655ecb

View File

@@ -1,5 +1,7 @@
# [Sem_Cache](https://github.com/kjannette/semantic-cache-script)
![Script usage screenshot](semcache.jpg)
Sem_Cache is a tiny command-line tool written in [Python](https://www.python.org/) that
demonstrates semantic caching for Large Language Model (LLM) queries. It uses the
[Sentence Transformers](https://www.sbert.net/) library with [NumPy](https://numpy.org/)
@@ -21,6 +23,8 @@ The output can be diff’d against the behavior of the system under development
---
## How It Works
If two queries mean the same thing, they should return the same answer. Rather than comparing strings character-by-character, it converts each query into a dense vector embedding using the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model (a lightweight transformer that runs locally). It then calculates the cosine similarity between the cached query embedding and the new query embedding.
@@ -31,13 +35,19 @@ a **cache miss** and routes the request to the LLM.
---
## Getting Started
### Prerequisites
- [Python](https://www.python.org/) 3.10 or higher
- [pip](https://pip.pypa.io/) (Python package installer)
### Installation
Clone the repository and set up a virtual environment:
@@ -70,6 +80,8 @@ The program will prompt you to enter two sentences:
1. **Sentence 1**: The baseline cached query (simulating a query already in the cache)
2. **Sentence 2**: The new incoming query (simulating a user's new request)
### Example Session
```
@@ -101,6 +113,8 @@ a cache hit with a similarity score of 0.9412.
---
## Configuration
The similarity threshold is set to 0.92 by default. This is a highly-conservative value, set to reduce false positives (treating dissimilar queries as matches). To adjust the threshold,
@@ -115,6 +129,8 @@ Higher values reduce false positives but may miss valid semantic matches.
---
## Project Structure
```
@@ -127,8 +143,12 @@ semantic-cache-script/
---
## Participation
### Bug Reports
Bug reports are accepted via [Git issues](https://github.com/kjannette/semantic-cache-script/issues).
@@ -146,12 +166,16 @@ This project follows [PEP 8](https://peps.python.org/pep-0008/) style guidelines
---
## Author
- @ sjDev
---
## Ideology
This project does not have a formal Code of Conduct.
@@ -163,6 +187,8 @@ executes on-device; no external API calls are made by the caching logic itself).
---
## Roadmap / TO-DO
(This README also serves as a development notebook.)
@@ -172,6 +198,8 @@ executes on-device; no external API calls are made by the caching logic itself).
- [ ] Print result to command line (completed in current version)
- [ ] Perform and output quantified metrics of what a cache hit conserves
### Possible Metrics to Implement
1. **Estimated completion tokens** — Calculate tokens saved by popular model
@@ -192,6 +220,8 @@ executes on-device; no external API calls are made by the caching logic itself).
---
## License
This softeare is released under the [GNU General Public License Version 3](https://opensource.org/license/gpl-3-0).
This softeare is released under the [GNU General Public License Version 3](https://opensource.org/license/gpl-3-0).