quick edit readme
This commit is contained in:
30
README.md
30
README.md
@@ -1,5 +1,7 @@
|
||||
# [Sem_Cache](https://github.com/kjannette/semantic-cache-script)
|
||||
|
||||

|
||||
|
||||
Sem_Cache is a tiny command-line tool written in [Python](https://www.python.org/) that
|
||||
demonstrates semantic caching for Large Language Model (LLM) queries. It uses the
|
||||
[Sentence Transformers](https://www.sbert.net/) library with [NumPy](https://numpy.org/)
|
||||
@@ -21,6 +23,8 @@ The output can be diff’d against the behavior of the system under development
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## How It Works
|
||||
|
||||
If two queries mean the same thing, they should return the same answer. Rather than comparing strings character-by-character, it converts each query into a dense vector embedding using the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) model (a lightweight transformer that runs locally). It then calculates the cosine similarity between the cached query embedding and the new query embedding.
|
||||
@@ -31,13 +35,19 @@ a **cache miss** and routes the request to the LLM.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- [Python](https://www.python.org/) 3.10 or higher
|
||||
- [pip](https://pip.pypa.io/) (Python package installer)
|
||||
|
||||
|
||||
|
||||
### Installation
|
||||
|
||||
Clone the repository and set up a virtual environment:
|
||||
@@ -70,6 +80,8 @@ The program will prompt you to enter two sentences:
|
||||
1. **Sentence 1**: The baseline cached query (simulating a query already in the cache)
|
||||
2. **Sentence 2**: The new incoming query (simulating a user's new request)
|
||||
|
||||
|
||||
|
||||
### Example Session
|
||||
|
||||
```
|
||||
@@ -101,6 +113,8 @@ a cache hit with a similarity score of 0.9412.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Configuration
|
||||
|
||||
The similarity threshold is set to 0.92 by default. This is a highly-conservative value, set to reduce false positives (treating dissimilar queries as matches). To adjust the threshold,
|
||||
@@ -115,6 +129,8 @@ Higher values reduce false positives but may miss valid semantic matches.
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
@@ -127,8 +143,12 @@ semantic-cache-script/
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Participation
|
||||
|
||||
|
||||
|
||||
### Bug Reports
|
||||
|
||||
Bug reports are accepted via [Git issues](https://github.com/kjannette/semantic-cache-script/issues).
|
||||
@@ -146,12 +166,16 @@ This project follows [PEP 8](https://peps.python.org/pep-0008/) style guidelines
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Author
|
||||
|
||||
- @ sjDev
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Ideology
|
||||
|
||||
This project does not have a formal Code of Conduct.
|
||||
@@ -163,6 +187,8 @@ executes on-device; no external API calls are made by the caching logic itself).
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## Roadmap / TO-DO
|
||||
|
||||
(This README also serves as a development notebook.)
|
||||
@@ -172,6 +198,8 @@ executes on-device; no external API calls are made by the caching logic itself).
|
||||
- [ ] Print result to command line (completed in current version)
|
||||
- [ ] Perform and output quantified metrics of what a cache hit conserves
|
||||
|
||||
|
||||
|
||||
### Possible Metrics to Implement
|
||||
|
||||
1. **Estimated completion tokens** — Calculate tokens saved by popular model
|
||||
@@ -192,6 +220,8 @@ executes on-device; no external API calls are made by the caching logic itself).
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
## License
|
||||
|
||||
This softeare is released under the [GNU General Public License Version 3](https://opensource.org/license/gpl-3-0).
|
||||
Reference in New Issue
Block a user