kongruity/README.md

# kongruity

“...All those moments will be lost in time, like tears in rain.”

kongruity pulls in unstructured artifacts of the creative-engineering process -- to-dos, action items, agile tickets, Jira thread comments, Slack thread comments, retrospective notes -- and synthesizes them into semantically coherent, prioritized clusters that can be incorporated into implementation planning.

In kongruity, the artifacts become "sticky notes." A board full of them looks chaotic.

With a click, they are semantically evaluated, grouped into thematic clusters with descriptive headers, rankable and exportable to project planning and execution tools.

## Clustering and evaluation: methodology

Two models run in parallel, and neither sees the other's work. Anthropic's `claude-sonnet-5` (`backend/services/clustering.service.js`) reads the raw text of every note and groups them into labeled thematic clusters. 

At the same time, Voyage AI's voyage-3 model (`backend/services/embedding.service.js`) converts each note's text into a numeric representation of its semantic meaning aka vector. 

Once the LLM returns, kongruity scores that grouping (`backend/services/validation.service.js`) using an established silhouette coefficient, with cosine distance rather than Euclidean as the distance metric. 

For each note, it weighs the average distance to the other notes in its own cluster against the average distance to the notes in the nearest neighboring cluster. Averaged across every note, this yields a single cohesion score in the range [−1, 1], displayed at the top of the results.

This yields an empirical groundedness evaluation. One model proposes the grouping; an independent model evaluates grouping accuracy.

Note that: before scoring, structural validation confirms that each note landed in exactly one cluster, that no cluster is empty, and that no hallucinated note IDs appear. A malformed response to the validation completely fails, rather than quietly returning a partial board.

## Reading the cohesion score

Average silhouette width is a widely-used measure of clustering quality. Higher values indicate:

1. The qualitative semantic cohesiveness of clusters, and:

2. How well-separated each cluster is from its nearest neighboring cluster.

The score appears above the results with a plain-language band:

- **0.70 and above** — Strong
- **0.40 to 0.69** — Moderate
- **0.10 to 0.39** — Weak
- **Below 0.10** — Poor

Silhouette values are archetypically bounded below 1.0 for real-world data, so the number is best read as a relative measure. See Hugo Sträng, Tai Dinh. An upper bound on the silhouette evaluation metric for clustering. Pattern Recognition, Volume 178, 2026, 113402, ISSN 0031-3203.

## Organizing clusters, exporting to workflow software

Teams can drag-and-rank related task clusters by implementation priority - turning noise into an actionable workflow.

(Integrations with third-party project management, planning and workflow applications are action-items for next major version, see Roadmap, below)

## How it works

1. **Ingest** — Sticky notes are loaded and displayed on a board.
2. **Cluster** — An LLM reads every note and groups them by semantic similarity (not keywords).
3. **Evaluate** — In parallel, a separate embedding model (Voyage AI) generates vector representations of each note. A silhouette-based cohesion score measures how well-separated and internally consistent clusters are. The score is displayed alongside the results.
4. **Validate** — Structural checks confirm every note is assigned to exactly one cluster, no clusters are empty, and labels are present.
5. **Prioritize** — Clusters appear ranked and are drag-reorderable. Teams set implementation priority by dragging clusters into position.

## Dev implementation notes

Developers may swap in other LLM SDKs/APIs and alter prompt syntax in `backend/services/clustering.service.js` to experiment with LLMs and platforms of their choice.

## Development Roadmap

### Ingestion — pulling tagged artifacts in

- [ ] **Slack** — where decisions actually get made; a `:sticky:` emoji reaction fires an Events API webhook that pulls the message in.
- [ ] **Microsoft Teams** — same capture gesture for enterprise shops; message extension plus Graph change notifications.
- [ ] **Jira** — label- or mention-triggered webhook scoped by JQL. (This is where comments typically carry half the backlog's context.)
- [ ] **Linear** — engineering-side tickets and threads; label-triggered GraphQL webhook.
- [ ] **GitHub** — issue, PR review, and discussion comments; label- or mention-triggered webhook.
- [ ] **Miro / FigJam** — REST API import
- [ ] **Confluence / Notion** —  page and inline-comment fetch. (Where retro and planning notes are born).
- [ ] **Meeting transcripts (Granola, Otter, Zoom, Google Meet)** — where retros are now recorded, an option for action-item extraction from the transcript API.
- [ ] **Generic REST, email, and Zapier** — authenticated bulk `POST /v1/notes`.

### Export — pushing ranked clusters to workflow tools

- [ ] **Jira** — drag-rank written through the Agile API's board rank endpoint.
- [ ] **Asana** —  drag-rank written as task order within the section.
- [ ] **Rally** — drag-rank written as portfolio rank. (Clusters become features and notes, which become stories).
- [ ] **Linear** — drag-rank written to issue `sortOrder`.
- [ ] **Azure DevOps / GitHub Projects v2** — drag-rank written as project field ordering. Clusters become work-item parents.
- [ ] **CSV, JSON, and Markdown** — direct download from the cluster view. (Should ship before any OAuth work.)

## Prerequisites

- Node.js (v18 or later recommended)
- PostgreSQL (v14 or later recommended)
- An [Anthropic API key](https://console.anthropic.com/) (or other LLM platform, for clustering)
- A [Voyage AI API key](https://dash.voyageai.com/) (for embedding-based evaluation)

## Setup

### 1. Clone the repository

```bash
git clone https://github.com/kjannette/kongruity_
cd kongruity
```

### 2. Create an environment file

The backend expects a `.env` file in the `backend/` directory. This file is git-ignored and must be created manually:

```bash
cat > backend/.env << 'EOF'
ANTHROPIC_API_KEY=<your Anthropic API key> (or other LLM platform key)
VOYAGEAI_API_KEY=<your Voyage AI API key>
DATABASE_URL=postgresql://<user>:<password>@localhost:5432/kongruity
EOF
```

Replace placeholder values with your actual keys and database credentials.

### 3. Set up/run the database

Start DB  for local development (assumes local dev env MacOS and Homebrew installed)

```bash
brew services start postgresql@15
```

Create a PostgreSQL database for the project:

```bash
createdb kongruity
```

Run the migration to create tables:

```bash
cd backend
npm run db:migrate
```

Seed the database with the sample sticky notes:

```bash
npm run db:seed
```

### 4. Install dependencies

```bash
cd backend
npm install
```

```bash
cd frontend
npm install
```

## Run the app

### Start the backend — Production mode

From the `backend/` directory:

```bash
npm run start
```

The API server starts on **http://localhost:3001** (configurable via the `PORT` environment variable).

### Start the backend — Development mode

To start using Nodemon for hot reloads while developing:

```bash
npm run dev
```

### Build the frontend — Production mode

From the `frontend/` directory:

```bash
npm run build
```

### Start the frontend — Development mode

From the `frontend/` directory:

```bash
npm run dev
```

The Vite dev server starts on **http://localhost:5173** by default. Open that URL in a browser.

## Running tests

### Backend tests

From the `backend/` directory:

```bash
npm test
```

Backend tests use Vitest with Supertest for HTTP assertions.

### Frontend tests

From the `frontend/` directory:

```bash
npm test
```

This runs Vitest with jsdom. For watch mode during development:

```bash
npm run test:watch
```