# kongruity: Signal from noise - 2025-2026 @sjDev - LICENSE: MIT ![kongruity demo image.](kongruity_with_frame.jpg) kongruity pulls in unstructured artifacts of the creative-engineering process, capturing "AHA!" moments scattered across an engineering team's disparate tools: action items or backlog in Atlassian/Jira, user story comments, Slack discussions, retrospective feedback. It stores and synthesizes these into semantically coherent, prioritized clusters that can be incorporated into implementation planning. In kongruity, artifacts become "sticky notes." A board full looks chaotic. With a click, a RAG-pipeline levearges Models trained to semantically evaluate, group, cluster and add descriptive cluster headers. These are rankable, editable and exportable to sprint project planning tools. ## Clustering and evaluation: methodology Two models run in parallel, and neither sees the other's work. Anthropic's `claude-sonnet-5` (`backend/services/clustering.service.js`) reads the raw text of every note and groups them into labeled thematic clusters. At the same time, Voyage AI's voyage-3.5 model (`backend/services/embedding.service.js`) converts each note's text into a numeric representation of its semantic meaning aka vector. Once the LLM returns, kongruity scores that grouping (`backend/services/validation.service.js`) using an established silhouette coefficient, with cosine distance rather than Euclidean as the proximity metric. For each note, it weighs the average distance to the other notes in its own cluster against the average distance to the notes in the nearest neighboring cluster. Averaged across every note, this yields a single numeric cohesion score, displayed at the top of the results, along with plaintext: Strong, Moderate, Weak, Poor. This yields an empirical groundedness evaluation. One model proposes the grouping; an independent model evaluates grouping accuracy. Note that: before scoring, structural validation confirms that each note landed in exactly one cluster, that no cluster is empty, and that no hallucinated note IDs appear. A malformed response to the validation completely fails, rather than quietly returning a partial board. ## Voyage AI voyage-3.5: best-in-class embedding ![Embedding model benchmarking.](Voyage.jpg) ## Reading the cohesion score Average silhouette width is a widely-used measure of clustering quality. Higher values indicate: 1. The qualitative semantic cohesiveness of clusters, and: 2. How well-separated each cluster is from its nearest neighboring cluster. Although the coefficient is mathematically bounded by [−1, 1], cosine distance between high-dimensional text embeddings is compressed: unrelated notes sit close to orthogonal, so both the within-cluster and nearest-cluster distances land near 0.8. Because silhouette divides the gap between them by the larger of the two, the practical range on embedding data is roughly [−0.05, 0.10] rather than the full interval. The bands below are therefore calibrated against that observed range. On the seed board, the five ideal thematic clusters score 0.09; swapping a few notes between clusters drops it to 0.06; a scrambled assignment falls below zero. The score appears above the results with a plain-language band: - **0.07 and above** — Strong - **0.04 to 0.06** — Moderate - **0.01 to 0.03** — Weak - **Below 0.01** — Poor A score near 0.00 means the grouping is no better than chance. Bands are specific to `voyage-3` cosine distance and would need recalibration behind a different embedding model. See Hugo Sträng, Tai Dinh. An upper bound on the silhouette evaluation metric for clustering. Pattern Recognition, Volume 178, 2026, 113402, ISSN 0031-3203. ## Organizing clusters, exporting to workflow software Teams can drag-and-rank related task clusters by implementation priority - turning noise into an actionable workflow. (Integrations with third-party project management, planning and workflow applications are action-items for next major version, see Roadmap, below) ## How it works 1. **Ingest** — Sticky notes are loaded and displayed on a board. 2. **Cluster** — An LLM reads every note and groups them by semantic similarity (not keywords). 3. **Evaluate** — In parallel, a separate embedding model (Voyage AI) generates vector representations of each note. A silhouette-based cohesion score measures how well-separated and internally consistent clusters are. The score is displayed alongside the results. 4. **Validate** — Structural checks confirm every note is assigned to exactly one cluster, no clusters are empty, and labels are present. 5. **Prioritize** — Clusters appear ranked and are drag-reorderable. Teams set implementation priority by dragging clusters into position. ## Dev implementation notes Developers may swap in other LLM SDKs/APIs and alter prompt syntax in `backend/services/clustering.service.ts` to experiment with LLMs and platforms of their choice. ## Integration API The shared infrastructure that third-party integrations are built on. See [ARCHITECTURE.md](ARCHITECTURE.md) for the design, [ROADMAP.md](ROADMAP.md) for milestones, and [LOW_LEVEL_DESIGN.md](LOW_LEVEL_DESIGN.md) for module contracts. | Endpoint | Auth | Purpose | | --- | --- | --- | | `POST /v1/notes` | `Authorization: Bearer ` | Bulk push notes. Body `{ notes: [{ id, text, author, x?, y?, color? }] }`, max 5000 per request. | | `POST /v1/webhooks/:provider` | Per-provider request signature | Inbound provider deliveries. Acks immediately, inserts in the background. | | `POST /v1/slack/commands` | Slack `v0` request signature | The `/sticky` slash command. Form-encoded rather than JSON, and its response body is shown to the user in Slack. | Registered providers live in `backend/config/providers.ts`. `rest`, `linear`, and `slack` carry payload mappings today; `github` and `jira` declare transport and OAuth endpoints only, so adding one of those integrations means writing a single normalizer rather than new plumbing. A provider may also declare an async `enrich` hook. It runs inside the retry queue *after* the request is acknowledged, which is where any network call belongs: normalizers stay pure and synchronous so a delivery can be acked inside Slack's three-second budget. Slack needs this because a reaction event names a message without carrying its text. Deliveries are deduplicated on `(provider, external_id)` in the `ingest_events` table, which matters beyond tidiness: duplicate notes compress intra-cluster distance and depress the cohesion score. ### Adding a provider 1. Add a slug to `ProviderSlug` in `backend/types/integration.ts`. 2. Add an entry to `providers` in `backend/config/providers.ts` with its signature scheme, header names, and OAuth endpoints. 3. Write one normalizer in `backend/config/normalizers.ts` mapping that provider's payload to notes. 4. Add an `enrich` hook only if the provider's payload references content it does not include. 5. Set the provider's signing secret and OAuth credentials in `backend/.env`. ## Development Roadmap ### Ingestion — pulling tagged artifacts in - [x] **Slack** — where decisions actually get made; `/sticky` captures a new note, and an emoji reaction (`:pushpin:` by default) fires an Events API webhook that pulls an existing message in. - [ ] **Microsoft Teams** — same capture gesture for enterprise shops; message extension plus Graph change notifications. - [ ] **Jira** — label- or mention-triggered webhook scoped by JQL. (This is where comments typically carry half the backlog's context.) - [ ] **Linear** — engineering-side tickets and threads; label-triggered GraphQL webhook. - [ ] **GitHub** — issue, PR review, and discussion comments; label- or mention-triggered webhook. - [ ] **Miro / FigJam** — REST API import - [ ] **Confluence / Notion** — page and inline-comment fetch. (Where retro and planning notes are born). - [ ] **Meeting transcripts (Granola, Otter, Zoom, Google Meet)** — where retros are now recorded, an option for action-item extraction from the transcript API. - [ ] **Generic REST, email, and Zapier** — authenticated bulk `POST /v1/notes`. ### Export — pushing ranked clusters to workflow tools - [ ] **Jira** — drag-rank written through the Agile API's board rank endpoint. - [ ] **Asana** — drag-rank written as task order within the section. - [ ] **Rally** — drag-rank written as portfolio rank. (Clusters become features and notes, which become stories). - [ ] **Linear** — drag-rank written to issue `sortOrder`. - [ ] **Azure DevOps / GitHub Projects v2** — drag-rank written as project field ordering. Clusters become work-item parents. - [ ] **CSV, JSON, and Markdown** — direct download from the cluster view. (Should ship before any OAuth work.) ## Prerequisites - Node.js (v18 or later recommended) - PostgreSQL (v14 or later recommended) - An [Anthropic API key](https://console.anthropic.com/) (or other LLM platform, for clustering) - A [Voyage AI API key](https://dash.voyageai.com/) (for embedding-based evaluation) ## Setup ### 1. Clone the repository ```bash git clone https://github.com/kjannette/kongruity_ cd kongruity ``` ### 2. Create an environment file The backend expects a `.env` file in the `backend/` directory. This file is git-ignored and must be created manually: ```bash cat > backend/.env << 'EOF' ANTHROPIC_API_KEY= (or other LLM platform key) VOYAGEAI_API_KEY= DATABASE_URL=postgresql://:@localhost:5432/kongruity EOF ``` Replace placeholder values with your keys and database credentials. #### Integration variables Only needed once you start using the ingestion endpoints. `TOKEN_ENCRYPTION_KEY` is required by anything that stores third-party credentials; generate one with `openssl rand -base64 32`. ```bash cat >> backend/.env << 'EOF' TOKEN_ENCRYPTION_KEY=<32 bytes, base64 encoded> LINEAR_SIGNING_SECRET= SLACK_SIGNING_SECRET= GITHUB_WEBHOOK_SECRET= LINEAR_CLIENT_ID= LINEAR_CLIENT_SECRET= EOF ``` The key must decode to exactly 32 bytes; the backend refuses to encrypt otherwise rather than falling back to something weaker. ## Slack integration Two capture gestures, both landing in the same ingest pipeline: - **`/sticky `** creates a note from what you type. - **Reacting with an emoji** captures the message someone already wrote. The trigger defaults to `:pushpin:` and is set by `SLACK_CAPTURE_REACTION`. ### 1. Create the Slack app At [api.slack.com/apps](https://api.slack.com/apps), create an app from scratch in your workspace. Under **OAuth & Permissions**, add the bot token scopes `channels:history`, `reactions:read`, `users:read`, and `commands`, then install the app to the workspace and copy the **Bot User OAuth Token** (`xoxb-…`). Under **Basic Information**, copy the **Signing Secret**. ```bash cat >> backend/.env << 'EOF' SLACK_BOT_TOKEN=xoxb- SLACK_CAPTURE_REACTION=pushpin EOF ``` `SLACK_SIGNING_SECRET` is already in the integration variables above. There is no OAuth flow here on purpose: a single workspace reading credentials from the environment is far less machinery than an install dance, and the OAuth endpoints in the provider registry stay unused until multi-workspace support is actually needed. ### 2. Expose backend Slack only delivers to a public HTTPS URL, so a tunnel is required for local development. ```bash brew install ngrok ngrok http 3001 ``` Copy the `https://` forwarding URL ngrok prints. It changes every restart on the free plan, and both URLs below have to be updated when it does. ### 3. Point Slack at the tunnel Under **Slash Commands**, create `/sticky` with the request URL `https:///v1/slack/commands`. Under **Event Subscriptions**, enable events and set the request URL to `https:///v1/webhooks/slack`. Slack immediately sends a `url_verification` challenge, which the webhook route answers before signature checking, so the backend must already be running when you save. Subscribe to the bot event `reaction_added`. Reinstall the app if Slack prompts you, then invite the bot to any channel you want to capture from with `/invite @`. A reaction in a channel the bot is not a member of arrives as a `not_in_channel` error, which is classified as permanent and will not be retried. ### 3. Set up/run the database Start DB for local development (assumes local dev env MacOS and Homebrew installed) ```bash brew services start postgresql@15 ``` Create a PostgreSQL database for the project: ```bash createdb kongruity ``` Run the migration to create tables: ```bash cd backend npm run db:migrate ``` Seed the database with the sample sticky notes: ```bash npm run db:seed ``` ### 4. Install dependencies ```bash cd backend npm install ``` ```bash cd frontend npm install ``` ## Run the app ### Start the backend — Production mode From the `backend/` directory, compile the TypeScript sources and run the output: ```bash npm run build npm run start ``` The API server starts on **http://localhost:3001** (configurable via the `PORT` environment variable). ### Start the backend — Development mode To run the TypeScript sources directly with hot reloads while developing: ```bash npm run dev ``` To type-check without emitting: ```bash npm run type-check ``` ### Build the frontend — Production mode From the `frontend/` directory: ```bash npm run build ``` ### Start the frontend — Development mode From the `frontend/` directory: ```bash npm run dev ``` The Vite dev server starts on **http://localhost:5173** by default. Open that URL in a browser. ## Running tests ### Backend tests From the `backend/` directory: ```bash npm test ``` Backend tests use Vitest with Supertest for HTTP assertions. ### Frontend tests From the `frontend/` directory: ```bash npm test ``` This runs Vitest with jsdom. For watch mode during development: ```bash npm run test:watch ```