Merge pull request 'FEAT-nonblocking-backend-io-imporvements' (#1 ) from FEAT-nonblocking-backend-io-imporvements into master

Reviewed-on: #1
Cleanup
2026-08-01 05:43:24 +00:00 · 2026-08-01 01:42:43 -04:00 · 2026-08-01 01:35:27 -04:00 · 2026-08-01 01:06:19 -04:00 · 2026-07-31 13:45:35 +00:00 · 2026-03-06 03:52:47 -05:00
35 changed files with 2036 additions and 579 deletions
--- a/21
+++ b/21
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Steven Jannette
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
--- a/README.md
+++ b/README.md
@@ -1,40 +1,88 @@
-# kongruity app
+# kongruity

-kongruity employs Large Language Model ("LLM") semantic grouping functionality to cluster large volumes of "to dos" or issue tags in development (or other) settings, according to thematic or topical similarity.
+kongruity pulls in unstructured artifacts of the creative-engineering process -- to-dos, action items, agile tickets, Jira thread comments, Slack thread comments, retrospective notes -- and synthesizes them into semantically coherent, prioritized clusters that can be incorporated into implementation planning.

-(To learn more about this topic, see, e.g., [Kozlowski A., Boutyline A., Semantic Structure in Large Language Model Embeddings Aug. 2025, arXiv:2508.10003v1:04 Aug 2025](https://arxiv.org/html/2508.10003v1)).
+In kongruity world, the artifacts become "sticky notes." A board full of them looks chaotic. With a click, the LLM analyzes their semantic meaning and groups them into thematic clusters, each with a descriptive header. 

-In the world of kongruity, these "to dos" are called "sticky notes."  kongruity's React/Vite UI views a board of seemingly chaotic "sticky notes".  But with one click, they are transformed into manageable, actionable groups, each with a header that explains the group semantic interrelation.
+## Result evalutation/validation:

-The backend is an Express API that serves "sticky note" data and proxies semantic grouping requests to Antrhopic Claude.  
+An independent RAG embedding-based model scores evaluates quality using cosine similiarty, so the output is qualitatively refined. From there, teams can drag-and-rank related task clusters by implementation priority - turning noise into an actionable workflow.

-Developers may feel free to install other LLM SDKs and alter the syntax at backend/services/clustering.service.js to experiment with any LLM model/platform they prefer.
+## How it works
+
+1. **Ingest** — Sticky notes are loaded and displayed on a board.
+2. **Cluster** — An LLM reads every note and groups them by semantic similarity (not keywords).
+3. **Evaluate** — In parallel, a separate embedding model (Voyage AI) generates vector representations of each note. A silhouette-based cohesion score measures how well-separated and internally consistent clusters are. The score is displayed alongside the results.
+4. **Validate** — Structural checks confirm every note is assigned to exactly one cluster, no clusters are empty, and labels are present.
+5. **Prioritize** — Clusters appear ranked and are drag-reorderable. Teams set implementation priority by dragging clusters into position.
+
+## Dev implementation notes 
+
+As of 03.04.2026, two of the above-described features are in the planning and implementation phase:
+
+1. **Ingestion** - The implementation goal is a system for easily tagging items/issues mentioned in Slack, Jira comments, etc. (similar to hashtagging), and running batch "pulls" of these tagged items into kongruity via REST API interface.
+2. **Prioritization** -- The final prioritization and planning stage will ultimately result in "pushing" these ordered items back **out** into Jira, Asana, Rally (etc.), for incorporation into Epic/Sprint workflows. 
+
+Also, developers may swap in other LLM SDKs/APIs and alter prompt syntax in `backend/services/clustering.service.js` to experiment with any model or platform of their choice.

 ## Prerequisites

 - Node.js (v18 or later recommended)
- An [Anthropic API key](https://console.anthropic.com/)
+- PostgreSQL (v14 or later recommended)
+- An [Anthropic API key](https://console.anthropic.com/) (or other LLM platform, for clustering)
+- A [Voyage AI API key](https://dash.voyageai.com/) (for embedding-based evaluation)

 ## Setup

-### 1. Unzip the project
+### 1. Clone the repository

 ```bash
-unzip kongruity.zip
+git clone https://github.com/kjannette/kongruity_
 cd kongruity
 ```

 ### 2. Create an environment file

-The backend expects a `.env` file containing an Anthropic API key in the root `backend/` directory. This file is git-ignored and must be created manually:
+The backend expects a `.env` file in the `backend/` directory. This file is git-ignored and must be created manually:

 ```bash
-echo 'ANTHROPIC_API_KEY=<your Anthropic API key>' > backend/.env
+cat > backend/.env << 'EOF'
+ANTHROPIC_API_KEY=<your Anthropic API key> (or other LLM platform key)
+VOYAGEAI_API_KEY=<your Voyage AI API key>
+DATABASE_URL=postgresql://<user>:<password>@localhost:5432/kongruity
+EOF
 ```

-Replace `<your Anthropic API key>` with your actual key.
+Replace placeholder values with your actual keys and database credentials.

-### 3. Install dependencies
+### 3. Set up/run the database
+
+Start DB  for local development (assumes local dev env MacOS and Homebrew installed)
+
+```bash
+brew services start postgresql@15
+```
+
+Create a PostgreSQL database for the project:
+
+```bash
+createdb kongruity
+```
+
+Run the migration to create tables:
+
+```bash
+cd backend
+npm run db:migrate
+```
+
+Seed the database with the sample sticky notes:
+
+```bash
+npm run db:seed
+```
+
+### 4. Install dependencies

 ```bash
 cd backend
@@ -48,7 +96,7 @@ npm install

 ## Run the app

-### Start the backend - Production Mode
+### Start the backend — Production mode

 From the `backend/` directory:

@@ -58,15 +106,15 @@ npm run start

 The API server starts on **http://localhost:3001** (configurable via the `PORT` environment variable).

-### Start the backend - Development mode
+### Start the backend — Development mode

-To start using Nodemon for "hot reloads," if developing your own features:
+To start using Nodemon for hot reloads while developing:

 ```bash
 npm run dev
 ```

-### Build the frontend - Production mode
+### Build the frontend — Production mode

 From the `frontend/` directory:

@@ -74,13 +122,14 @@ From the `frontend/` directory:
 npm run build
 ```

-### Start the frontend - Development mode
+### Start the frontend — Development mode

 From the `frontend/` directory:

 ```bash
 npm run dev
 ```
+
 The Vite dev server starts on **http://localhost:5173** by default. Open that URL in a browser.

 ## Running tests
@@ -92,7 +141,8 @@ From the `backend/` directory:
 ```bash
 npm test
 ```
-Backend tests use Supertest for HTTP assertions.
+
+Backend tests use Vitest with Supertest for HTTP assertions.

 ### Frontend tests

@@ -102,8 +152,10 @@ From the `frontend/` directory:
 npm test
 ```

-This runs `vitest run` with jsdom. For watch mode, during development:
+This runs Vitest with jsdom. For watch mode during development:

 ```bash
 npm run test:watch
 ```
+
+
--- a/backend/app.js
+++ b/backend/app.js
@@ -6,11 +6,10 @@ const app = express();

 app.use(cors());
 app.use(express.json());
-
 app.use('/v1/notes', router);

 app.use((req, res) => {
  res.status(404).json({ error: `Requested path is invalid or does not exist: ${req.method} ${req.originalUrl}` });
- });
+});

 export default app;
--- a/backend/data/notes.json
+++ b/backend/data/notes.json
@@ -1,402 +0,0 @@
-[
-  {
-    "id": "note_038",
-    "text": "Live follow mode frequently breaks",
-    "x": 710,
-    "y": 579,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_005",
-    "text": "SSO login loops back to the sign-in page",
-    "x": 184,
-    "y": 124,
-    "author": "user_7",
-    "color": "yellow"
-  },
-  {
-    "id": "note_023",
-    "text": "Undo/redo sometimes lags and applies late",
-    "x": 236,
-    "y": 687,
-    "author": "user_9",
-    "color": "purple"
-  },
-  {
-    "id": "note_017",
-    "text": "Exported file names are inconsistent and hard to track",
-    "x": 676,
-    "y": 201,
-    "author": "user_4",
-    "color": "blue"
-  },
-  {
-    "id": "note_049",
-    "text": "Need better controls for aligning and distributing notes",
-    "x": 462,
-    "y": 487,
-    "author": "user_7",
-    "color": "green"
-  },
-  {
-    "id": "note_010",
-    "text": "OAuth consent screen appears every time I sign in",
-    "x": 228,
-    "y": 124,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_036",
-    "text": "Cursor presence is distracting and overlaps content",
-    "x": 788,
-    "y": 605,
-    "author": "user_5",
-    "color": "yellow"
-  },
-  {
-    "id": "note_011",
-    "text": "Need better export options (PDF quality is too low)",
-    "x": 748,
-    "y": 212,
-    "author": "user_9",
-    "color": "green"
-  },
-  {
-    "id": "note_012",
-    "text": "Exported PDF cuts off content near the edges",
-    "x": 733,
-    "y": 159,
-    "author": "user_6",
-    "color": "blue"
-  },
-  {
-    "id": "note_019",
-    "text": "Embedded exports don’t update when the board changes",
-    "x": 662,
-    "y": 132,
-    "author": "user_1",
-    "color": "blue"
-  },
-  {
-    "id": "note_020",
-    "text": "Can’t export comments and reactions with the content",
-    "x": 739,
-    "y": 139,
-    "author": "user_7",
-    "color": "green"
-  },
-  {
-    "id": "note_039",
-    "text": "Guest collaborators can’t see updates without refreshing",
-    "x": 759,
-    "y": 711,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_024",
-    "text": "App crashes when opening a large board",
-    "x": 239,
-    "y": 597,
-    "author": "user_10",
-    "color": "purple"
-  },
-  {
-    "id": "note_041",
-    "text": "Hard to find the right template quickly",
-    "x": 536,
-    "y": 394,
-    "author": "user_4",
-    "color": "orange"
-  },
-  {
-    "id": "note_040",
-    "text": "Activity feed lacks context about what changed",
-    "x": 710,
-    "y": 771,
-    "author": "user_7",
-    "color": "yellow"
-  },
-  {
-    "id": "note_014",
-    "text": "Export takes too long and sometimes never finishes",
-    "x": 798,
-    "y": 211,
-    "author": "user_2",
-    "color": "green"
-  },
-  {
-    "id": "note_031",
-    "text": "Hard to tell who is editing what in real time",
-    "x": 761,
-    "y": 704,
-    "author": "user_8",
-    "color": "yellow"
-  },
-  {
-    "id": "note_032",
-    "text": "Comments get lost — no clear thread view",
-    "x": 818,
-    "y": 641,
-    "author": "user_5",
-    "color": "yellow"
-  },
-  {
-    "id": "note_050",
-    "text": "Can’t lock sections to prevent accidental edits during workshops",
-    "x": 521,
-    "y": 471,
-    "author": "user_6",
-    "color": "orange"
-  },
-  {
-    "id": "note_001",
-    "text": "Login flow feels confusing",
-    "x": 193,
-    "y": 191,
-    "author": "user_5",
-    "color": "yellow"
-  },
-  {
-    "id": "note_025",
-    "text": "Saving indicator spins but changes aren’t saved",
-    "x": 129,
-    "y": 781,
-    "author": "user_3",
-    "color": "purple"
-  },
-  {
-    "id": "note_021",
-    "text": "Board feels slow when there are many sticky notes",
-    "x": 341,
-    "y": 695,
-    "author": "user_10",
-    "color": "purple"
-  },
-  {
-    "id": "note_002",
-    "text": "Login flow is broken on mobile",
-    "x": 214,
-    "y": 281,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_046",
-    "text": "Tags are missing — I need better categorization",
-    "x": 472,
-    "y": 435,
-    "author": "user_6",
-    "color": "green"
-  },
-  {
-    "id": "note_018",
-    "text": "No way to schedule recurring exports for stakeholders",
-    "x": 661,
-    "y": 229,
-    "author": "user_9",
-    "color": "blue"
-  },
-  {
-    "id": "note_034",
-    "text": "Too many notification emails for minor edits",
-    "x": 769,
-    "y": 739,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_003",
-    "text": "When I enter my username and password I get an unknown error",
-    "x": 189,
-    "y": 193,
-    "author": "user_2",
-    "color": "yellow"
-  },
-  {
-    "id": "note_008",
-    "text": "Cannot change my password — save button does nothing",
-    "x": 244,
-    "y": 188,
-    "author": "user_2",
-    "color": "orange"
-  },
-  {
-    "id": "note_015",
-    "text": "Sharing link permissions are confusing",
-    "x": 688,
-    "y": 290,
-    "author": "user_6",
-    "color": "blue"
-  },
-  {
-    "id": "note_033",
-    "text": "Mentions (@) don’t notify the right people",
-    "x": 696,
-    "y": 669,
-    "author": "user_5",
-    "color": "yellow"
-  },
-  {
-    "id": "note_006",
-    "text": "Two-factor code is rejected even when correct",
-    "x": 162,
-    "y": 213,
-    "author": "user_1",
-    "color": "yellow"
-  },
-  {
-    "id": "note_028",
-    "text": "High CPU usage even when idle on a board",
-    "x": 190,
-    "y": 695,
-    "author": "user_8",
-    "color": "purple"
-  },
-  {
-    "id": "note_009",
-    "text": "Account gets locked too quickly after one failed attempt",
-    "x": 280,
-    "y": 255,
-    "author": "user_10",
-    "color": "orange"
-  },
-  {
-    "id": "note_029",
-    "text": "Offline mode loses edits when reconnecting",
-    "x": 338,
-    "y": 632,
-    "author": "user_1",
-    "color": "pink"
-  },
-  {
-    "id": "note_035",
-    "text": "No notification when someone resolves my comment",
-    "x": 673,
-    "y": 636,
-    "author": "user_2",
-    "color": "blue"
-  },
-  {
-    "id": "note_037",
-    "text": "Can’t easily hand off facilitation to another user",
-    "x": 816,
-    "y": 707,
-    "author": "user_2",
-    "color": "yellow"
-  },
-  {
-    "id": "note_013",
-    "text": "Cannot export selected area — only full board exports",
-    "x": 696,
-    "y": 205,
-    "author": "user_4",
-    "color": "green"
-  },
-  {
-    "id": "note_016",
-    "text": "Downloaded image is blurry compared to the canvas",
-    "x": 677,
-    "y": 245,
-    "author": "user_5",
-    "color": "blue"
-  },
-  {
-    "id": "note_043",
-    "text": "Can’t organize boards into folders the way I need",
-    "x": 504,
-    "y": 398,
-    "author": "user_4",
-    "color": "green"
-  },
-  {
-    "id": "note_022",
-    "text": "Canvas freezes for a few seconds when zooming",
-    "x": 254,
-    "y": 707,
-    "author": "user_8",
-    "color": "pink"
-  },
-  {
-    "id": "note_026",
-    "text": "Search is slow on boards with lots of content",
-    "x": 253,
-    "y": 658,
-    "author": "user_2",
-    "color": "pink"
-  },
-  {
-    "id": "note_048",
-    "text": "Duplicating a board loses some formatting",
-    "x": 382,
-    "y": 410,
-    "author": "user_10",
-    "color": "orange"
-  },
-  {
-    "id": "note_007",
-    "text": "I’m logged out unexpectedly after a few minutes",
-    "x": 185,
-    "y": 157,
-    "author": "user_3",
-    "color": "yellow"
-  },
-  {
-    "id": "note_045",
-    "text": "No bulk rename for multiple sticky notes",
-    "x": 455,
-    "y": 427,
-    "author": "user_1",
-    "color": "green"
-  },
-  {
-    "id": "note_042",
-    "text": "Template search results feel irrelevant",
-    "x": 400,
-    "y": 474,
-    "author": "user_6",
-    "color": "orange"
-  },
-  {
-    "id": "note_044",
-    "text": "Naming conventions aren’t enforced and things get messy",
-    "x": 469,
-    "y": 434,
-    "author": "user_9",
-    "color": "green"
-  },
-  {
-    "id": "note_004",
-    "text": "Password reset email never arrives",
-    "x": 207,
-    "y": 267,
-    "author": "user_9",
-    "color": "yellow"
-  },
-  {
-    "id": "note_047",
-    "text": "Hard to keep consistent styles across boards",
-    "x": 420,
-    "y": 426,
-    "author": "user_8",
-    "color": "green"
-  },
-  {
-    "id": "note_030",
-    "text": "I see random 'something went wrong' banners with no details",
-    "x": 214,
-    "y": 594,
-    "author": "user_5",
-    "color": "purple"
-  },
-  {
-    "id": "note_027",
-    "text": "Scrolling stutters on older laptops",
-    "x": 178,
-    "y": 586,
-    "author": "user_7",
-    "color": "pink"
-  }
-]
--- a/backend/db/README.md
+++ b/backend/db/README.md
@@ -0,0 +1,131 @@
+# Database — kongruity backend
+
+PostgreSQL database layer for the kongruity backend. All database modules live in this directory.
+
+## Prerequisites
+
+- PostgreSQL v14 or later
+- The `DATABASE_URL` environment variable set in `backend/.env`
+
+## Configuration
+
+### Environment variable
+
+The connection pool reads a single env var:
+
+```
+DATABASE_URL=postgresql://<user>:<password>@localhost:5432/kongruity
+```
+
+This is loaded via `dotenv` from `backend/.env` (git-ignored). The pool is created once in `index.js` and shared across the application.
+
+### Connection pool (`index.js`)
+
+| Export      | Description                                      |
+|-------------|--------------------------------------------------|
+| `query`     | Execute a parameterized SQL statement             |
+| `getPool`   | Return the underlying `pg.Pool` instance          |
+| `close`     | Gracefully shut down the pool (`pool.end()`)      |
+| `default`   | The pool itself (default export)                  |
+
+## Schema
+
+### `notes` table
+
+Created by the migration in `migrate.js`. The DDL is idempotent (`CREATE TABLE IF NOT EXISTS`).
+
+| Column        | Type           | Constraints / Defaults       |
+|---------------|----------------|------------------------------|
+| `id`          | `VARCHAR(64)`  | `PRIMARY KEY`                |
+| `text`        | `TEXT`         | `NOT NULL`                   |
+| `x`           | `INTEGER`      | `DEFAULT 0`                  |
+| `y`           | `INTEGER`      | `DEFAULT 0`                  |
+| `author`      | `VARCHAR(128)` |                              |
+| `color`       | `VARCHAR(32)`  | `DEFAULT 'yellow'`           |
+| `source_meta` | `JSONB`        | `DEFAULT '{}'`               |
+| `created_at`  | `TIMESTAMPTZ`  | `DEFAULT NOW()`              |
+| `updated_at`  | `TIMESTAMPTZ`  | `DEFAULT NOW()`              |
+
+## npm scripts
+
+Run these from the `backend/` directory.
+
+### Migrate
+
+Creates the `notes` table (safe to re-run):
+
+```bash
+npm run db:migrate
+```
+
+### Seed
+
+Inserts 50 sample sticky notes. Uses `ON CONFLICT (id) DO NOTHING`, so re-running is safe and will not duplicate data:
+
+```bash
+npm run db:seed
+```
+
+## Data access layer (`notes.dao.js`)
+
+| Function                  | Description                                                        |
+|---------------------------|--------------------------------------------------------------------|
+| `getAllNotes()`           | Returns all notes ordered by `id`                                   |
+| `getNoteById(id)`        | Returns a single note by primary key, or `null` if not found        |
+| `createNote(note)`       | Inserts one note and returns the created row                        |
+| `createNotes(notes)`     | Bulk-inserts an array of notes in a single query and returns rows   |
+
+All functions return plain objects with columns: `id`, `text`, `x`, `y`, `author`, `color`.
+
+## File overview
+
+```
+db/
+├── index.js         # Connection pool and query helper
+├── migrate.js       # Table creation (run via npm run db:migrate)
+├── notes.dao.js     # Data access functions for the notes table
+├── seed.js          # Sample data seeder (run via npm run db:seed)
+└── README.md        # This file
+```
+
+## Useful psql commands
+
+Connect to the database:
+
+```bash
+psql kongruity
+```
+
+Quick checks:
+
+```sql
+-- Row count
+SELECT count(*) FROM notes;
+
+-- Preview data
+SELECT id, text, color FROM notes LIMIT 10;
+
+-- Full schema info
+\d notes
+
+-- Drop and re-seed (destructive)
+TRUNCATE notes;
+```
+
+Then re-seed:
+
+```bash
+npm run db:seed
+```
+
+## Resetting the database
+
+To start completely fresh:
+
+```bash
+dropdb kongruity
+createdb kongruity
+cd backend
+npm run db:migrate
+npm run db:seed
+```
--- a/backend/db/fixtures/notes.jsonl
+++ b/backend/db/fixtures/notes.jsonl
@@ -0,0 +1,50 @@
+{"id":"note_001","text":"Login flow feels confusing","x":193,"y":191,"author":"user_5","color":"yellow"}
+{"id":"note_002","text":"Login flow is broken on mobile","x":214,"y":281,"author":"user_9","color":"yellow"}
+{"id":"note_003","text":"When I enter my username and password I get an unknown error","x":189,"y":193,"author":"user_2","color":"yellow"}
+{"id":"note_004","text":"Password reset email never arrives","x":207,"y":267,"author":"user_9","color":"yellow"}
+{"id":"note_005","text":"SSO login loops back to the sign-in page","x":184,"y":124,"author":"user_7","color":"yellow"}
+{"id":"note_006","text":"Two-factor code is rejected even when correct","x":162,"y":213,"author":"user_1","color":"yellow"}
+{"id":"note_007","text":"I'm logged out unexpectedly after a few minutes","x":185,"y":157,"author":"user_3","color":"yellow"}
+{"id":"note_008","text":"Cannot change my password — save button does nothing","x":244,"y":188,"author":"user_2","color":"orange"}
+{"id":"note_009","text":"Account gets locked too quickly after one failed attempt","x":280,"y":255,"author":"user_10","color":"orange"}
+{"id":"note_010","text":"OAuth consent screen appears every time I sign in","x":228,"y":124,"author":"user_9","color":"yellow"}
+{"id":"note_011","text":"Need better export options (PDF quality is too low)","x":748,"y":212,"author":"user_9","color":"green"}
+{"id":"note_012","text":"Exported PDF cuts off content near the edges","x":733,"y":159,"author":"user_6","color":"blue"}
+{"id":"note_013","text":"Cannot export selected area — only full board exports","x":696,"y":205,"author":"user_4","color":"green"}
+{"id":"note_014","text":"Export takes too long and sometimes never finishes","x":798,"y":211,"author":"user_2","color":"green"}
+{"id":"note_015","text":"Sharing link permissions are confusing","x":688,"y":290,"author":"user_6","color":"blue"}
+{"id":"note_016","text":"Downloaded image is blurry compared to the canvas","x":677,"y":245,"author":"user_5","color":"blue"}
+{"id":"note_017","text":"Exported file names are inconsistent and hard to track","x":676,"y":201,"author":"user_4","color":"blue"}
+{"id":"note_018","text":"No way to schedule recurring exports for stakeholders","x":661,"y":229,"author":"user_9","color":"blue"}
+{"id":"note_019","text":"Embedded exports don't update when the board changes","x":662,"y":132,"author":"user_1","color":"blue"}
+{"id":"note_020","text":"Can't export comments and reactions with the content","x":739,"y":139,"author":"user_7","color":"green"}
+{"id":"note_021","text":"Board feels slow when there are many sticky notes","x":341,"y":695,"author":"user_10","color":"purple"}
+{"id":"note_022","text":"Canvas freezes for a few seconds when zooming","x":254,"y":707,"author":"user_8","color":"pink"}
+{"id":"note_023","text":"Undo/redo sometimes lags and applies late","x":236,"y":687,"author":"user_9","color":"purple"}
+{"id":"note_024","text":"App crashes when opening a large board","x":239,"y":597,"author":"user_10","color":"purple"}
+{"id":"note_025","text":"Saving indicator spins but changes aren't saved","x":129,"y":781,"author":"user_3","color":"purple"}
+{"id":"note_026","text":"Search is slow on boards with lots of content","x":253,"y":658,"author":"user_2","color":"pink"}
+{"id":"note_027","text":"Scrolling stutters on older laptops","x":178,"y":586,"author":"user_7","color":"pink"}
+{"id":"note_028","text":"High CPU usage even when idle on a board","x":190,"y":695,"author":"user_8","color":"purple"}
+{"id":"note_029","text":"Offline mode loses edits when reconnecting","x":338,"y":632,"author":"user_1","color":"pink"}
+{"id":"note_030","text":"I see random 'something went wrong' banners with no details","x":214,"y":594,"author":"user_5","color":"purple"}
+{"id":"note_031","text":"Hard to tell who is editing what in real time","x":761,"y":704,"author":"user_8","color":"yellow"}
+{"id":"note_032","text":"Comments get lost — no clear thread view","x":818,"y":641,"author":"user_5","color":"yellow"}
+{"id":"note_033","text":"Mentions (@) don't notify the right people","x":696,"y":669,"author":"user_5","color":"yellow"}
+{"id":"note_034","text":"Too many notification emails for minor edits","x":769,"y":739,"author":"user_9","color":"yellow"}
+{"id":"note_035","text":"No notification when someone resolves my comment","x":673,"y":636,"author":"user_2","color":"blue"}
+{"id":"note_036","text":"Cursor presence is distracting and overlaps content","x":788,"y":605,"author":"user_5","color":"yellow"}
+{"id":"note_037","text":"Can't easily hand off facilitation to another user","x":816,"y":707,"author":"user_2","color":"yellow"}
+{"id":"note_038","text":"Live follow mode frequently breaks","x":710,"y":579,"author":"user_9","color":"yellow"}
+{"id":"note_039","text":"Guest collaborators can't see updates without refreshing","x":759,"y":711,"author":"user_9","color":"yellow"}
+{"id":"note_040","text":"Activity feed lacks context about what changed","x":710,"y":771,"author":"user_7","color":"yellow"}
+{"id":"note_041","text":"Hard to find the right template quickly","x":536,"y":394,"author":"user_4","color":"orange"}
+{"id":"note_042","text":"Template search results feel irrelevant","x":400,"y":474,"author":"user_6","color":"orange"}
+{"id":"note_043","text":"Can't organize boards into folders the way I need","x":504,"y":398,"author":"user_4","color":"green"}
+{"id":"note_044","text":"Naming conventions aren't enforced and things get messy","x":469,"y":434,"author":"user_9","color":"green"}
+{"id":"note_045","text":"No bulk rename for multiple sticky notes","x":455,"y":427,"author":"user_1","color":"green"}
+{"id":"note_046","text":"Tags are missing — I need better categorization","x":472,"y":435,"author":"user_6","color":"green"}
+{"id":"note_047","text":"Hard to keep consistent styles across boards","x":420,"y":426,"author":"user_8","color":"green"}
+{"id":"note_048","text":"Duplicating a board loses some formatting","x":382,"y":410,"author":"user_10","color":"orange"}
+{"id":"note_049","text":"Need better controls for aligning and distributing notes","x":462,"y":487,"author":"user_7","color":"green"}
+{"id":"note_050","text":"Can't lock sections to prevent accidental edits during workshops","x":521,"y":471,"author":"user_6","color":"orange"}
--- a/backend/db/index.js
+++ b/backend/db/index.js
@@ -0,0 +1,13 @@
+import pg from 'pg';
+
+const pool = new pg.Pool({
+  connectionString: process.env.DATABASE_URL,
+});
+
+export const query = (text, params) => pool.query(text, params);
+
+export const getPool = () => pool;
+
+export const close = () => pool.end();
+
+export default pool;
--- a/backend/db/migrate.js
+++ b/backend/db/migrate.js
@@ -0,0 +1,30 @@
+import 'dotenv/config';
+import { query, close } from './index.js';
+
+const up = `
+CREATE TABLE IF NOT EXISTS notes (
+  id          VARCHAR(64) PRIMARY KEY,
+  text        TEXT NOT NULL,
+  x           INTEGER DEFAULT 0,
+  y           INTEGER DEFAULT 0,
+  author      VARCHAR(128),
+  color       VARCHAR(32) DEFAULT 'yellow',
+  source_meta JSONB DEFAULT '{}',
+  created_at  TIMESTAMPTZ DEFAULT NOW(),
+  updated_at  TIMESTAMPTZ DEFAULT NOW()
+);
+`;
+
+const run = async () => {
+  try {
+    await query(up);
+    console.log('Migration complete — notes table ready.');
+  } catch (err) {
+    console.error('Migration failed:', err.message);
+    process.exit(1);
+  } finally {
+    await close();
+  }
+};
+
+run();
--- a/backend/db/notes.dao.js
+++ b/backend/db/notes.dao.js
@@ -0,0 +1,110 @@
+import QueryStream from 'pg-query-stream';
+import { pipeline } from 'node:stream/promises';
+import { Readable } from 'node:stream';
+import { query, getPool } from './index.js';
+import { batch } from '../lib/streams.js';
+
+const SELECT_NOTES = 'SELECT id, text, x, y, author, color FROM notes ORDER BY id';
+
+// Postgres caps a statement at 65535 bind parameters; six columns per note
+// leaves 10922 as the hard ceiling.
+const INSERT_BATCH_SIZE = 1000;
+
+export const getAllNotes = async () => {
+    const { rows } = await query(SELECT_NOTES);
+    return rows;
+};
+
+/**
+ * Streams every note as an object-mode Readable. The pooled client is released on end, error, or
+ * destruction by consumer.
+ *
+ * @returns {Promise<import('node:stream').Readable>}
+ */
+export const streamAllNotes = async () => {
+    const client = await getPool().connect();
+
+    let released = false;
+    const release = () => {
+        if (released) return;
+        released = true;
+        client.release();
+    };
+
+    try {
+        const rows = client.query(new QueryStream(SELECT_NOTES));
+        rows.once('end', release);
+        rows.once('error', release);
+        rows.once('close', release);
+        return rows;
+    } catch (err) {
+        release();
+        throw err;
+    }
+};
+
+export const getNoteById = async (id) => {
+    const { rows } = await query(
+        'SELECT id, text, x, y, author, color FROM notes WHERE id = $1',
+        [id]
+    );
+    return rows[0] || null;
+};
+
+export const createNote = async (note) => {
+    const { rows } = await query(
+        `INSERT INTO notes (id, text, x, y, author, color)
+     VALUES ($1, $2, $3, $4, $5, $6)
+     RETURNING id, text, x, y, author, color`,
+        [note.id, note.text, note.x ?? 0, note.y ?? 0, note.author, note.color ?? 'yellow']
+    );
+    return rows[0];
+};
+
+const insertNoteBatch = async (notes) => {
+    const values = [];
+    const placeholders = [];
+
+    notes.forEach((note, i) => {
+        const offset = i * 6;
+        placeholders.push(
+            `($${offset + 1}, $${offset + 2}, $${offset + 3}, $${offset + 4}, $${offset + 5}, $${offset + 6})`
+        );
+        values.push(
+            note.id,
+            note.text,
+            note.x ?? 0,
+            note.y ?? 0,
+            note.author,
+            note.color ?? 'yellow'
+        );
+    });
+
+    const { rows } = await query(
+        `INSERT INTO notes (id, text, x, y, author, color)
+     VALUES ${placeholders.join(', ')}
+     RETURNING id, text, x, y, author, color`,
+        values
+    );
+    return rows;
+};
+
+export const createNotes = async (notes) => {
+    if (!notes || notes.length === 0) {
+        return [];
+    }
+
+    const inserted = [];
+
+    await pipeline(
+        Readable.from(notes, { objectMode: true }),
+        batch(INSERT_BATCH_SIZE),
+        async (batches) => {
+            for await (const chunk of batches) {
+                inserted.push(...await insertNoteBatch(chunk));
+            }
+        }
+    );
+
+    return inserted;
+};
--- a/backend/db/seed.js
+++ b/backend/db/seed.js
@@ -0,0 +1,80 @@
+import 'dotenv/config';
+import split2 from 'split2';
+import { from as copyFrom } from 'pg-copy-streams';
+import { createReadStream } from 'node:fs';
+import { Transform } from 'node:stream';
+import { pipeline } from 'node:stream/promises';
+import { fileURLToPath } from 'node:url';
+import { getPool, close } from './index.js';
+
+const FIXTURE = fileURLToPath(new URL('./fixtures/notes.jsonl', import.meta.url));
+
+const COLUMNS = ['id', 'text', 'x', 'y', 'author', 'color'];
+
+const DEFAULTS = { x: 0, y: 0, color: 'yellow' };
+
+const csvField = (value) => {
+  if (value === null || value === undefined) return '';
+  return `"${String(value).replaceAll('"', '""')}"`;
+};
+
+const toCsvRows = () => new Transform({
+  writableObjectMode: true,
+  transform(line, _encoding, callback) {
+    if (line.trim().length === 0) {
+      callback();
+      return;
+    }
+
+    let note;
+    try {
+      note = JSON.parse(line);
+    } catch {
+      callback(new Error(`Fixture contains a malformed JSON line: ${line.slice(0, 80)}`));
+      return;
+    }
+
+    const row = COLUMNS.map((col) => csvField(note[col] ?? DEFAULTS[col]));
+    callback(null, `${row.join(',')}\n`);
+  },
+});
+
+const run = async () => {
+  const client = await getPool().connect();
+
+  try {
+    await client.query('BEGIN');
+
+    // COPY has no ON CONFLICT, so land the fixture in a temp table first and
+    // let a single INSERT ... SELECT apply the existing idempotency.
+    await client.query(
+      'CREATE TEMP TABLE notes_import (LIKE notes INCLUDING DEFAULTS) ON COMMIT DROP'
+    );
+
+    const copy = client.query(
+      copyFrom(`COPY notes_import (${COLUMNS.join(', ')}) FROM STDIN WITH (FORMAT csv)`)
+    );
+
+    await pipeline(createReadStream(FIXTURE), split2(), toCsvRows(), copy);
+
+    const { rowCount: staged } = await client.query('SELECT 1 FROM notes_import');
+    const { rowCount: inserted } = await client.query(
+      `INSERT INTO notes (${COLUMNS.join(', ')})
+       SELECT ${COLUMNS.join(', ')} FROM notes_import
+       ON CONFLICT (id) DO NOTHING`
+    );
+
+    await client.query('COMMIT');
+
+    console.log(`Seed complete — ${inserted} notes inserted (${staged - inserted} already existed).`);
+  } catch (err) {
+    await client.query('ROLLBACK').catch(() => {});
+    console.error('Seed failed:', err.message);
+    process.exitCode = 1;
+  } finally {
+    client.release();
+    await close();
+  }
+};
+
+run();
--- a/backend/lib/streams.js
+++ b/backend/lib/streams.js
@@ -0,0 +1,72 @@
+import { Transform } from 'node:stream';
+
+/**
+ * Groups an object-mode stream into fixed-size arrays. Backpressure on the
+ * readable side is what limits how many batches are ever in flight.
+ *
+ * @param {number} size - maximum items per emitted batch
+ * @returns {Transform}
+ */
+export const batch = (size) => {
+  if (!Number.isInteger(size) || size < 1) {
+    throw new TypeError('batch(size) requires a positive integer size');
+  }
+
+  let pending = [];
+
+  return new Transform({
+    objectMode: true,
+    transform(item, _encoding, callback) {
+      pending.push(item);
+
+      if (pending.length < size) {
+        callback();
+        return;
+      }
+
+      const full = pending;
+      pending = [];
+      callback(null, full);
+    },
+    flush(callback) {
+      if (pending.length === 0) {
+        callback();
+        return;
+      }
+
+      const remainder = pending;
+      pending = [];
+      callback(null, remainder);
+    },
+  });
+};
+
+/**
+ * Serializes an object-mode stream into a JSON array, one element at a time,
+ * so no complete copy of the payload is ever held in memory.
+ *
+ * @returns {Transform}
+ */
+export const jsonArray = () => {
+  let wroteFirst = false;
+
+  return new Transform({
+    writableObjectMode: true,
+    transform(item, _encoding, callback) {
+      let serialized;
+      try {
+        serialized = JSON.stringify(item);
+      } catch (err) {
+        callback(err);
+        return;
+      }
+
+      const prefix = wroteFirst ? ',' : '[';
+      wroteFirst = true;
+      callback(null, prefix + serialized);
+    },
+    flush(callback) {
+      callback(null, wroteFirst ? ']' : '[]');
+    },
+  });
+};
--- a/backend/package-lock.json
+++ b/backend/package-lock.json
@@ -11,7 +11,11 @@
                "@anthropic-ai/sdk": "^0.74.0",
                "cors": "^2.8.5",
                "dotenv": "^16.4.7",
-                "express": "^4.21.2"
+                "express": "^4.21.2",
+                "pg": "^8.18.0",
+                "pg-copy-streams": "^7.0.0",
+                "pg-query-stream": "^4.16.0",
+                "voyageai": "^0.1.0"
            },
            "devDependencies": {
                "nodemon": "^3.1.9",
@@ -1013,6 +1017,18 @@
                "url": "https://opencollective.com/vitest"
            }
        },
+        "node_modules/abort-controller": {
+            "version": "3.0.0",
+            "resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
+            "integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==",
+            "license": "MIT",
+            "dependencies": {
+                "event-target-shim": "^5.0.0"
+            },
+            "engines": {
+                "node": ">=6.5"
+            }
+        },
        "node_modules/accepts": {
            "version": "1.3.8",
            "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
@@ -1067,7 +1083,6 @@
            "version": "0.4.0",
            "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
            "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
-            "dev": true,
            "license": "MIT"
        },
        "node_modules/balanced-match": {
@@ -1077,6 +1092,26 @@
            "dev": true,
            "license": "MIT"
        },
+        "node_modules/base64-js": {
+            "version": "1.5.1",
+            "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
+            "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
+            "funding": [
+                {
+                    "type": "github",
+                    "url": "https://github.com/sponsors/feross"
+                },
+                {
+                    "type": "patreon",
+                    "url": "https://www.patreon.com/feross"
+                },
+                {
+                    "type": "consulting",
+                    "url": "https://feross.org/support"
+                }
+            ],
+            "license": "MIT"
+        },
        "node_modules/binary-extensions": {
            "version": "2.3.0",
            "resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.3.0.tgz",
@@ -1138,6 +1173,30 @@
                "node": ">=8"
            }
        },
+        "node_modules/buffer": {
+            "version": "6.0.3",
+            "resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz",
+            "integrity": "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==",
+            "funding": [
+                {
+                    "type": "github",
+                    "url": "https://github.com/sponsors/feross"
+                },
+                {
+                    "type": "patreon",
+                    "url": "https://www.patreon.com/feross"
+                },
+                {
+                    "type": "consulting",
+                    "url": "https://feross.org/support"
+                }
+            ],
+            "license": "MIT",
+            "dependencies": {
+                "base64-js": "^1.3.1",
+                "ieee754": "^1.2.1"
+            }
+        },
        "node_modules/bytes": {
            "version": "3.1.2",
            "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz",
@@ -1215,7 +1274,6 @@
            "version": "1.0.8",
            "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
            "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
-            "dev": true,
            "license": "MIT",
            "dependencies": {
                "delayed-stream": "~1.0.0"
@@ -1314,7 +1372,6 @@
            "version": "1.0.0",
            "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
            "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
-            "dev": true,
            "license": "MIT",
            "engines": {
                "node": ">=0.4.0"
@@ -1432,7 +1489,6 @@
            "version": "2.1.0",
            "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
            "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
-            "dev": true,
            "license": "MIT",
            "dependencies": {
                "es-errors": "^1.3.0",
@@ -1511,6 +1567,24 @@
                "node": ">= 0.6"
            }
        },
+        "node_modules/event-target-shim": {
+            "version": "5.0.1",
+            "resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz",
+            "integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=6"
+            }
+        },
+        "node_modules/events": {
+            "version": "3.3.0",
+            "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
+            "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=0.8.x"
+            }
+        },
        "node_modules/expect-type": {
            "version": "1.3.0",
            "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.3.0.tgz",
@@ -1609,7 +1683,6 @@
            "version": "4.0.5",
            "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz",
            "integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==",
-            "dev": true,
            "license": "MIT",
            "dependencies": {
                "asynckit": "^0.4.0",
@@ -1622,6 +1695,15 @@
                "node": ">= 6"
            }
        },
+        "node_modules/formdata-node": {
+            "version": "6.0.3",
+            "resolved": "https://registry.npmjs.org/formdata-node/-/formdata-node-6.0.3.tgz",
+            "integrity": "sha512-8e1++BCiTzUno9v5IZ2J6bv4RU+3UKDmqWUQD0MIMVCd9AdhWkO1gw57oo1mNEX1dMq2EGI+FbWz4B92pscSQg==",
+            "license": "MIT",
+            "engines": {
+                "node": ">= 18"
+            }
+        },
        "node_modules/formidable": {
            "version": "3.5.4",
            "resolved": "https://registry.npmjs.org/formidable/-/formidable-3.5.4.tgz",
@@ -1770,7 +1852,6 @@
            "version": "1.0.2",
            "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
            "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
-            "dev": true,
            "license": "MIT",
            "dependencies": {
                "has-symbols": "^1.0.3"
@@ -1826,6 +1907,26 @@
                "node": ">=0.10.0"
            }
        },
+        "node_modules/ieee754": {
+            "version": "1.2.1",
+            "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
+            "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
+            "funding": [
+                {
+                    "type": "github",
+                    "url": "https://github.com/sponsors/feross"
+                },
+                {
+                    "type": "patreon",
+                    "url": "https://www.patreon.com/feross"
+                },
+                {
+                    "type": "consulting",
+                    "url": "https://feross.org/support"
+                }
+            ],
+            "license": "BSD-3-Clause"
+        },
        "node_modules/ignore-by-default": {
            "version": "1.0.1",
            "resolved": "https://registry.npmjs.org/ignore-by-default/-/ignore-by-default-1.0.1.tgz",
@@ -1894,6 +1995,12 @@
                "node": ">=0.12.0"
            }
        },
+        "node_modules/js-base64": {
+            "version": "3.7.2",
+            "resolved": "https://registry.npmjs.org/js-base64/-/js-base64-3.7.2.tgz",
+            "integrity": "sha512-NnRs6dsyqUXejqk/yv2aiXlAvOs56sLkX6nUdeaNezI5LFFLlsZjOThmwnrcwh5ZZRwZlCMnVAY3CvhIhoVEKQ==",
+            "license": "BSD-3-Clause"
+        },
        "node_modules/json-schema-to-ts": {
            "version": "3.1.1",
            "resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
@@ -2033,6 +2140,26 @@
                "node": ">= 0.6"
            }
        },
+        "node_modules/node-fetch": {
+            "version": "2.7.0",
+            "resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz",
+            "integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==",
+            "license": "MIT",
+            "dependencies": {
+                "whatwg-url": "^5.0.0"
+            },
+            "engines": {
+                "node": "4.x || >=6.0.0"
+            },
+            "peerDependencies": {
+                "encoding": "^0.1.0"
+            },
+            "peerDependenciesMeta": {
+                "encoding": {
+                    "optional": true
+                }
+            }
+        },
        "node_modules/nodemon": {
            "version": "3.1.11",
            "resolved": "https://registry.npmjs.org/nodemon/-/nodemon-3.1.11.tgz",
@@ -2173,6 +2300,123 @@
            "dev": true,
            "license": "MIT"
        },
+        "node_modules/pg": {
+            "version": "8.18.0",
+            "resolved": "https://registry.npmjs.org/pg/-/pg-8.18.0.tgz",
+            "integrity": "sha512-xqrUDL1b9MbkydY/s+VZ6v+xiMUmOUk7SS9d/1kpyQxoJ6U9AO1oIJyUWVZojbfe5Cc/oluutcgFG4L9RDP1iQ==",
+            "license": "MIT",
+            "peer": true,
+            "dependencies": {
+                "pg-connection-string": "^2.11.0",
+                "pg-pool": "^3.11.0",
+                "pg-protocol": "^1.11.0",
+                "pg-types": "2.2.0",
+                "pgpass": "1.0.5"
+            },
+            "engines": {
+                "node": ">= 16.0.0"
+            },
+            "optionalDependencies": {
+                "pg-cloudflare": "^1.3.0"
+            },
+            "peerDependencies": {
+                "pg-native": ">=3.0.1"
+            },
+            "peerDependenciesMeta": {
+                "pg-native": {
+                    "optional": true
+                }
+            }
+        },
+        "node_modules/pg-cloudflare": {
+            "version": "1.3.0",
+            "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.3.0.tgz",
+            "integrity": "sha512-6lswVVSztmHiRtD6I8hw4qP/nDm1EJbKMRhf3HCYaqud7frGysPv7FYJ5noZQdhQtN2xJnimfMtvQq21pdbzyQ==",
+            "license": "MIT",
+            "optional": true
+        },
+        "node_modules/pg-connection-string": {
+            "version": "2.11.0",
+            "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.11.0.tgz",
+            "integrity": "sha512-kecgoJwhOpxYU21rZjULrmrBJ698U2RxXofKVzOn5UDj61BPj/qMb7diYUR1nLScCDbrztQFl1TaQZT0t1EtzQ==",
+            "license": "MIT"
+        },
+        "node_modules/pg-copy-streams": {
+            "version": "7.0.0",
+            "resolved": "https://registry.npmjs.org/pg-copy-streams/-/pg-copy-streams-7.0.0.tgz",
+            "integrity": "sha512-zBvnY6wtaBRE2ae2xXWOOGMaNVPkXh1vhypAkNSKgMdciJeTyIQAHZaEeRAxUjs/p1El5jgzYmwG5u871Zj3dQ==",
+            "license": "MIT"
+        },
+        "node_modules/pg-cursor": {
+            "version": "2.21.0",
+            "resolved": "https://registry.npmjs.org/pg-cursor/-/pg-cursor-2.21.0.tgz",
+            "integrity": "sha512-IYvk/j+Suhtbo/C3uOf4JLsLK/gWxOTUOmYbDsbKnLaVJDq+KwhwK6ngpRfiCk8eDMS3AmGQABZCv0cREEzHQw==",
+            "license": "MIT",
+            "peerDependencies": {
+                "pg": "^8"
+            }
+        },
+        "node_modules/pg-int8": {
+            "version": "1.0.1",
+            "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
+            "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
+            "license": "ISC",
+            "engines": {
+                "node": ">=4.0.0"
+            }
+        },
+        "node_modules/pg-pool": {
+            "version": "3.11.0",
+            "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.11.0.tgz",
+            "integrity": "sha512-MJYfvHwtGp870aeusDh+hg9apvOe2zmpZJpyt+BMtzUWlVqbhFmMK6bOBXLBUPd7iRtIF9fZplDc7KrPN3PN7w==",
+            "license": "MIT",
+            "peerDependencies": {
+                "pg": ">=8.0"
+            }
+        },
+        "node_modules/pg-protocol": {
+            "version": "1.11.0",
+            "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.11.0.tgz",
+            "integrity": "sha512-pfsxk2M9M3BuGgDOfuy37VNRRX3jmKgMjcvAcWqNDpZSf4cUmv8HSOl5ViRQFsfARFn0KuUQTgLxVMbNq5NW3g==",
+            "license": "MIT"
+        },
+        "node_modules/pg-query-stream": {
+            "version": "4.16.0",
+            "resolved": "https://registry.npmjs.org/pg-query-stream/-/pg-query-stream-4.16.0.tgz",
+            "integrity": "sha512-vyqxAG4YVax43BCSfqcKxHSbh8YQshhVR0CjLNdo7MIM2UOqI+XsIYk0bVZJ0aKjQfjJQcGiGo9xK3hz0JcFyg==",
+            "license": "MIT",
+            "dependencies": {
+                "pg-cursor": "^2.21.0"
+            },
+            "peerDependencies": {
+                "pg": "^8"
+            }
+        },
+        "node_modules/pg-types": {
+            "version": "2.2.0",
+            "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
+            "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
+            "license": "MIT",
+            "dependencies": {
+                "pg-int8": "1.0.1",
+                "postgres-array": "~2.0.0",
+                "postgres-bytea": "~1.0.0",
+                "postgres-date": "~1.0.4",
+                "postgres-interval": "^1.1.0"
+            },
+            "engines": {
+                "node": ">=4"
+            }
+        },
+        "node_modules/pgpass": {
+            "version": "1.0.5",
+            "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
+            "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
+            "license": "MIT",
+            "dependencies": {
+                "split2": "^4.1.0"
+            }
+        },
        "node_modules/picocolors": {
            "version": "1.1.1",
            "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
@@ -2222,6 +2466,54 @@
                "node": "^10 || ^12 || >=14"
            }
        },
+        "node_modules/postgres-array": {
+            "version": "2.0.0",
+            "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
+            "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=4"
+            }
+        },
+        "node_modules/postgres-bytea": {
+            "version": "1.0.1",
+            "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
+            "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=0.10.0"
+            }
+        },
+        "node_modules/postgres-date": {
+            "version": "1.0.7",
+            "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
+            "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=0.10.0"
+            }
+        },
+        "node_modules/postgres-interval": {
+            "version": "1.2.0",
+            "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
+            "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
+            "license": "MIT",
+            "dependencies": {
+                "xtend": "^4.0.0"
+            },
+            "engines": {
+                "node": ">=0.10.0"
+            }
+        },
+        "node_modules/process": {
+            "version": "0.11.10",
+            "resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz",
+            "integrity": "sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==",
+            "license": "MIT",
+            "engines": {
+                "node": ">= 0.6.0"
+            }
+        },
        "node_modules/proxy-addr": {
            "version": "2.0.7",
            "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -2281,6 +2573,22 @@
                "node": ">= 0.8"
            }
        },
+        "node_modules/readable-stream": {
+            "version": "4.7.0",
+            "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.7.0.tgz",
+            "integrity": "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==",
+            "license": "MIT",
+            "dependencies": {
+                "abort-controller": "^3.0.0",
+                "buffer": "^6.0.3",
+                "events": "^3.3.0",
+                "process": "^0.11.10",
+                "string_decoder": "^1.3.0"
+            },
+            "engines": {
+                "node": "^12.22.0 || ^14.17.0 || >=16.0.0"
+            }
+        },
        "node_modules/readdirp": {
            "version": "3.6.0",
            "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz",
@@ -2531,6 +2839,15 @@
                "node": ">=0.10.0"
            }
        },
+        "node_modules/split2": {
+            "version": "4.2.0",
+            "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+            "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+            "license": "ISC",
+            "engines": {
+                "node": ">= 10.x"
+            }
+        },
        "node_modules/stackback": {
            "version": "0.0.2",
            "resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz",
@@ -2554,6 +2871,15 @@
            "dev": true,
            "license": "MIT"
        },
+        "node_modules/string_decoder": {
+            "version": "1.3.0",
+            "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
+            "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
+            "license": "MIT",
+            "dependencies": {
+                "safe-buffer": "~5.2.0"
+            }
+        },
        "node_modules/superagent": {
            "version": "10.3.0",
            "resolved": "https://registry.npmjs.org/superagent/-/superagent-10.3.0.tgz",
@@ -2759,6 +3085,12 @@
                "nodetouch": "bin/nodetouch.js"
            }
        },
+        "node_modules/tr46": {
+            "version": "0.0.3",
+            "resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
+            "integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==",
+            "license": "MIT"
+        },
        "node_modules/ts-algebra": {
            "version": "2.0.0",
            "resolved": "https://registry.npmjs.org/ts-algebra/-/ts-algebra-2.0.0.tgz",
@@ -2794,6 +3126,12 @@
                "node": ">= 0.8"
            }
        },
+        "node_modules/url-join": {
+            "version": "4.0.1",
+            "resolved": "https://registry.npmjs.org/url-join/-/url-join-4.0.1.tgz",
+            "integrity": "sha512-jk1+QP6ZJqyOiuEI9AEWQfju/nB2Pw466kbA0LEZljHwKeMgd9WrAEgEGxjPDD2+TNbbb37rTyhEfrCXfuKXnA==",
+            "license": "MIT"
+        },
        "node_modules/utils-merge": {
            "version": "1.0.1",
            "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
@@ -3011,6 +3349,52 @@
                "url": "https://github.com/sponsors/jonschlinkert"
            }
        },
+        "node_modules/voyageai": {
+            "version": "0.1.0",
+            "resolved": "https://registry.npmjs.org/voyageai/-/voyageai-0.1.0.tgz",
+            "integrity": "sha512-QnalSSHRigZqzKJpXLsgp3tKaKkSv2kGO4rcXQ6mr6PKHa0vRirs0jZ72olvFJrGJkbu1hHijcyr6/eI4L5aRw==",
+            "license": "MIT",
+            "dependencies": {
+                "form-data": "^4.0.0",
+                "formdata-node": "^6.0.3",
+                "js-base64": "3.7.2",
+                "node-fetch": "2.7.0",
+                "qs": "6.11.2",
+                "readable-stream": "^4.5.2",
+                "url-join": "4.0.1"
+            }
+        },
+        "node_modules/voyageai/node_modules/qs": {
+            "version": "6.11.2",
+            "resolved": "https://registry.npmjs.org/qs/-/qs-6.11.2.tgz",
+            "integrity": "sha512-tDNIz22aBzCDxLtVH++VnTfzxlfeK5CbqohpSqpJgj1Wg/cQbStNAz3NuqCs5vV+pjBsK4x4pN9HlVh7rcYRiA==",
+            "license": "BSD-3-Clause",
+            "dependencies": {
+                "side-channel": "^1.0.4"
+            },
+            "engines": {
+                "node": ">=0.6"
+            },
+            "funding": {
+                "url": "https://github.com/sponsors/ljharb"
+            }
+        },
+        "node_modules/webidl-conversions": {
+            "version": "3.0.1",
+            "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
+            "integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
+            "license": "BSD-2-Clause"
+        },
+        "node_modules/whatwg-url": {
+            "version": "5.0.0",
+            "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
+            "integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
+            "license": "MIT",
+            "dependencies": {
+                "tr46": "~0.0.3",
+                "webidl-conversions": "^3.0.0"
+            }
+        },
        "node_modules/why-is-node-running": {
            "version": "2.3.0",
            "resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz",
@@ -3034,6 +3418,15 @@
            "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
            "dev": true,
            "license": "ISC"
+        },
+        "node_modules/xtend": {
+            "version": "4.0.2",
+            "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+            "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+            "license": "MIT",
+            "engines": {
+                "node": ">=0.4"
+            }
        }
    }
 }
--- a/backend/package.json
+++ b/backend/package.json
@@ -7,13 +7,19 @@
    "scripts": {
        "start": "node server.js",
        "dev": "nodemon server.js",
-        "test": "vitest run"
+        "test": "vitest run",
+        "db:migrate": "node db/migrate.js",
+        "db:seed": "node db/seed.js"
    },
    "dependencies": {
        "@anthropic-ai/sdk": "^0.74.0",
        "cors": "^2.8.5",
        "dotenv": "^16.4.7",
-        "express": "^4.21.2"
+        "express": "^4.21.2",
+        "pg": "^8.18.0",
+        "pg-copy-streams": "^7.0.0",
+        "pg-query-stream": "^4.16.0",
+        "voyageai": "^0.1.0"
    },
    "devDependencies": {
        "nodemon": "^3.1.9",
--- a/backend/routes/notes.routes.js
+++ b/backend/routes/notes.routes.js
@@ -1,33 +1,40 @@
 import { Router } from 'express';
-import { readFile } from 'fs/promises';
+import { pipeline } from 'node:stream/promises';
+import { getAllNotes, streamAllNotes } from '../db/notes.dao.js';
 import { clusterNotes } from '../services/clustering.service.js';
+import { jsonArray } from '../lib/streams.js';

 const router = Router();
-const DATA_PATH = '../data/notes.json'
-
-const loadNotes = async () => {
-  const raw = await readFile(DATA_PATH, 'utf-8');
-  return JSON.parse(raw);
-};

 router.get('/', async (req, res) => {
  try {
-    const notes = await loadNotes();
-    res.json(notes);
+    const rows = await streamAllNotes();
+    res.type('application/json');
+    await pipeline(rows, jsonArray(), res);
  } catch (err) {
-    console.error(`Error loading notes: ${err}`)
+    console.error(`Error loading notes: ${err}`);
+    if (res.headersSent) {
+      res.destroy(err);
+      return;
+    }
    res.status(500).json({ error: 'Failed to load notes' });
  }
 });

 router.post('/cluster', async (req, res) => {
+  const controller = new AbortController();
+  res.on('close', () => {
+    if (!res.writableEnded) controller.abort();
+  });
+
  try {
-    const notes = await loadNotes();
-    const clusters = await clusterNotes(notes);
-    res.json(clusters);
+    const notes = await getAllNotes();
+    const result = await clusterNotes(notes, { signal: controller.signal });
+    res.json(result);
  } catch (err) {
-    console.error(`Clustering failed: ${err}` )
-    res.status(500).json({ error: `Clustering failed: ${err}` });
+    if (controller.signal.aborted) return;
+    console.error(`Clustering failed: ${err}`);
+    res.status(500).json({ error: 'Clustering failed' });
  }
 });

--- a/backend/services/clustering.service.js
+++ b/backend/services/clustering.service.js
@@ -1,4 +1,8 @@
 import Anthropic from "@anthropic-ai/sdk";
+import { pipeline } from "node:stream/promises";
+import { Transform, Writable } from "node:stream";
+import { embedNotes } from "./embedding.service.js";
+import { validateStructure, computeCohesionScore } from "./validation.service.js";

 const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
@@ -31,15 +35,81 @@ Here are the notes:
 ${notesJson}`;
 };

-export const clusterNotes = async (notes) => {
-  const response = await client.messages.create({
-   model: "claude-sonnet-4-20250514",
+const textDeltas = () => new Transform({
+  objectMode: true,
+  transform(event, _encoding, callback) {
+    if (event?.type === 'content_block_delta' && event.delta?.type === 'text_delta') {
+      callback(null, event.delta.text);
+      return;
+    }
+    callback();
+  },
+});
+
+// Rejects as soon as the first non-whitespace character proves the response
+// is not the JSON array we asked for, rather than after the full generation.
+const collectClusterJson = (sink) => new Writable({
+  objectMode: true,
+  write(text, _encoding, callback) {
+    if (!sink.sawOpeningBracket) {
+      const leading = (sink.parts.join('') + text).trimStart();
+      if (leading.length > 0) {
+        if (!leading.startsWith('[')) {
+          callback(new Error('LLM API returned non-JSON response'));
+          return;
+        }
+        sink.sawOpeningBracket = true;
+      }
+    }
+    sink.parts.push(text);
+    callback();
+  },
+});
+
+const requestClusters = async (notes, signal) => {
+  const sink = { parts: [], sawOpeningBracket: false };
+
+  const options = signal ? [{ signal }] : [];
+
+  const events = client.messages.stream(
+    {
+      model: "claude-sonnet-5",
      max_tokens: 4096,
      messages: [
        { role: "user", content: buildPrompt(notes) },
      ],
-  });
+    },
+    ...options
+  );

-  const raw = response.content[0].text;
-  return JSON.parse(raw);
+  await pipeline(events, textDeltas(), collectClusterJson(sink), ...options);
+
+  const text = sink.parts.join('');
+
+  if (text.trim().length === 0) {
+    throw new Error('Unexpected response from LLM API: no text content returned');
+  }
+
+  try {
+    return JSON.parse(text);
+  } catch {
+    throw new Error('LLM API returned non-JSON response');
+  }
+};
+
+export const clusterNotes = async (notes, { signal } = {}) => {
+  const [clusters, embeddingMap] = await Promise.all([
+    requestClusters(notes, signal),
+    embedNotes(notes),
+  ]);
+
+  const noteIds = notes.map((n) => n.id);
+  const { valid, reasons } = validateStructure(clusters, noteIds);
+  if (!valid) {
+    throw new Error(`Cluster validation failed: ${reasons.join('; ')}`);
+  }
+
+  const score = computeCohesionScore(clusters, embeddingMap);
+
+  return { clusters, score: Math.round(score * 100) / 100 };
 };
--- a/backend/services/embedding.service.js
+++ b/backend/services/embedding.service.js
@@ -0,0 +1,41 @@
+import { VoyageAIClient } from "voyageai";
+import { pipeline } from "node:stream/promises";
+import { Readable } from "node:stream";
+import { batch } from "../lib/streams.js";
+
+const client = new VoyageAIClient({
+    apiKey: process.env.VOYAGEAI_API_KEY,
+});
+
+const EMBED_BATCH_SIZE = 128;
+
+/**
+ * @param {Array<{id: string, text: string}>} notes
+ * @returns {Promise<Map<string, number[]>>} noteId → embedding vector
+ */
+export const embedNotes = async (notes) => {
+    const embeddingMap = new Map();
+
+    if (!notes || notes.length === 0) {
+        return embeddingMap;
+    }
+
+    await pipeline(
+        Readable.from(notes, { objectMode: true }),
+        batch(EMBED_BATCH_SIZE),
+        async (batches) => {
+            for await (const chunk of batches) {
+                const response = await client.embed({
+                    input: chunk.map((n) => n.text),
+                    model: "voyage-3",
+                });
+
+                response.data.forEach((item, i) => {
+                    embeddingMap.set(chunk[i].id, item.embedding);
+                });
+            }
+        }
+    );
+
+    return embeddingMap;
+};
--- a/backend/services/validation.service.js
+++ b/backend/services/validation.service.js
@@ -0,0 +1,128 @@
+/**
+ * Structural validation for LLM output
+ *
+ * @param {Array<{label: string, noteIds: string[]}>} clusters
+ * @param {string[]} inputNoteIds - the original note IDs that were sent to the LLM
+ * @returns {{valid: boolean, reasons: string[]}}
+ */
+export const validateStructure = (clusters, inputNoteIds) => {
+    const reasons = [];
+
+    if (!Array.isArray(clusters) || clusters.length === 0) {
+        return { valid: false, reasons: ['Response is not a non-empty array'] };
+    }
+
+    const assignedIds = [];
+    for (const cluster of clusters) {
+        if (!cluster.label || typeof cluster.label !== 'string') {
+            reasons.push(`Cluster missing a valid label`);
+        }
+        if (!Array.isArray(cluster.noteIds) || cluster.noteIds.length === 0) {
+            reasons.push(`Cluster "${cluster.label ?? '(unlabeled)'}" has no noteIds`);
+        }
+        assignedIds.push(...(cluster.noteIds ?? []));
+    }
+
+    const inputSet = new Set(inputNoteIds);
+    const assignedSet = new Set(assignedIds);
+
+    if (assignedIds.length !== assignedSet.size) {
+        reasons.push('One or more notes appear in multiple clusters');
+    }
+
+    const missing = inputNoteIds.filter((id) => !assignedSet.has(id));
+    if (missing.length > 0) {
+        reasons.push(`Notes missing from clusters: ${missing.join(', ')}`);
+    }
+
+    const extra = assignedIds.filter((id) => !inputSet.has(id));
+    if (extra.length > 0) {
+        reasons.push(`Unknown noteIds in clusters: ${[...new Set(extra)].join(', ')}`);
+    }
+
+    if (clusters.length > inputNoteIds.length) {
+        reasons.push(`More clusters (${clusters.length}) than notes (${inputNoteIds.length})`);
+    }
+
+    return { valid: reasons.length === 0, reasons };
+};
+
+const cosineSimilarity = (a, b) => {
+    let dot = 0;
+    let magA = 0;
+    let magB = 0;
+    for (let i = 0; i < a.length; i++) {
+        dot += a[i] * b[i];
+        magA += a[i] * a[i];
+        magB += b[i] * b[i];
+    }
+    const denom = Math.sqrt(magA) * Math.sqrt(magB);
+    return denom === 0 ? 0 : dot / denom;
+};
+
+/**
+ * Computes silhouette-style cohesion score for the clustering.
+ *
+ * For each note, measures how much more similar it is to its own cluster
+ * versus the nearest neighboring cluster. Returns a score in [-1, 1]
+ * where higher is better.
+ *
+ * @param {Array<{label: string, noteIds: string[]}>} clusters
+ * @param {Map<string, number[]>} embeddingMap - noteId → vector
+ * @returns {number} average silhouette score
+ */
+export const computeCohesionScore = (clusters, embeddingMap) => {
+    if (clusters.length <= 1) return 1.0;
+
+    const scores = [];
+
+    for (let ci = 0; ci < clusters.length; ci++) {
+        const clusterIds = clusters[ci].noteIds;
+        if (clusterIds.length <= 1) {
+            scores.push(0);
+            continue;
+        }
+
+        for (const noteId of clusterIds) {
+            const vec = embeddingMap.get(noteId);
+            if (!vec) continue;
+
+            // a(i): avg distance to other notes in same cluster
+            let intraSum = 0;
+            let intraCount = 0;
+            for (const otherId of clusterIds) {
+                if (otherId === noteId) continue;
+                const otherVec = embeddingMap.get(otherId);
+                if (!otherVec) continue;
+                intraSum += 1 - cosineSimilarity(vec, otherVec);
+                intraCount++;
+            }
+            const a = intraCount > 0 ? intraSum / intraCount : 0;
+
+            // b(i): min avg distance to notes in any other cluster
+            let b = Infinity;
+            for (let oi = 0; oi < clusters.length; oi++) {
+                if (oi === ci) continue;
+                const otherClusterIds = clusters[oi].noteIds;
+                let interSum = 0;
+                let interCount = 0;
+                for (const otherId of otherClusterIds) {
+                    const otherVec = embeddingMap.get(otherId);
+                    if (!otherVec) continue;
+                    interSum += 1 - cosineSimilarity(vec, otherVec);
+                    interCount++;
+                }
+                if (interCount > 0) {
+                    b = Math.min(b, interSum / interCount);
+                }
+            }
+            if (b === Infinity) b = 0;
+
+            const max = Math.max(a, b);
+            scores.push(max === 0 ? 0 : (b - a) / max);
+        }
+    }
+
+    if (scores.length === 0) return 0;
+    return scores.reduce((sum, s) => sum + s, 0) / scores.length;
+};
--- a/backend/tests/Clustering.service.test.js
+++ b/backend/tests/Clustering.service.test.js
@@ -1,19 +1,27 @@
 import { describe, it, expect, vi, beforeEach } from 'vitest';

-const { createMock } = vi.hoisted(() => {
-  return { createMock: vi.fn() };
+const { streamMock, mockEmbeddings } = vi.hoisted(() => {
+  const embeddings = new Map([
+    ['note_001', [1.0, 0.0, 0.0]],
+    ['note_002', [0.0, 1.0, 0.0]],
+  ]);
+  return { streamMock: vi.fn(), mockEmbeddings: embeddings };
 });

 vi.mock('@anthropic-ai/sdk', () => {
  return {
    default: class MockAnthropic {
      constructor() {
-        this.messages = { create: createMock };
+        this.messages = { stream: streamMock };
      }
    },
  };
 });

+vi.mock('../services/embedding.service.js', () => ({
+  embedNotes: vi.fn().mockResolvedValue(mockEmbeddings),
+}));
+
 import { clusterNotes } from '../services/clustering.service.js';

 const MOCK_NOTES = [
@@ -26,6 +34,40 @@ const MOCK_CLUSTERS = [
  { label: 'Export Issues', noteIds: ['note_002'] },
 ];

+// Splits text into several text_delta events so the service is exercised
+// against a genuinely incremental stream rather than one whole payload.
+const textEvents = (text, pieces = 4) => {
+  const size = Math.max(1, Math.ceil(text.length / pieces));
+  const events = [];
+  for (let i = 0; i < text.length; i += size) {
+    events.push({
+      type: 'content_block_delta',
+      delta: { type: 'text_delta', text: text.slice(i, i + size) },
+    });
+  }
+  return events;
+};
+
+const mockStreamOf = (text) => {
+  const events = [
+    { type: 'message_start' },
+    ...textEvents(text),
+    { type: 'message_stop' },
+  ];
+  streamMock.mockImplementation(() => ({
+    async *[Symbol.asyncIterator]() {
+      for (const event of events) yield event;
+    },
+  }));
+};
+
+const mockStreamThrowing = (err) => {
+  streamMock.mockImplementation(() => ({
+    async *[Symbol.asyncIterator]() {
+      throw err;
+    },
+  }));
+};

 describe('clusterNotes service', () => {

@@ -33,54 +75,88 @@ describe('clusterNotes service', () => {
    vi.clearAllMocks();
  });

-  it('should call Anthropic messages.create with the correct model', async () => {
-    createMock.mockResolvedValue({
-      content: [{ text: JSON.stringify(MOCK_CLUSTERS) }],
-    });
+  it('should call Anthropic messages.stream with the correct model', async () => {
+    mockStreamOf(JSON.stringify(MOCK_CLUSTERS));

    await clusterNotes(MOCK_NOTES);

-    expect(createMock).toHaveBeenCalledOnce();
-    const callArgs = createMock.mock.calls[0][0];
-    expect(callArgs.model).toBe('claude-sonnet-4-20250514');
+    expect(streamMock).toHaveBeenCalledOnce();
+    const callArgs = streamMock.mock.calls[0][0];
+    expect(callArgs.model).toBe('claude-sonnet-5');
    expect(callArgs.max_tokens).toBe(4096);
  });

-  it('should include all note texts in prompt sent to the LLM API', async () => {
-    createMock.mockResolvedValue({
-      content: [{ text: JSON.stringify(MOCK_CLUSTERS) }],
+  it('should forward an abort signal to the LLM request', async () => {
+    mockStreamOf(JSON.stringify(MOCK_CLUSTERS));
+    const controller = new AbortController();
+
+    await clusterNotes(MOCK_NOTES, { signal: controller.signal });
+
+    expect(streamMock.mock.calls[0][1]).toEqual({ signal: controller.signal });
  });

+  it('should include all note texts in prompt sent to the LLM API', async () => {
+    mockStreamOf(JSON.stringify(MOCK_CLUSTERS));
+
    await clusterNotes(MOCK_NOTES);

-    const prompt = createMock.mock.calls[0][0].messages[0].content;
+    const prompt = streamMock.mock.calls[0][0].messages[0].content;
    expect(prompt).toContain('note_001');
    expect(prompt).toContain('Login is broken');
    expect(prompt).toContain('note_002');
    expect(prompt).toContain('Export fails');
  });

-  it('should parse and return the clustered JSON from the API response', async () => {
-    createMock.mockResolvedValue({
-      content: [{ text: JSON.stringify(MOCK_CLUSTERS) }],
-    });
+  it('should return clusters and a cohesion score', async () => {
+    mockStreamOf(JSON.stringify(MOCK_CLUSTERS));

    const result = await clusterNotes(MOCK_NOTES);

-    expect(result).toEqual(MOCK_CLUSTERS);
+    expect(result.clusters).toEqual(MOCK_CLUSTERS);
+    expect(typeof result.score).toBe('number');
+    expect(result.score).toBeGreaterThanOrEqual(-1);
+    expect(result.score).toBeLessThanOrEqual(1);
+  });
+
+  it('should reassemble clusters split across many stream deltas', async () => {
+    const json = JSON.stringify(MOCK_CLUSTERS);
+    streamMock.mockImplementation(() => ({
+      async *[Symbol.asyncIterator]() {
+        for (const char of json) {
+          yield { type: 'content_block_delta', delta: { type: 'text_delta', text: char } };
+        }
+      },
+    }));
+
+    const result = await clusterNotes(MOCK_NOTES);
+
+    expect(result.clusters).toEqual(MOCK_CLUSTERS);
  });

  it('should throw error when the API returns non-JSON', async () => {
-    createMock.mockResolvedValue({
-      content: [{ text: 'An unknown error occured when generting structured response.' }],
+    mockStreamOf('An unknown error occured when generting structured response.');
+
+    await expect(clusterNotes(MOCK_NOTES)).rejects.toThrow('non-JSON response');
  });

-    await expect(clusterNotes(MOCK_NOTES)).rejects.toThrow();
+  it('should throw error when the API response has no text content', async () => {
+    mockStreamOf('');
+
+    await expect(clusterNotes(MOCK_NOTES)).rejects.toThrow('no text content returned');
  });

  it('should throw error when Anthropic API authentication fails', async () => {
-    createMock.mockRejectedValue(new Error('401 Unauthorized'));
+    mockStreamThrowing(new Error('401 Unauthorized'));

    await expect(clusterNotes(MOCK_NOTES)).rejects.toThrow('401 Unauthorized');
  });
+
+  it('should throw a validation error when a note is missing from clusters', async () => {
+    const incompleteClusters = [
+      { label: 'Auth Issues', noteIds: ['note_001'] },
+    ];
+    mockStreamOf(JSON.stringify(incompleteClusters));
+
+    await expect(clusterNotes(MOCK_NOTES)).rejects.toThrow('Cluster validation failed');
+  });
 });
--- a/backend/tests/Notes.test.js
+++ b/backend/tests/Notes.test.js
@@ -1,17 +1,19 @@
 import { describe, it, expect, vi, beforeEach } from 'vitest';
 import request from 'supertest';
+import { Readable } from 'node:stream';
 import app from '../app.js';

+vi.mock('../db/notes.dao.js', () => ({
+  getAllNotes: vi.fn(),
+  streamAllNotes: vi.fn(),
+}));
+
 vi.mock('../services/clustering.service.js', () => ({
  clusterNotes: vi.fn(),
 }));

-vi.mock('fs/promises', () => ({
-  readFile: vi.fn(),
-}));
-
+import { getAllNotes, streamAllNotes } from '../db/notes.dao.js';
 import { clusterNotes } from '../services/clustering.service.js';
-import { readFile } from 'fs/promises';

 const MOCK_NOTES = [
  { id: 'note_001', text: 'Login flow feels confusing', x: 193, y: 191, author: 'user_5', color: 'yellow' },
@@ -24,6 +26,8 @@ const MOCK_CLUSTERS = [
  { label: 'Export Problems', noteIds: ['note_003'] },
 ];

+const rowStream = (rows) => Readable.from(rows, { objectMode: true });
+
 describe('GET /v1/notes', () => {

  beforeEach(() => {
@@ -31,7 +35,7 @@ describe('GET /v1/notes', () => {
  });

  it('should return 200 and an array of notes', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
+    streamAllNotes.mockResolvedValue(rowStream(MOCK_NOTES));

    const res = await request(app).get('/v1/notes');

@@ -40,8 +44,26 @@ describe('GET /v1/notes', () => {
    expect(Array.isArray(res.body)).toBe(true);
  });

+  it('should send JSON incrementally rather than buffering the row set', async () => {
+    streamAllNotes.mockResolvedValue(rowStream(MOCK_NOTES));
+
+    const res = await request(app).get('/v1/notes');
+
+    expect(res.headers['content-type']).toMatch(/application\/json/);
+    expect(res.headers['content-length']).toBeUndefined();
+  });
+
+  it('should return an empty array when there are no notes', async () => {
+    streamAllNotes.mockResolvedValue(rowStream([]));
+
+    const res = await request(app).get('/v1/notes');
+
+    expect(res.status).toBe(200);
+    expect(res.body).toEqual([]);
+  });
+
  it('should return notes with expected properties', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
+    streamAllNotes.mockResolvedValue(rowStream(MOCK_NOTES));

    const res = await request(app).get('/v1/notes');
    const note = res.body[0];
@@ -54,8 +76,8 @@ describe('GET /v1/notes', () => {
    expect(note).toHaveProperty('color');
  });

-  it('should return 500 when the data file cannot be read', async () => {
-    readFile.mockRejectedValue(new Error('ENOENT: file not found'));
+  it('should return 500 when the database query fails', async () => {
+    streamAllNotes.mockRejectedValue(new Error('connection refused'));

    const res = await request(app).get('/v1/notes');

@@ -64,13 +86,17 @@ describe('GET /v1/notes', () => {
    expect(res.body.error).toBe('Failed to load notes');
  });

-  it('should return 500 when data file is invalid JSON', async () => {
-    readFile.mockResolvedValue('{ this is not valid json }');
+  it('should abort the response when the row stream fails mid-flight', async () => {
+    const failing = new Readable({
+      objectMode: true,
+      read() {
+        this.push(MOCK_NOTES[0]);
+        this.destroy(new Error('connection lost'));
+      },
+    });
+    streamAllNotes.mockResolvedValue(failing);

-    const res = await request(app).get('/v1/notes');
-
-    expect(res.status).toBe(500);
-    expect(res.body).toHaveProperty('error');
+    await expect(request(app).get('/v1/notes')).rejects.toThrow();
  });
 });

@@ -81,7 +107,7 @@ describe('POST /v1/notes/cluster', () => {
  });

  it('should return 200 and clustered results', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
+    getAllNotes.mockResolvedValue(MOCK_NOTES);
    clusterNotes.mockResolvedValue(MOCK_CLUSTERS);

    const res = await request(app).post('/v1/notes/cluster');
@@ -91,7 +117,7 @@ describe('POST /v1/notes/cluster', () => {
  });

  it('should return clusters with the expected shape (label, noteIds)', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
+    getAllNotes.mockResolvedValue(MOCK_NOTES);
    clusterNotes.mockResolvedValue(MOCK_CLUSTERS);

    const res = await request(app).post('/v1/notes/cluster');
@@ -104,18 +130,33 @@ describe('POST /v1/notes/cluster', () => {
  });

  it('should pass the loaded notes to clusterNotes', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
+    getAllNotes.mockResolvedValue(MOCK_NOTES);
    clusterNotes.mockResolvedValue(MOCK_CLUSTERS);

    await request(app).post('/v1/notes/cluster');

    expect(clusterNotes).toHaveBeenCalledOnce();
-    expect(clusterNotes).toHaveBeenCalledWith(MOCK_NOTES);
+    expect(clusterNotes).toHaveBeenCalledWith(
+      MOCK_NOTES,
+      expect.objectContaining({ signal: expect.any(AbortSignal) })
+    );
+  });
+
+  it('should pass a signal that is not aborted while the request is open', async () => {
+    getAllNotes.mockResolvedValue(MOCK_NOTES);
+    clusterNotes.mockImplementation(async (_notes, { signal }) => {
+      expect(signal.aborted).toBe(false);
+      return MOCK_CLUSTERS;
+    });
+
+    const res = await request(app).post('/v1/notes/cluster');
+
+    expect(res.status).toBe(200);
  });

  it('should return 500 when clusterNotes (API call) fails', async () => {
-    readFile.mockResolvedValue(JSON.stringify(MOCK_NOTES));
-    clusterNotes.mockRejectedValue(new Error('Anthropic API error'));
+    getAllNotes.mockResolvedValue(MOCK_NOTES);
+    clusterNotes.mockRejectedValue(new Error('LLM API error'));

    const res = await request(app).post('/v1/notes/cluster');

@@ -124,8 +165,8 @@ describe('POST /v1/notes/cluster', () => {
    expect(res.body.error).toMatch(/^Clustering failed/);
  });

-  it('should return 500 when the data file cannot be read', async () => {
-    readFile.mockRejectedValue(new Error('ENOENT: file not found'));
+  it('should return 500 when the database query fails', async () => {
+    getAllNotes.mockRejectedValue(new Error('connection refused'));

    const res = await request(app).post('/v1/notes/cluster');

--- a/backend/tests/notes.dao.test.js
+++ b/backend/tests/notes.dao.test.js
@@ -0,0 +1,200 @@
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+import { Readable } from 'node:stream';
+
+const { mockQuery, mockConnect } = vi.hoisted(() => ({
+  mockQuery: vi.fn(),
+  mockConnect: vi.fn(),
+}));
+
+vi.mock('../db/index.js', () => ({
+  query: mockQuery,
+  getPool: () => ({ connect: mockConnect }),
+}));
+
+import {
+  getAllNotes,
+  streamAllNotes,
+  getNoteById,
+  createNote,
+  createNotes,
+} from '../db/notes.dao.js';
+
+const MOCK_ROWS = [
+  { id: 'note_001', text: 'Login flow feels confusing', x: 193, y: 191, author: 'user_5', color: 'yellow' },
+  { id: 'note_002', text: 'Login flow is broken on mobile', x: 214, y: 281, author: 'user_9', color: 'yellow' },
+];
+
+describe('notes.dao', () => {
+
+  beforeEach(() => {
+    vi.clearAllMocks();
+  });
+
+  describe('getAllNotes', () => {
+    it('should return all notes ordered by id', async () => {
+      mockQuery.mockResolvedValue({ rows: MOCK_ROWS });
+
+      const result = await getAllNotes();
+
+      expect(result).toEqual(MOCK_ROWS);
+      expect(mockQuery).toHaveBeenCalledWith(
+        'SELECT id, text, x, y, author, color FROM notes ORDER BY id'
+      );
+    });
+
+    it('should return an empty array when no notes exist', async () => {
+      mockQuery.mockResolvedValue({ rows: [] });
+
+      const result = await getAllNotes();
+
+      expect(result).toEqual([]);
+    });
+
+    it('should propagate database errors', async () => {
+      mockQuery.mockRejectedValue(new Error('connection refused'));
+
+      await expect(getAllNotes()).rejects.toThrow('connection refused');
+    });
+  });
+
+  describe('streamAllNotes', () => {
+    const mockClient = (rows) => {
+      const release = vi.fn();
+      const client = {
+        release,
+        query: vi.fn(() => Readable.from(rows, { objectMode: true })),
+      };
+      mockConnect.mockResolvedValue(client);
+      return { client, release };
+    };
+
+    it('should stream rows without buffering them into an array', async () => {
+      mockClient(MOCK_ROWS);
+
+      const stream = await streamAllNotes();
+      const received = [];
+      for await (const row of stream) received.push(row);
+
+      expect(received).toEqual(MOCK_ROWS);
+    });
+
+    it('should release the pooled client once the stream ends', async () => {
+      const { release } = mockClient(MOCK_ROWS);
+
+      const stream = await streamAllNotes();
+      for await (const _row of stream) { /* drain */ }
+
+      expect(release).toHaveBeenCalledOnce();
+    });
+
+    it('should release the pooled client when a consumer destroys the stream early', async () => {
+      const { release } = mockClient(MOCK_ROWS);
+
+      const stream = await streamAllNotes();
+      stream.destroy();
+      await new Promise((resolve) => stream.once('close', resolve));
+
+      expect(release).toHaveBeenCalledOnce();
+    });
+
+    it('should release the pooled client when starting the query throws', async () => {
+      const release = vi.fn();
+      mockConnect.mockResolvedValue({
+        release,
+        query: vi.fn(() => { throw new Error('cursor failed'); }),
+      });
+
+      await expect(streamAllNotes()).rejects.toThrow('cursor failed');
+      expect(release).toHaveBeenCalledOnce();
+    });
+  });
+
+  describe('getNoteById', () => {
+    it('should return a single note when found', async () => {
+      mockQuery.mockResolvedValue({ rows: [MOCK_ROWS[0]] });
+
+      const result = await getNoteById('note_001');
+
+      expect(result).toEqual(MOCK_ROWS[0]);
+      expect(mockQuery).toHaveBeenCalledWith(
+        'SELECT id, text, x, y, author, color FROM notes WHERE id = $1',
+        ['note_001']
+      );
+    });
+
+    it('should return null when note is not found', async () => {
+      mockQuery.mockResolvedValue({ rows: [] });
+
+      const result = await getNoteById('note_999');
+
+      expect(result).toBeNull();
+    });
+  });
+
+  describe('createNote', () => {
+    it('should insert a note and return it', async () => {
+      const input = { id: 'note_003', text: 'Export fails', x: 100, y: 200, author: 'user_1', color: 'blue' };
+      mockQuery.mockResolvedValue({ rows: [input] });
+
+      const result = await createNote(input);
+
+      expect(result).toEqual(input);
+      expect(mockQuery).toHaveBeenCalledOnce();
+      const [sql, params] = mockQuery.mock.calls[0];
+      expect(sql).toContain('INSERT INTO notes');
+      expect(params).toEqual(['note_003', 'Export fails', 100, 200, 'user_1', 'blue']);
+    });
+
+    it('should use defaults for missing x, y, and color', async () => {
+      const input = { id: 'note_004', text: 'Needs fixing', author: 'user_2' };
+      mockQuery.mockResolvedValue({ rows: [{ ...input, x: 0, y: 0, color: 'yellow' }] });
+
+      await createNote(input);
+
+      const params = mockQuery.mock.calls[0][1];
+      expect(params[2]).toBe(0);
+      expect(params[3]).toBe(0);
+      expect(params[5]).toBe('yellow');
+    });
+  });
+
+  describe('createNotes', () => {
+    it('should batch-insert multiple notes and return them', async () => {
+      mockQuery.mockResolvedValue({ rows: MOCK_ROWS });
+
+      const result = await createNotes(MOCK_ROWS);
+
+      expect(result).toEqual(MOCK_ROWS);
+      expect(mockQuery).toHaveBeenCalledOnce();
+      const [sql, params] = mockQuery.mock.calls[0];
+      expect(sql).toContain('INSERT INTO notes');
+      expect(params).toHaveLength(12);
+    });
+
+    it('should return an empty array without querying when given no notes', async () => {
+      const result = await createNotes([]);
+
+      expect(result).toEqual([]);
+      expect(mockQuery).not.toHaveBeenCalled();
+    });
+
+    it('should split large inputs into multiple statements under the bind-parameter limit', async () => {
+      const many = Array.from({ length: 2500 }, (_, i) => ({
+        id: `note_${i}`,
+        text: `text ${i}`,
+        author: 'user_1',
+      }));
+      mockQuery.mockImplementation(async (_sql, params) => ({
+        rows: new Array(params.length / 6).fill(null).map((_, i) => ({ i })),
+      }));
+
+      const result = await createNotes(many);
+
+      expect(mockQuery).toHaveBeenCalledTimes(3);
+      expect(result).toHaveLength(2500);
+      for (const [, params] of mockQuery.mock.calls) {
+        expect(params.length).toBeLessThan(65535);
+      }
+    });
+  });
+});
--- a/backend/tests/streams.test.js
+++ b/backend/tests/streams.test.js
@@ -0,0 +1,82 @@
+import { describe, it, expect } from 'vitest';
+import { Readable } from 'node:stream';
+import { pipeline } from 'node:stream/promises';
+import { batch, jsonArray } from '../lib/streams.js';
+
+const collect = async (source, transform) => {
+  const out = [];
+  await pipeline(source, transform, async (results) => {
+    for await (const item of results) out.push(item);
+  });
+  return out;
+};
+
+describe('batch', () => {
+  it('should group items into fixed-size arrays', async () => {
+    const source = Readable.from([1, 2, 3, 4], { objectMode: true });
+
+    const result = await collect(source, batch(2));
+
+    expect(result).toEqual([[1, 2], [3, 4]]);
+  });
+
+  it('should flush a partial trailing batch', async () => {
+    const source = Readable.from([1, 2, 3, 4, 5], { objectMode: true });
+
+    const result = await collect(source, batch(2));
+
+    expect(result).toEqual([[1, 2], [3, 4], [5]]);
+  });
+
+  it('should emit nothing for an empty source', async () => {
+    const source = Readable.from([], { objectMode: true });
+
+    const result = await collect(source, batch(3));
+
+    expect(result).toEqual([]);
+  });
+
+  it('should reject a non-positive size', () => {
+    expect(() => batch(0)).toThrow(TypeError);
+    expect(() => batch(1.5)).toThrow(TypeError);
+  });
+});
+
+describe('jsonArray', () => {
+  const serialize = async (items) => {
+    const chunks = await collect(
+      Readable.from(items, { objectMode: true }),
+      jsonArray()
+    );
+    return chunks.map(String).join('');
+  };
+
+  it('should serialize objects into a JSON array', async () => {
+    const items = [{ id: 'a' }, { id: 'b' }];
+
+    const output = await serialize(items);
+
+    expect(output).toBe('[{"id":"a"},{"id":"b"}]');
+    expect(JSON.parse(output)).toEqual(items);
+  });
+
+  it('should emit an empty array when the source yields nothing', async () => {
+    const output = await serialize([]);
+
+    expect(output).toBe('[]');
+    expect(JSON.parse(output)).toEqual([]);
+  });
+
+  it('should emit a valid single-element array', async () => {
+    const output = await serialize([{ id: 'only' }]);
+
+    expect(JSON.parse(output)).toEqual([{ id: 'only' }]);
+  });
+
+  it('should propagate serialization errors', async () => {
+    const circular = {};
+    circular.self = circular;
+
+    await expect(serialize([circular])).rejects.toThrow();
+  });
+});
--- a/frontend/.env.development
+++ b/frontend/.env.development
@@ -0,0 +1,2 @@
+VITE_APP_URL=localhost:3000
+VITE_API_BASE=http://localhost:3001
--- a/frontend/.env.production
+++ b/frontend/.env.production
@@ -0,0 +1,2 @@
+VITE_APP_URL=https://example.com
+VITE_API_BASE=https://www.example.com:4000
--- a/frontend/src/api/api.tsx
+++ b/frontend/src/api/api.tsx
@@ -1,7 +1,7 @@
 import { useQuery, useMutation } from '@tanstack/react-query';
-import type { Sticky, Cluster } from '../types/types';
+import type { Sticky, ClusterResponse } from '../types/types';

-const API_BASE = 'http://localhost:3001/v1/notes';
+const API_BASE = `${import.meta.env.VITE_API_BASE}/v1/notes`;

 const fetchStickies = async (): Promise<Sticky[]> => {
  const response = await fetch(API_BASE);
@@ -11,7 +11,7 @@ const fetchStickies = async (): Promise<Sticky[]> => {
  return response.json();
 };

-const fetchClusters = async (): Promise<Cluster[]> => {
+const fetchClusters = async (): Promise<ClusterResponse> => {
  const response = await fetch(`${API_BASE}/cluster`, {
    method: 'POST',
  });
@@ -29,7 +29,7 @@ export const useGetStickies = () => {
 };

 export const useClusterStickies = () => {
-  return useMutation<Cluster[]>({
+  return useMutation<ClusterResponse>({
    mutationFn: fetchClusters,
  });
 };
--- a/frontend/src/components/stickies.tsx
+++ b/frontend/src/components/stickies.tsx
@@ -1,16 +1,39 @@
+import { useState, useEffect, useRef } from 'react';
+import type { DragEvent } from 'react';
 import { useGetStickies, useClusterStickies } from '../api/api';
-import type { Sticky as StickyType } from '../types/types';
+import type { Sticky as StickyType, RankedCluster } from '../types/types';
 import Sticky from './sticky';
 import Button from './button';
 import '../styles/stickies.css';

+const scoreLabel = (score: number): string => {
+  if (score >= 0.7) return 'Strong';
+  if (score >= 0.4) return 'Moderate';
+  if (score >= 0.1) return 'Weak';
+  return 'Poor';
+};
+
 const Stickies = () => {
  const { data: stickies, isLoading, error } = useGetStickies();
-  const { mutate: cluster, data: clusters, isPending } = useClusterStickies();
+  const { mutate: cluster, data: clusterResponse, isPending } = useClusterStickies();
+
+  const [rankedClusters, setRankedClusters] = useState<RankedCluster[]>([]);
+  const dragIndex = useRef<number | null>(null);
+  const [dragOverIndex, setDragOverIndex] = useState<number | null>(null);
+
+  useEffect(() => {
+    if (clusterResponse?.clusters) {
+      setRankedClusters(
+        clusterResponse.clusters.map((c, i) => ({ ...c, rank: i + 1 }))
+      );
+    }
+  }, [clusterResponse]);

  if (isLoading) return <div>Loading...</div>;
  if (error) return <div>Error: {error.message}</div>;

+  const score = clusterResponse?.score;
+
  const handleCluster = () => {
    cluster();
  };
@@ -24,20 +47,78 @@ const Stickies = () => {
  const stickyMap = buildStickyMap();

  const renderStickies = (items: StickyType[]) =>
-    items.map((sticky) => <Sticky key={sticky.id} sticky={sticky} />);
+    items?.map((sticky) => <Sticky key={sticky.id} sticky={sticky} />);
+
+  const handleDragStart = (index: number) => {
+    dragIndex.current = index;
+  };
+
+  const handleDragOver = (e: DragEvent, index: number) => {
+    e.preventDefault();
+    setDragOverIndex(index);
+  };
+
+  const handleDragLeave = () => {
+    setDragOverIndex(null);
+  };
+
+  const handleDrop = (targetIndex: number) => {
+    const sourceIndex = dragIndex.current;
+    if (sourceIndex === null || sourceIndex === targetIndex) {
+      dragIndex.current = null;
+      setDragOverIndex(null);
+      return;
+    }
+
+    const reordered = [...rankedClusters];
+    const [moved] = reordered.splice(sourceIndex, 1);
+    reordered.splice(targetIndex, 0, moved);
+
+    setRankedClusters(reordered.map((c, i) => ({ ...c, rank: i + 1 })));
+    dragIndex.current = null;
+    setDragOverIndex(null);
+  };
+
+  const handleDragEnd = () => {
+    dragIndex.current = null;
+    setDragOverIndex(null);
+  };

  return (
    <div className="stickies-container">
      <Button onClick={handleCluster} isLoading={isPending} label="Group Stickies By Topic" />
-      {clusters ? (
+      {rankedClusters.length > 0 ? (
        <div className="clusters-container">
-          {clusters.map((group) => (
-            <div key={group.label} className="cluster-group">
+          {score != null && (
+            <div className="cohesion-score">
+              Cluster cohesion: <strong>{score.toFixed(2)}</strong> — {scoreLabel(score)}
+            </div>
+          )}
+          {rankedClusters.map((group, index) => (
+            <div
+              key={group.label}
+              className={`cluster-group cluster-draggable${dragOverIndex === index ? ' cluster-drag-over' : ''}`}
+              draggable
+              onDragStart={() => handleDragStart(index)}
+              onDragOver={(e) => handleDragOver(e, index)}
+              onDragLeave={handleDragLeave}
+              onDrop={() => handleDrop(index)}
+              onDragEnd={handleDragEnd}
+            >
+              <div className="cluster-header">
+                <span className="cluster-rank" aria-label={`Priority ${group.rank}`}>
+                  {group.rank}
+                </span>
+                {group.rank === 1 && (
+                  <span className="cluster-reorder-hint">Drag and drop to reorganize cluster priority</span>
+                )}
                <h3 className="cluster-label">{group.label}</h3>
+                <span className="cluster-drag-handle" aria-hidden="true">⠿</span>
+              </div>
              <div className="stickies-grid">
                {renderStickies(
-                  group.noteIds
-                    .map((id) => stickyMap.get(id))
+                  group?.noteIds
+                    .map((id) => stickyMap?.get(id))
                    .filter((s): s is StickyType => !!s)
                )}
              </div>
--- a/frontend/src/main.tsx
+++ b/frontend/src/main.tsx
@@ -14,5 +14,5 @@ createRoot(document.getElementById('root')!).render(
        <App />
      </BrowserRouter>
    </QueryClientProvider>
-  </StrictMode>,
+  </StrictMode>
 )
--- a/frontend/src/pages/Home.tsx
+++ b/frontend/src/pages/Home.tsx
@@ -4,14 +4,10 @@ import '../styles/home.css'

 const Home = () => {
  return (
-      <div>
    <div>
      <Navbar />
-         </div>
-         <div>
      <Stickies />
    </div>
-      </div>
  );
 };

--- a/frontend/src/styles/home.css
+++ b/frontend/src/styles/home.css
@@ -1,25 +1,25 @@
 .main-head-box {
  border-radius: 8px;
  border: 1px solid #6dd6f4;
-    background-color: rgb(92, 0 91);
+  background-color: rgb(92, 0, 91);
  display: flex;
-  }
+}

-  .main-head-subbox-right {
+.main-head-subbox-right {
  display: flex;
  justify-content: flex-end;
  margin-right: 34px;
  width: 50%;
-  }
+}

-  .main-head-subbox-left {
+.main-head-subbox-left {
  display: flex;
  justify-content: flex-start;
  margin-left: 34px;
  width: 50%;
-  }
+}

-  .main-head {
+.main-head {
  font-size: 5rem;
  color: #6dd6f4;
  margin: 6px 0px 24px 22px;
@@ -27,9 +27,9 @@
  font-weight: 500;
  letter-spacing: 4px;
  text-decoration: underline;
-  }
+}

-  .material-symbols-outlined {
+.material-symbols-outlined {
  margin: 13px 74px 0px 0px;
  font-size: 82px;
  color: #6dd6f4;
@@ -37,5 +37,37 @@
    'FILL' 0,
    'wght' 300,
    'GRAD' 0,
-    'opsz' 24
+    'opsz' 24;
+}
+
+@media screen and (max-width: 478px) {
+  .main-head-subbox-right {
+    display: none;
  }
+
+  .main-head-subbox-left {
+    display: flex;
+    justify-content: center;
+    margin-left: auto;
+    margin-right: auto;
+    width: 100%;
+  }
+
+  .main-head {
+    font-size: 4rem;
+    color: #6dd6f4;
+    font-family: "Sulphur Point", sans-serif;
+    display: flex;
+    justify-content: center;
+    align-items: center;
+    margin-left: auto;
+    margin-right: auto;
+    font-weight: 400;
+    letter-spacing: 2px;
+    text-decoration: underline;
+  }
+
+  .material-symbols-outlined {
+    display: none !important;
+  }
+}
--- a/frontend/src/styles/stickies.css
+++ b/frontend/src/styles/stickies.css
@@ -3,6 +3,7 @@
  flex-wrap: wrap;
  gap: 16px;
  justify-content: center;
+  margin-top: 18px;
  padding: 24px;
 }

@@ -23,8 +24,77 @@
  padding: 16px;
 }

+.cluster-draggable {
+  cursor: grab;
+  transition: box-shadow 0.2s ease, border-color 0.2s ease, transform 0.15s ease;
+}
+
+.cluster-draggable:active {
+  cursor: grabbing;
+}
+
+.cluster-drag-over {
+  border-color: #ffb7ce;
+  box-shadow: 0 0 12px rgba(255, 183, 206, 0.4);
+  transform: scale(1.01);
+}
+
+.cluster-header {
+  display: flex;
+  align-items: center;
+  gap: 12px;
+  margin-bottom: 16px;
+}
+
+.cluster-rank {
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  width: 32px;
+  height: 32px;
+  border-radius: 50%;
+  background: #6dd6f4;
+  color: #1a1a2e;
+  font-weight: 700;
+  font-size: 0.95em;
+  flex-shrink: 0;
+}
+
+.cluster-reorder-hint {
+  font-size: 0.8em;
+  color: #9ca3af;
+  font-style: italic;
+  white-space: nowrap;
+  flex-shrink: 0;
+}
+
 .cluster-label {
-  margin: 0 0 16px 0;
+  margin: 0;
  font-size: 1.2em;
  font-weight: 600;
+  flex: 1;
+}
+
+.cluster-drag-handle {
+  font-size: 1.4em;
+  color: #6dd6f4;
+  opacity: 0.4;
+  user-select: none;
+  transition: opacity 0.2s ease;
+  flex-shrink: 0;
+}
+
+.cluster-draggable:hover .cluster-drag-handle {
+  opacity: 0.8;
+}
+
+.cohesion-score {
+  text-align: center;
+  font-size: 0.95em;
+  color: #e0e0e0;
+  padding: 8px 16px;
+  background: rgba(109, 214, 244, 0.1);
+  border-radius: 6px;
+  width: fit-content;
+  margin: 0 auto;
 }
--- a/frontend/src/tests/stickies.test.tsx
+++ b/frontend/src/tests/stickies.test.tsx
@@ -9,10 +9,13 @@ const MOCK_STICKIES = [
  { id: 'note_002', text: 'Export takes too long', x: 798, y: 211, author: 'user_2', color: 'green' },
 ];

-const MOCK_CLUSTERS = [
+const MOCK_CLUSTER_RESPONSE = {
+  clusters: [
    { label: 'Auth Issues', noteIds: ['note_001'] },
    { label: 'Export Issues', noteIds: ['note_002'] },
-];
+  ],
+  score: 0.74,
+};

 let fetchMock: ReturnType<typeof vi.fn>;

@@ -70,10 +73,9 @@ describe('Stickies', () => {
  });

  it('should show cluster labels after clustering succeeds', async () => {
-    // First call = GET notes, second call = POST cluster
    fetchMock
      .mockReturnValueOnce(mockFetchOk(MOCK_STICKIES))
-      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTERS));
+      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTER_RESPONSE));

    const { Wrapper } = createTestWrapper();
    render(<Stickies />, { wrapper: Wrapper });
@@ -90,7 +92,7 @@ describe('Stickies', () => {
  it('should still render sticky note text inside clusters', async () => {
    fetchMock
      .mockReturnValueOnce(mockFetchOk(MOCK_STICKIES))
-      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTERS));
+      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTER_RESPONSE));

    const { Wrapper } = createTestWrapper();
    render(<Stickies />, { wrapper: Wrapper });
@@ -101,4 +103,41 @@ describe('Stickies', () => {
    expect(await screen.findByText('Login flow feels confusing')).toBeInTheDocument();
    expect(screen.getByText('Export takes too long')).toBeInTheDocument();
  });
+
+  it('should display rank badges on clustered groups', async () => {
+    fetchMock
+      .mockReturnValueOnce(mockFetchOk(MOCK_STICKIES))
+      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTER_RESPONSE));
+
+    const { Wrapper } = createTestWrapper();
+    render(<Stickies />, { wrapper: Wrapper });
+
+    const btn = await screen.findByRole('button', { name: 'Group Stickies By Topic' });
+    await userEvent.click(btn);
+
+    await waitFor(() => {
+      expect(screen.getByLabelText('Priority 1')).toBeInTheDocument();
+      expect(screen.getByLabelText('Priority 2')).toBeInTheDocument();
+    });
+  });
+
+  it('should make cluster groups draggable', async () => {
+    fetchMock
+      .mockReturnValueOnce(mockFetchOk(MOCK_STICKIES))
+      .mockReturnValueOnce(mockFetchOk(MOCK_CLUSTER_RESPONSE));
+
+    const { Wrapper } = createTestWrapper();
+    render(<Stickies />, { wrapper: Wrapper });
+
+    const btn = await screen.findByRole('button', { name: 'Group Stickies By Topic' });
+    await userEvent.click(btn);
+
+    await waitFor(() => {
+      const groups = document.querySelectorAll('.cluster-draggable');
+      groups.forEach((group) => {
+        expect(group).toHaveAttribute('draggable', 'true');
+      });
+      expect(groups.length).toBe(2);
+    });
+  });
 });
--- a/frontend/src/types/types.d.ts
+++ b/frontend/src/types/types.d.ts
@@ -11,3 +11,12 @@ export type Cluster = {
  label: string;
  noteIds: string[];
 };
+
+export type ClusterResponse = {
+  clusters: Cluster[];
+  score: number;
+};
+
+export type RankedCluster = Cluster & {
+  rank: number;
+};
--- a/package-lock.json
+++ b/package-lock.json
@@ -0,0 +1,21 @@
+{
+  "name": "kongruity",
+  "lockfileVersion": 3,
+  "requires": true,
+  "packages": {
+    "": {
+      "dependencies": {
+        "split2": "^4.2.0"
+      }
+    },
+    "node_modules/split2": {
+      "version": "4.2.0",
+      "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+      "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+      "license": "ISC",
+      "engines": {
+        "node": ">= 10.x"
+      }
+    }
+  }
+}
--- a/package.json
+++ b/package.json
@@ -0,0 +1,5 @@
+{
+  "dependencies": {
+    "split2": "^4.2.0"
+  }
+}
Author	SHA1	Message	Date
kjannette	e11122ed14	Merge pull request 'FEAT-nonblocking-backend-io-imporvements' (#1 ) from FEAT-nonblocking-backend-io-imporvements into master Reviewed-on: #1	2026-08-01 05:43:24 +00:00
KS Jannette	7b47e46852	Cleanup	2026-08-01 01:42:43 -04:00
KS Jannette	62477d009c	Updated Claude model	2026-08-01 01:35:27 -04:00
KS Jannette	ee6fa9e576	Add nonblocking/asyn I/O operations	2026-08-01 01:06:19 -04:00
kjannette	90035b3568	Update README.ms hotfix	2026-07-31 13:45:35 +00:00
S Jannette	9636be78c9	Fix typos and enhance README content Improved clarity in the README.	2026-03-06 03:52:47 -05:00
S Jannette	a50ca5c171	Add MIT License to the project	2026-02-25 18:50:59 -05:00
S Jannette	e271702bca	Fix typo in README.md regarding implementation planning Corrected 'incoporated in' to 'incoporated into' for clarity.	2026-02-24 21:11:24 -05:00
S Jannette	d3d76b1d0d	Refine README.md content for clarity and accuracy Updated descriptions for clarity and corrected typos.	2026-02-24 21:10:51 -05:00
S Jannette	8cae0665db	Merge pull request #16 from kjannette/refinements Refinements	2026-02-24 21:04:56 -05:00
KS Jannette	21453aab7d	UI development	2026-02-24 21:02:43 -05:00
KS Jannette	691186e02c	upgraded scoring algorithm model	2026-02-24 20:55:19 -05:00
S Jannette	fd48aaab4d	Merge pull request #15 from kjannette/test-data-xfer further DB infra buildout	2026-02-24 19:41:31 -05:00
KS Jannette	0b56d0d6c4	further DB infra buildout	2026-02-24 19:38:35 -05:00
S Jannette	0d22998c2e	Merge pull request #14 from kjannette/db-build db created, added tests -- tentative	2026-02-24 19:26:55 -05:00
KS Jannette	ce7bc2cde9	db created, added tests -- tentative	2026-02-24 19:25:42 -05:00
S Jannette	0ee426e1f3	Update README.md	2026-02-24 14:12:30 -05:00
KS Jannette	90a19e4525	edit	2026-02-24 13:51:19 -05:00
S Jannette	8b555bf729	Fix typo in README.md Corrected a typo in the README regarding 'simply'.	2026-02-24 13:48:33 -05:00
KS Jannette	c8ee194bc5	hot edit readme	2026-02-24 13:47:24 -05:00
S Jannette	f77362e684	Merge pull request #13 from kjannette/workflow-UI Add UI/logical temporal ordering feature	2026-02-24 13:32:29 -05:00
KS Jannette	9ba13206c5	Add UI/logical temporal ordering feature	2026-02-24 13:32:06 -05:00
KS Jannette	0c7baeb9d8	hotfix	2026-02-24 13:12:47 -05:00
S Jannette	07b0309adb	Merge pull request #12 from kjannette/query-validation Query validation	2026-02-24 13:07:39 -05:00
KS Jannette	7ec074a043	update documentation	2026-02-24 13:06:56 -05:00
KS Jannette	b7757ca406	add second layer of result validation	2026-02-24 12:56:18 -05:00
KS Jannette	d55f58985b	more	2026-02-14 08:11:36 -05:00
KS Jannette	61835ae3aa	format	2026-02-13 16:43:00 -05:00
KS Jannette	091315c188	m	2026-02-13 16:35:26 -05:00
KS Jannette	089b9e7e9d	more	2026-02-13 16:31:39 -05:00
KS Jannette	3e7978c918	cccccLean	2026-02-13 15:41:50 -05:00
KS Jannette	317468b1ed	more	2026-02-13 15:34:29 -05:00
KS Jannette	a720673896	clean	2026-02-13 15:27:10 -05:00
KS Jannette	1a85c6e1e0	fin	2026-02-13 15:21:02 -05:00
S Jannette	7f4c322280	Merge pull request #11 from kjannette/frontFact more	2026-02-13 15:15:08 -05:00
KS Jannette	5208327b5d	more	2026-02-13 15:12:18 -05:00
KS Jannette	ac76871d60	hottie	2026-02-13 13:47:43 -05:00
S Jannette	66c86c2625	Merge pull request #10 from kjannette/refact5 clean	2026-02-13 13:45:24 -05:00
KS Jannette	56d0c21b79	clean	2026-02-13 13:45:00 -05:00
S Jannette	37e81e773e	Merge pull request #9 from kjannette/refactor4 more	2026-02-13 13:42:52 -05:00
KS Jannette	534da11217	more	2026-02-13 13:42:31 -05:00
S Jannette	2212755772	Merge pull request #8 from kjannette/refact3 cleanup	2026-02-13 13:35:29 -05:00