From b607ee91219f04de7520de6b87ef20d31124e97a Mon Sep 17 00:00:00 2001 From: KS Jannette Date: Sun, 2 Aug 2026 06:13:01 -0400 Subject: [PATCH 1/2] Building Slack integration --- README.md | 49 +- backend/app.ts | 5 + backend/config/normalizers.ts | 79 +++ backend/config/providers.ts | 14 +- backend/lib/queue.ts | 15 + backend/lib/slackText.ts | 53 ++ backend/routes/slack.routes.ts | 158 ++++++ backend/routes/webhooks.routes.ts | 12 +- backend/services/slack.service.ts | 284 +++++++++++ backend/tests/normalizers.test.ts | 3 +- backend/tests/queue.test.ts | 34 ++ backend/tests/slack.integration.test.ts | 283 +++++++++++ backend/tests/slack.normalizer.test.ts | 245 +++++++++ backend/tests/slack.routes.test.ts | 402 +++++++++++++++ backend/tests/slack.service.test.ts | 650 ++++++++++++++++++++++++ backend/tests/slackText.test.ts | 73 +++ 16 files changed, 2351 insertions(+), 8 deletions(-) create mode 100644 backend/lib/slackText.ts create mode 100644 backend/routes/slack.routes.ts create mode 100644 backend/services/slack.service.ts create mode 100644 backend/tests/slack.integration.test.ts create mode 100644 backend/tests/slack.normalizer.test.ts create mode 100644 backend/tests/slack.routes.test.ts create mode 100644 backend/tests/slack.service.test.ts create mode 100644 backend/tests/slackText.test.ts diff --git a/README.md b/README.md index 14aa8a6..93b4678 100644 --- a/README.md +++ b/README.md @@ -73,8 +73,11 @@ The shared infrastructure that third-party integrations are built on. See [ARCHI | --- | --- | --- | | `POST /v1/notes` | `Authorization: Bearer ` | Bulk push notes. Body `{ notes: [{ id, text, author, x?, y?, color? }] }`, max 5000 per request. | | `POST /v1/webhooks/:provider` | Per-provider request signature | Inbound provider deliveries. Acks immediately, inserts in the background. | +| `POST /v1/slack/commands` | Slack `v0` request signature | The `/sticky` slash command. Form-encoded rather than JSON, and its response body is shown to the user in Slack. | -Registered providers live in `backend/config/providers.ts`. `rest` and `linear` carry payload mappings today; `slack`, `github`, and `jira` declare transport and OAuth endpoints only, so adding one of those integrations means writing a single normalizer rather than new plumbing. +Registered providers live in `backend/config/providers.ts`. `rest`, `linear`, and `slack` carry payload mappings today; `github` and `jira` declare transport and OAuth endpoints only, so adding one of those integrations means writing a single normalizer rather than new plumbing. + +A provider may also declare an async `enrich` hook. It runs inside the retry queue *after* the request is acknowledged, which is where any network call belongs: normalizers stay pure and synchronous so a delivery can be acked inside Slack's three-second budget. Slack needs this because a reaction event names a message without carrying its text. Deliveries are deduplicated on `(provider, external_id)` in the `ingest_events` table, which matters beyond tidiness: duplicate notes compress intra-cluster distance and depress the cohesion score. @@ -83,13 +86,14 @@ Deliveries are deduplicated on `(provider, external_id)` in the `ingest_events` 1. Add a slug to `ProviderSlug` in `backend/types/integration.ts`. 2. Add an entry to `providers` in `backend/config/providers.ts` with its signature scheme, header names, and OAuth endpoints. 3. Write one normalizer in `backend/config/normalizers.ts` mapping that provider's payload to notes. -4. Set the provider's signing secret and OAuth credentials in `backend/.env`. +4. Add an `enrich` hook only if the provider's payload references content it does not include. +5. Set the provider's signing secret and OAuth credentials in `backend/.env`. ## Development Roadmap ### Ingestion — pulling tagged artifacts in -- [ ] **Slack** — where decisions actually get made; a `:sticky:` emoji reaction fires an Events API webhook that pulls the message in. +- [x] **Slack** — where decisions actually get made; `/sticky` captures a new note, and an emoji reaction (`:pushpin:` by default) fires an Events API webhook that pulls an existing message in. - [ ] **Microsoft Teams** — same capture gesture for enterprise shops; message extension plus Graph change notifications. - [ ] **Jira** — label- or mention-triggered webhook scoped by JQL. (This is where comments typically carry half the backlog's context.) - [ ] **Linear** — engineering-side tickets and threads; label-triggered GraphQL webhook. @@ -155,6 +159,45 @@ EOF The key must decode to exactly 32 bytes; the backend refuses to encrypt otherwise rather than falling back to something weaker. +## Slack integration + +Two capture gestures, both landing in the same ingest pipeline: + +- **`/sticky `** creates a note from what you type. +- **Reacting with an emoji** captures the message someone already wrote. The trigger defaults to `:pushpin:` and is set by `SLACK_CAPTURE_REACTION`. + +### 1. Create the Slack app + +At [api.slack.com/apps](https://api.slack.com/apps), create an app from scratch in your workspace. Under **OAuth & Permissions**, add the bot token scopes `channels:history`, `reactions:read`, `users:read`, and `commands`, then install the app to the workspace and copy the **Bot User OAuth Token** (`xoxb-…`). Under **Basic Information**, copy the **Signing Secret**. + +```bash +cat >> backend/.env << 'EOF' +SLACK_BOT_TOKEN=xoxb- +SLACK_CAPTURE_REACTION=pushpin +EOF +``` + +`SLACK_SIGNING_SECRET` is already in the integration variables above. There is no OAuth flow here on purpose: a single workspace reading credentials from the environment is far less machinery than an install dance, and the OAuth endpoints in the provider registry stay unused until multi-workspace support is actually needed. + +### 2. Expose your local backend + +Slack only delivers to a public HTTPS URL, so a tunnel is required for local development. + +```bash +brew install ngrok +ngrok http 3001 +``` + +Copy the `https://` forwarding URL ngrok prints. It changes every restart on the free plan, and both URLs below have to be updated when it does. + +### 3. Point Slack at the tunnel + +Under **Slash Commands**, create `/sticky` with the request URL `https:///v1/slack/commands`. + +Under **Event Subscriptions**, enable events and set the request URL to `https:///v1/webhooks/slack`. Slack immediately sends a `url_verification` challenge, which the webhook route answers before signature checking, so the backend must already be running when you save. Subscribe to the bot event `reaction_added`. + +Reinstall the app if Slack prompts you, then invite the bot to any channel you want to capture from with `/invite @`. A reaction in a channel the bot is not a member of arrives as a `not_in_channel` error, which is classified as permanent and will not be retried. + ### 3. Set up/run the database Start DB for local development (assumes local dev env MacOS and Homebrew installed) diff --git a/backend/app.ts b/backend/app.ts index 6ed3735..e312978 100644 --- a/backend/app.ts +++ b/backend/app.ts @@ -3,6 +3,7 @@ import cors from 'cors'; import notesRouter from './routes/notes.routes.js'; import ingestRouter from './routes/ingest.routes.js'; import webhooksRouter from './routes/webhooks.routes.js'; +import slackRouter from './routes/slack.routes.js'; const app = express(); @@ -13,6 +14,10 @@ app.use(cors()); // Moving this below express.json silently breaks every signature check. app.use('/v1/webhooks', express.raw({ type: '*/*', limit: '2mb' }), webhooksRouter); +// Slash commands are form-encoded and signed over the same raw bytes, so this +// router sits above express.json for the reason described above. +app.use('/v1/slack', express.raw({ type: '*/*', limit: '2mb' }), slackRouter); + app.use(express.json({ limit: '2mb' })); app.use('/v1/notes', ingestRouter); app.use('/v1/notes', notesRouter); diff --git a/backend/config/normalizers.ts b/backend/config/normalizers.ts index aa94b6d..fb40894 100644 --- a/backend/config/normalizers.ts +++ b/backend/config/normalizers.ts @@ -111,3 +111,82 @@ export const normalizeLinear = (payload: unknown): NormalizedDelivery => { ], }; }; + +/** Stands in for the message body until the enrichment hook backfills it. */ +export const SLACK_PENDING_TEXT = '(pending Slack message text)'; + +const DEFAULT_CAPTURE_REACTION = 'pushpin'; + +/** Slack suffixes skin-tone modifiers onto emoji names; only the base name identifies the reaction. */ +const baseReaction = (name: string): string => name.split('::')[0]; + +const nested = (value: unknown): Record => + typeof value === 'object' && value !== null && !Array.isArray(value) + ? (value as Record) + : {}; + +/** + * Slack Events API. Only one configured reaction is interesting; every other + * delivery is ordinary traffic and yields no notes rather than an error. + * + * A reaction_added event names a message but never carries its text, and this + * runs ahead of the HTTP ack inside Slack's three-second budget, so it cannot + * go fetch it. The note therefore carries placeholder text and needsMessageText, + * which is the enrichment hook's cue to backfill the real body out of band. + */ +export const normalizeSlack = (payload: unknown): NormalizedDelivery => { + const body = asRecord(payload); + + if (body.type !== 'event_callback') { + throw new NormalizationError('Slack payload is not an event_callback'); + } + + const externalId = asString(body.event_id); + if (!externalId) { + throw new NormalizationError('Slack payload is missing event_id'); + } + + const event = nested(body.event); + if (event.type !== 'reaction_added') { + return { externalId, notes: [] }; + } + + // Read per call: the captured reaction is operator configuration, not a build-time constant. + const configured = asString(process.env.SLACK_CAPTURE_REACTION) ?? DEFAULT_CAPTURE_REACTION; + const reaction = asString(event.reaction); + if (!reaction || baseReaction(reaction) !== baseReaction(configured)) { + return { externalId, notes: [] }; + } + + const item = nested(event.item); + const channelId = asString(item.channel); + const messageTs = asString(item.ts); + if (item.type !== 'message' || !channelId || !messageTs) { + return { externalId, notes: [] }; + } + + const teamId = asString(body.team_id); + const reactedBy = asString(event.user); + const authorUserId = asString(event.item_user); + + return { + externalId, + notes: [ + { + id: noteId('slack', externalId), + text: SLACK_PENDING_TEXT, + author: authorUserId ?? reactedBy ?? 'unknown', + sourceMeta: { + ...provenance({ provider: 'slack', externalId }), + needsMessageText: true, + channelId, + messageTs, + reaction, + ...(teamId !== undefined ? { teamId } : {}), + ...(reactedBy !== undefined ? { reactedBy } : {}), + ...(authorUserId !== undefined ? { authorUserId } : {}), + }, + }, + ], + }; +}; diff --git a/backend/config/providers.ts b/backend/config/providers.ts index f031936..d32cc37 100644 --- a/backend/config/providers.ts +++ b/backend/config/providers.ts @@ -3,7 +3,8 @@ import type { ProviderSlug, SignatureScheme, } from '../types/integration.js'; -import { normalizeLinear, normalizeRest } from './normalizers.js'; +import { normalizeLinear, normalizeRest, normalizeSlack } from './normalizers.js'; +import { enrichSlackDelivery } from '../services/slack.service.js'; export type ChallengeResponse = { status: number; body: unknown }; @@ -22,6 +23,15 @@ export type ProviderConfig = { * infrastructure; the payload mapping belongs to the integration itself. */ normalize?: (payload: unknown) => NormalizedDelivery; + /** + * Runs after the ack, inside the retry queue, so it may make network calls. + * Receives the raw payload because some providers deliver a reference rather + * than content: a Slack reaction names a message without including its text. + */ + enrich?: ( + delivery: NormalizedDelivery, + payload: unknown + ) => Promise; oauth?: { authorizeUrl: string; tokenUrl: string; @@ -63,6 +73,8 @@ export const providers: Record = { timestampHeader: 'x-slack-request-timestamp', secretEnvVar: 'SLACK_SIGNING_SECRET', challenge: slackChallenge, + normalize: normalizeSlack, + enrich: enrichSlackDelivery, oauth: { authorizeUrl: 'https://slack.com/oauth/v2/authorize', tokenUrl: 'https://slack.com/api/oauth.v2.access', diff --git a/backend/lib/queue.ts b/backend/lib/queue.ts index 780df2d..27ca7d0 100644 --- a/backend/lib/queue.ts +++ b/backend/lib/queue.ts @@ -22,12 +22,27 @@ const backoffDelay = (attempt: number): number => { return window + Math.random() * window; }; +/** + * A job signals that retrying cannot change the outcome by throwing an error + * carrying `permanent: true`. A revoked credential or a deleted channel is a + * settled fact; spending the remaining attempts on it only delays every job + * behind it and buries the real reason under retry noise. + */ +const isPermanent = (err: unknown): boolean => + typeof err === 'object' && + err !== null && + (err as { permanent?: unknown }).permanent === true; + const runEntry = async (entry: QueueEntry): Promise => { for (let attempt = 1; ; attempt += 1) { try { await entry.job(); return; } catch (err) { + if (isPermanent(err)) { + console.error(`[queue] job "${entry.name}" abandoned as permanent`, err); + return; + } if (attempt >= settings.maxAttempts) { console.error( `[queue] job "${entry.name}" abandoned after ${attempt} attempt(s)`, diff --git a/backend/lib/slackText.ts b/backend/lib/slackText.ts new file mode 100644 index 0000000..53bd9b2 --- /dev/null +++ b/backend/lib/slackText.ts @@ -0,0 +1,53 @@ +/** + * Slack delivers mrkdwn, not plain text. Link syntax and HTML entities that + * survive into a note degrade embedding quality and read as noise on a sticky, + * so they are unwrapped here. Pure and dependency-free so both the normalizer + * and the API client can use it. + */ + +const ENTITIES: Record = { + '&': '&', + '<': '<', + '>': '>', +}; + +const unescapeEntities = (text: string): string => + text.replace(/&(?:amp|lt|gt);/g, (match) => ENTITIES[match] ?? match); + +const splitOnce = (value: string, separator: string): [string, string | undefined] => { + const index = value.indexOf(separator); + if (index === -1) return [value, undefined]; + return [value.slice(0, index), value.slice(index + separator.length)]; +}; + +const BROADCASTS: readonly string[] = ['here', 'channel', 'everyone']; + +const MARKUP = /<([^<>]*)>/g; + +const unwrap = (inner: string): string => { + const [target, rawLabel] = splitOnce(inner, '|'); + const label = rawLabel !== undefined && rawLabel.length > 0 ? rawLabel : undefined; + + if (target.startsWith('@')) return `@${label ?? target.slice(1)}`; + if (target.startsWith('#')) return `#${label ?? target.slice(1)}`; + + if (target.startsWith('!')) { + const special = target.slice(1); + if (BROADCASTS.includes(special)) return `@${special}`; + return label ?? `@${special}`; + } + + if (target.startsWith('mailto:')) return label ?? target.slice('mailto:'.length); + + return label ?? target; +}; + +/** + * Entities are unescaped only after markup is unwrapped. Slack escapes a + * literal `<` as `<` precisely so it is not read as markup, and reversing + * that order would turn user text into parsed link syntax. + */ +export const cleanSlackText = (input: string): string => + unescapeEntities(input.replace(MARKUP, (_match, inner: string) => unwrap(inner))) + .replace(/[ \t]+$/gm, '') + .trim(); diff --git a/backend/routes/slack.routes.ts b/backend/routes/slack.routes.ts new file mode 100644 index 0000000..b33cbc2 --- /dev/null +++ b/backend/routes/slack.routes.ts @@ -0,0 +1,158 @@ +import { Router, type NextFunction, type Request, type Response } from 'express'; +import { verifySignature } from '../lib/signatures.js'; +import { cleanSlackText } from '../lib/slackText.js'; +import { enqueue } from '../lib/queue.js'; +import { + markDone, + markFailed, + markProcessing, + recordDelivery, +} from '../db/ingest_events.dao.js'; +import { createNotes } from '../db/notes.dao.js'; +import type { NoteInput } from '../types/domain.js'; + +/** + * Slash commands are a separate surface from the event webhook: the body is + * form-encoded rather than JSON, and Slack renders whatever this route returns + * straight into the channel. Mount at '/v1/slack' behind a raw body parser so + * signature verification sees the exact bytes Slack signed. + */ +const router = Router(); + +const SLASH_COMMAND = '/sticky'; + +/** notes.id is VARCHAR(64). */ +const NOTE_ID_MAX = 64; + +/** Long enough to confirm the note, short enough not to flood the channel. */ +const ECHO_MAX = 120; + +type SlackCommandResponse = { + response_type: 'ephemeral'; + text: string; +}; + +const field = (form: URLSearchParams, name: string): string | undefined => { + const value = form.get(name); + if (value === null || value.length === 0) return undefined; + return value; +}; + +const truncate = (value: string, max: number): string => + value.length <= max ? value : `${value.slice(0, max - 1)}…`; + +/** + * Slack shows the user a generic failure banner instead of the body on any + * non-2xx, so every outcome the user is meant to read is a 200. + */ +const ephemeral = (res: Response, text: string): void => { + const body: SlackCommandResponse = { response_type: 'ephemeral', text }; + res.status(200).json(body); +}; + +router.post('/commands', async (req: Request, res: Response, next: NextFunction) => { + try { + const raw = Buffer.isBuffer(req.body) ? req.body : Buffer.alloc(0); + + const secret = process.env.SLACK_SIGNING_SECRET; + if (!secret) { + console.error('No signing secret configured for provider "slack"'); + res.status(500).json({ error: 'Slack is not configured' }); + return; + } + + // Verified against the raw bytes before any parsing: URLSearchParams + // re-encoding would not reproduce what Slack signed. + const signature = verifySignature({ + provider: 'slack', + rawBody: raw, + headers: req.headers, + secret, + }); + + if (!signature.ok) { + res.status(401).json({ error: 'Invalid signature' }); + return; + } + + const form = new URLSearchParams(raw.toString('utf8')); + const command = field(form, 'command'); + const userId = field(form, 'user_id'); + const userName = field(form, 'user_name'); + const teamId = field(form, 'team_id'); + const channelId = field(form, 'channel_id'); + const channelName = field(form, 'channel_name'); + const triggerId = field(form, 'trigger_id'); + + if (command !== undefined && command !== SLASH_COMMAND) { + ephemeral(res, `Unknown command ${command}.`); + return; + } + + const text = (field(form, 'text') ?? '').trim(); + + // Cleaning happens before the delivery is recorded: markup that reduces to + // nothing must not consume the externalId that dedups a Slack retry. + const cleaned = cleanSlackText(text); + if (cleaned.length === 0) { + ephemeral(res, `Usage: ${SLASH_COMMAND} `); + return; + } + + // trigger_id is unique per invocation, so a retry of a command we were too + // slow to answer dedups against the first attempt instead of duplicating it. + const externalId = + triggerId ?? `${teamId ?? ''}:${userId ?? ''}:${Date.now()}`; + + const eventId = await recordDelivery({ provider: 'slack', externalId }); + if (eventId === null) { + ephemeral(res, 'Already captured.'); + return; + } + + const sourceMeta: Record = { + provider: 'slack', + externalId, + receivedAt: new Date().toISOString(), + via: 'slash-command', + }; + + const optionalMeta: Record = { + channelId, + channelName, + authorHandle: userName, + authorUserId: userId, + teamId, + }; + + for (const [key, value] of Object.entries(optionalMeta)) { + if (value !== undefined) sourceMeta[key] = value; + } + + const note: NoteInput = { + id: `slack_${externalId}`.slice(0, NOTE_ID_MAX), + text: cleaned, + author: userName ?? userId ?? 'unknown', + sourceMeta, + }; + + // Slack abandons the command after three seconds, so the write happens + // behind the ack rather than in front of it. + enqueue(`slack:${externalId}`, async () => { + await markProcessing(eventId); + try { + await createNotes([note]); + await markDone(eventId); + } catch (err) { + await markFailed(eventId, err instanceof Error ? err.message : String(err)); + throw err; + } + }); + + ephemeral(res, `Added to Kongruity: "${truncate(cleaned, ECHO_MAX)}"`); + } catch (err) { + next(err); + } +}); + +export default router; diff --git a/backend/routes/webhooks.routes.ts b/backend/routes/webhooks.routes.ts index d00ee41..698d53b 100644 --- a/backend/routes/webhooks.routes.ts +++ b/backend/routes/webhooks.routes.ts @@ -96,7 +96,10 @@ router.post('/:provider', async (req: Request, res: Response, next: NextFunction enqueue(`${slug}:${delivery.externalId}`, async () => { await markProcessing(eventId); try { - await createNotes(notes); + const enriched = config.enrich + ? await config.enrich(delivery, payload) + : delivery; + await createNotes(enriched.notes); await markDone(eventId); } catch (err) { await markFailed(eventId, err instanceof Error ? err.message : String(err)); @@ -104,7 +107,12 @@ router.post('/:provider', async (req: Request, res: Response, next: NextFunction } }); - res.status(200).json({ accepted: true, notes: notes.length }); + // The count is omitted for providers that enrich, because enrichment runs + // after this response and may drop notes whose content cannot be fetched. + // No number is better than one that is sometimes wrong. + res.status(200).json( + config.enrich ? { accepted: true } : { accepted: true, notes: notes.length } + ); } catch (err) { next(err); } diff --git a/backend/services/slack.service.ts b/backend/services/slack.service.ts new file mode 100644 index 0000000..ec662d4 --- /dev/null +++ b/backend/services/slack.service.ts @@ -0,0 +1,284 @@ +import { requestJson } from '../lib/httpClient.js'; +import { cleanSlackText } from '../lib/slackText.js'; +import type { NormalizedDelivery } from '../types/integration.js'; +import type { NoteInput } from '../types/domain.js'; + +const SLACK_API_BASE = 'https://slack.com/api'; + +export type SlackErrorKind = 'permanent' | 'retryable'; + +/** + * Errors a retry cannot fix: a bad credential, a missing scope, or a message + * that is gone. Everything absent from this list is treated as retryable, + * including error strings we do not recognize, so a transient failure Slack + * adds tomorrow is not silently dropped today. The queue bounds attempts, so + * an unknown-but-really-permanent error wastes a few calls rather than looping. + */ +const PERMANENT_ERRORS: ReadonlySet = new Set([ + 'no_token', + 'invalid_auth', + 'not_authed', + 'account_inactive', + 'token_revoked', + 'token_expired', + 'no_permission', + 'missing_scope', + 'channel_not_found', + 'not_in_channel', + 'message_not_found', + 'user_not_found', + 'invalid_arguments', + 'invalid_form_data', + 'is_archived', +]); + +export class SlackApiError extends Error { + readonly method: string; + readonly slackError: string; + readonly kind: SlackErrorKind; + /** The flag lib/queue.ts reads to stop retrying a settled failure. */ + readonly permanent: boolean; + + constructor(method: string, slackError: string) { + super(`Slack API ${method} failed: ${slackError}`); + this.name = 'SlackApiError'; + this.method = method; + this.slackError = slackError; + this.kind = PERMANENT_ERRORS.has(slackError) ? 'permanent' : 'retryable'; + this.permanent = this.kind === 'permanent'; + } +} + +export const isSlackApiError = (err: unknown): err is SlackApiError => + err instanceof SlackApiError; + +const asRecord = (value: unknown): Record | undefined => + typeof value === 'object' && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; + +const asNonEmptyString = (value: unknown): string | undefined => + typeof value === 'string' && value.length > 0 ? value : undefined; + +/** + * Posts a form-encoded Web API call and unwraps Slack's envelope. + * + * Slack answers HTTP 200 even for failures, putting the outcome in the body as + * `{ ok: false, error: "invalid_auth" }`. requestJson cannot see that, so the + * `ok` field is checked here; transport-level failures (429, 5xx, timeouts) are + * left to requestJson's retry loop and surface as HttpRequestError unchanged. + */ +export const callSlack = async ( + method: string, + params: Record = {} +): Promise => { + // Read at call time, never at module load: the token may be rotated into the + // environment after import, and tests set and unset it per case. + // Trimmed because a token pasted into a .env file often carries a newline. + const token = process.env.SLACK_BOT_TOKEN?.trim(); + if (token === undefined || token === '') { + throw new SlackApiError(method, 'no_token'); + } + + const body = await requestJson(`${SLACK_API_BASE}/${method}`, { + method: 'POST', + headers: { + authorization: `Bearer ${token}`, + 'content-type': 'application/x-www-form-urlencoded; charset=utf-8', + }, + body: new URLSearchParams(params).toString(), + }); + + const envelope = asRecord(body); + if (envelope === undefined || envelope.ok !== true) { + const slackError = asNonEmptyString(envelope?.error) ?? 'unknown_error'; + throw new SlackApiError(method, slackError); + } + + return body as T; +}; + +/** + * Credential health check. `ok: true` means Slack accepted the token, so the + * identity fields are present in practice; they fall back to empty strings + * rather than throwing, since a malformed success body is not a credential + * problem and should not read as one. + */ +export const authTest = async (): Promise<{ + teamId: string; + teamName: string; + botUserId: string; +}> => { + const body = await callSlack('auth.test'); + const envelope = asRecord(body) ?? {}; + + return { + teamId: asNonEmptyString(envelope.team_id) ?? '', + teamName: asNonEmptyString(envelope.team) ?? '', + botUserId: asNonEmptyString(envelope.user_id) ?? '', + }; +}; + +/** userId to resolved display name. Only successful lookups are stored. */ +const displayNames = new Map(); + +export const resetSlackCaches = (): void => { + displayNames.clear(); +}; + +const pickDisplayName = (body: unknown, userId: string): string => { + const user = asRecord(asRecord(body)?.user); + const profile = asRecord(user?.profile); + + return ( + asNonEmptyString(profile?.display_name) ?? + asNonEmptyString(profile?.real_name) ?? + asNonEmptyString(user?.real_name) ?? + asNonEmptyString(user?.name) ?? + userId + ); +}; + +/** + * Resolves a human-readable name for a Slack user, memoized for the process. + * A permanent failure degrades to the raw user ID: a name we cannot look up is + * cosmetic, and failing the whole ingest over it would lose the note. Retryable + * failures propagate so the queue can try again. + */ +export const getUserDisplayName = async (userId: string): Promise => { + const cached = displayNames.get(userId); + if (cached !== undefined) { + return cached; + } + + let name: string; + try { + name = pickDisplayName(await callSlack('users.info', { user: userId }), userId); + } catch (err) { + if (isSlackApiError(err) && err.kind === 'permanent') { + // Deliberately not cached: a missing scope can be granted without a + // restart, and caching the degraded answer would outlive the cause. + return userId; + } + throw err; + } + + displayNames.set(userId, name); + return name; +}; + +/** + * Slack returns the whole thread when `conversations.replies` is given a reply's + * ts, so the exact timestamp is preferred over position; the first message is + * the fallback for `conversations.history`, which returns only the one asked for. + */ +const pickMessage = (body: unknown, ts: string): Record | undefined => { + const messages = asRecord(body)?.messages; + if (!Array.isArray(messages)) { + return undefined; + } + + const candidates = messages + .map(asRecord) + .filter((entry): entry is Record => entry !== undefined); + + return candidates.find((entry) => entry.ts === ts) ?? candidates[0]; +}; + +/** + * Reads one message's text by timestamp. `conversations.history` cannot see a + * threaded reply, so `conversations.replies` is tried before giving up: a + * reaction on a reply is otherwise indistinguishable from a deleted message. + * Returns null when there is nothing worth clustering. + */ +export const getMessageText = async ( + channelId: string, + ts: string +): Promise<{ text: string; userId?: string } | null> => { + const history = await callSlack('conversations.history', { + channel: channelId, + latest: ts, + oldest: ts, + inclusive: 'true', + limit: '1', + }); + + let message = pickMessage(history, ts); + + if (message === undefined) { + const replies = await callSlack('conversations.replies', { + channel: channelId, + ts, + limit: '1', + inclusive: 'true', + }); + message = pickMessage(replies, ts); + } + + if (message === undefined) { + return null; + } + + const raw = asNonEmptyString(message.text); + const text = raw === undefined ? '' : cleanSlackText(raw); + if (text === '') { + return null; + } + + const userId = asNonEmptyString(message.user); + return userId === undefined ? { text } : { text, userId }; +}; + +/** + * Fills in what a Slack event referenced but did not carry. Runs after the ack, + * inside the retry queue, so the network calls here are safe to be slow. + * + * Notes are processed in sequence rather than in parallel: a delivery carries + * one or two of them, and serializing keeps a burst of reactions from spending + * the per-method rate limit all at once. + */ +export const enrichSlackDelivery = async ( + delivery: NormalizedDelivery, + _payload: unknown +): Promise => { + const notes: NoteInput[] = []; + + for (const note of delivery.notes) { + const meta = note.sourceMeta; + + if (meta === undefined || meta.needsMessageText !== true) { + notes.push(note); + continue; + } + + const channelId = asNonEmptyString(meta.channelId); + const messageTs = asNonEmptyString(meta.messageTs); + + // No usable reference means the placeholder text can never be resolved, so + // the note is dropped for the same reason an unfetchable message is. + const message = + channelId === undefined || messageTs === undefined + ? null + : await getMessageText(channelId, messageTs); + + if (message === null) { + continue; + } + + // Stripped unconditionally so a stale marker can never reach the database. + const sourceMeta: Record = { ...meta }; + delete sourceMeta.needsMessageText; + + const authorUserId = asNonEmptyString(meta.authorUserId) ?? message.userId; + if (authorUserId === undefined) { + notes.push({ ...note, text: message.text, sourceMeta }); + continue; + } + + const author = await getUserDisplayName(authorUserId); + sourceMeta.authorHandle = author; + notes.push({ ...note, text: message.text, author, sourceMeta }); + } + + return { ...delivery, notes }; +}; diff --git a/backend/tests/normalizers.test.ts b/backend/tests/normalizers.test.ts index fa60475..957e509 100644 --- a/backend/tests/normalizers.test.ts +++ b/backend/tests/normalizers.test.ts @@ -322,7 +322,6 @@ describe('normalize', () => { expect(() => normalize('github', {})).toThrow(NormalizationError); expect(() => normalize('github', {})).toThrow(/No normalizer registered/); expect(() => normalize('jira', {})).toThrow(/No normalizer registered/); - expect(() => normalize('slack', {})).toThrow(/No normalizer registered/); }); }); @@ -330,11 +329,11 @@ describe('hasNormalizer', () => { it('should be true for providers with a normalizer', () => { expect(hasNormalizer('rest')).toBe(true); expect(hasNormalizer('linear')).toBe(true); + expect(hasNormalizer('slack')).toBe(true); }); it('should be false for providers awaiting an integration', () => { expect(hasNormalizer('github')).toBe(false); expect(hasNormalizer('jira')).toBe(false); - expect(hasNormalizer('slack')).toBe(false); }); }); diff --git a/backend/tests/queue.test.ts b/backend/tests/queue.test.ts index 4e866c3..362e997 100644 --- a/backend/tests/queue.test.ts +++ b/backend/tests/queue.test.ts @@ -64,6 +64,40 @@ describe('queue', () => { expect(errors).toHaveBeenCalled(); }); + /** + * The whole point of classifying a failure as permanent: a revoked token or + * a deleted channel cannot be fixed by trying again, and retrying it holds + * up every job behind it. + */ + it('should abandon a job immediately when the error is marked permanent', async () => { + configureQueue({ maxAttempts: 4 }); + let attempts = 0; + + enqueue('permanently-failing', async () => { + attempts += 1; + throw Object.assign(new Error('token_revoked'), { permanent: true }); + }); + + await drain(); + + expect(attempts).toBe(1); + expect(String(errors.mock.calls[0]?.[0])).toContain('abandoned as permanent'); + }); + + it('should still retry a job whose error carries a falsy permanent flag', async () => { + configureQueue({ maxAttempts: 3 }); + let attempts = 0; + + enqueue('transiently-failing', async () => { + attempts += 1; + throw Object.assign(new Error('ratelimited'), { permanent: false }); + }); + + await drain(); + + expect(attempts).toBe(3); + }); + it('should retry a failing job up to the configured cap and then give up', async () => { configureQueue({ maxAttempts: 4 }); let attempts = 0; diff --git a/backend/tests/slack.integration.test.ts b/backend/tests/slack.integration.test.ts new file mode 100644 index 0000000..2f9e6fb --- /dev/null +++ b/backend/tests/slack.integration.test.ts @@ -0,0 +1,283 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import request from 'supertest'; +import { createHmac } from 'node:crypto'; +import app from '../app.js'; +import { configureQueue, drain } from '../lib/queue.js'; + +vi.mock('../db/ingest_events.dao.js', () => ({ + recordDelivery: vi.fn(), + markProcessing: vi.fn(), + markDone: vi.fn(), + markFailed: vi.fn(), + resetStaleProcessing: vi.fn(), +})); + +vi.mock('../db/notes.dao.js', () => ({ + createNotes: vi.fn(), + getAllNotes: vi.fn(), + streamAllNotes: vi.fn(), +})); + +vi.mock('../lib/httpClient.js', () => ({ + requestJson: vi.fn(), +})); + +import { + recordDelivery, + markDone, + markFailed, + markProcessing, +} from '../db/ingest_events.dao.js'; +import { createNotes } from '../db/notes.dao.js'; +import { requestJson } from '../lib/httpClient.js'; +import { resetSlackCaches } from '../services/slack.service.js'; +import type { NoteInput } from '../types/domain.js'; + +const mockRecordDelivery = vi.mocked(recordDelivery); +const mockCreateNotes = vi.mocked(createNotes); +const mockMarkDone = vi.mocked(markDone); +const mockMarkFailed = vi.mocked(markFailed); +const mockMarkProcessing = vi.mocked(markProcessing); +const mockRequestJson = vi.mocked(requestJson); + +const SIGNING_SECRET = 'slack-signing-secret'; +const BOT_TOKEN = 'xoxb-test-token'; + +const reactionEvent = (overrides: Record = {}): string => + JSON.stringify({ + type: 'event_callback', + event_id: 'Ev0PIN1', + team_id: 'T999', + event: { + type: 'reaction_added', + user: 'U_REACTOR', + reaction: 'pushpin', + item_user: 'U_AUTHOR', + item: { type: 'message', channel: 'C555', ts: '1700000000.000100' }, + ...overrides, + }, + }); + +const sign = (body: string, timestamp: string, secret = SIGNING_SECRET): string => + `v0=${createHmac('sha256', secret).update(`v0:${timestamp}:${body}`).digest('hex')}`; + +const postEvent = (body: string) => { + const timestamp = Math.floor(Date.now() / 1000).toString(); + return request(app) + .post('/v1/webhooks/slack') + .set('Content-Type', 'application/json') + .set('x-slack-request-timestamp', timestamp) + .set('x-slack-signature', sign(body, timestamp)) + .send(body); +}; + +/** Routes a Slack Web API call by method name to a canned response. */ +const slackApi = (responses: Record): void => { + mockRequestJson.mockImplementation(async (url: string) => { + const method = url.split('/api/')[1] ?? ''; + if (!(method in responses)) { + throw new Error(`Unexpected Slack method: ${method}`); + } + return responses[method]; + }); +}; + +const historyWith = (text: string, user = 'U_AUTHOR') => ({ + ok: true, + messages: [{ text, user, ts: '1700000000.000100' }], +}); + +const userNamed = (displayName: string) => ({ + ok: true, + user: { id: 'U_AUTHOR', name: 'fallback', profile: { display_name: displayName } }, +}); + +const notesWritten = (): NoteInput[] => { + const call = mockCreateNotes.mock.calls[0]; + return call ? call[0] : []; +}; + +/** + * Exercises the seam the unit tests cannot: the provider registry wiring that + * carries a Slack reaction from the signed request through normalization, the + * ack, the queue, and enrichment into an actual note insert. + */ +describe('Slack reaction capture, end to end', () => { + const originalSecret = process.env.SLACK_SIGNING_SECRET; + const originalToken = process.env.SLACK_BOT_TOKEN; + const originalReaction = process.env.SLACK_CAPTURE_REACTION; + + beforeEach(() => { + vi.clearAllMocks(); + resetSlackCaches(); + process.env.SLACK_SIGNING_SECRET = SIGNING_SECRET; + process.env.SLACK_BOT_TOKEN = BOT_TOKEN; + delete process.env.SLACK_CAPTURE_REACTION; + configureQueue({ baseDelayMs: 0, maxAttempts: 2 }); + mockRecordDelivery.mockResolvedValue(42); + mockCreateNotes.mockResolvedValue([]); + }); + + afterEach(async () => { + await drain(); + const restore = (key: string, value: string | undefined): void => { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + }; + restore('SLACK_SIGNING_SECRET', originalSecret); + restore('SLACK_BOT_TOKEN', originalToken); + restore('SLACK_CAPTURE_REACTION', originalReaction); + }); + + it('should turn a pinned message into a note with fetched text and a resolved author', async () => { + slackApi({ + 'conversations.history': historyWith('Deploys are '), + 'users.info': userNamed('Jane Doe'), + }); + + const res = await postEvent(reactionEvent()); + expect(res.status).toBe(200); + + await drain(); + + const notes = notesWritten(); + expect(notes).toHaveLength(1); + expect(notes[0].text).toBe('Deploys are scary'); + expect(notes[0].author).toBe('Jane Doe'); + expect(notes[0].sourceMeta).toMatchObject({ + provider: 'slack', + externalId: 'Ev0PIN1', + channelId: 'C555', + authorHandle: 'Jane Doe', + }); + expect(mockMarkProcessing).toHaveBeenCalledWith(42); + expect(mockMarkDone).toHaveBeenCalledWith(42); + }); + + /** A placeholder reaching the database would be a visible bug on a sticky. */ + it('should never persist the placeholder text', async () => { + slackApi({ + 'conversations.history': historyWith('real message body'), + 'users.info': userNamed('Jane Doe'), + }); + + await postEvent(reactionEvent()); + await drain(); + + expect(notesWritten()[0].text).not.toMatch(/pending/i); + expect(notesWritten()[0].sourceMeta).not.toHaveProperty('needsMessageText'); + }); + + it('should drop the note when the message cannot be fetched', async () => { + slackApi({ + 'conversations.history': { ok: true, messages: [] }, + 'conversations.replies': { ok: true, messages: [] }, + }); + + await postEvent(reactionEvent()); + await drain(); + + expect(notesWritten()).toEqual([]); + expect(mockMarkDone).toHaveBeenCalledWith(42); + }); + + it('should omit the note count from the ack because enrichment runs after it', async () => { + slackApi({ + 'conversations.history': historyWith('anything'), + 'users.info': userNamed('Jane Doe'), + }); + + const res = await postEvent(reactionEvent()); + + expect(res.body).toEqual({ accepted: true }); + }); + + it('should ignore a reaction that is not the configured one', async () => { + const res = await postEvent(reactionEvent({ reaction: 'tada' })); + + expect(res.status).toBe(200); + await drain(); + + expect(mockRequestJson).not.toHaveBeenCalled(); + expect(notesWritten()).toEqual([]); + }); + + it('should honor a custom capture reaction from the environment', async () => { + process.env.SLACK_CAPTURE_REACTION = 'sticky'; + slackApi({ + 'conversations.history': historyWith('captured by custom emoji'), + 'users.info': userNamed('Jane Doe'), + }); + + await postEvent(reactionEvent({ reaction: 'sticky' })); + await drain(); + + expect(notesWritten()[0].text).toBe('captured by custom emoji'); + }); + + it('should answer the url_verification handshake', async () => { + const res = await postEvent( + JSON.stringify({ type: 'url_verification', challenge: 'abc123' }) + ); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ challenge: 'abc123' }); + }); + + it('should reject a delivery whose signature does not match', async () => { + const body = reactionEvent(); + const timestamp = Math.floor(Date.now() / 1000).toString(); + + const res = await request(app) + .post('/v1/webhooks/slack') + .set('Content-Type', 'application/json') + .set('x-slack-request-timestamp', timestamp) + .set('x-slack-signature', sign(body, timestamp, 'wrong-secret')) + .send(body); + + expect(res.status).toBe(401); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should stop at the dedup layer when Slack redelivers an event', async () => { + mockRecordDelivery.mockResolvedValue(null); + + const res = await postEvent(reactionEvent()); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ duplicate: true }); + await drain(); + expect(mockCreateNotes).not.toHaveBeenCalled(); + }); + + /** A failed enrichment must leave the delivery failed, not silently done. */ + it('should mark the delivery failed when the Slack API rejects the fetch', async () => { + mockRequestJson.mockResolvedValue({ ok: false, error: 'channel_not_found' }); + + await postEvent(reactionEvent()); + await drain(); + + expect(mockCreateNotes).not.toHaveBeenCalled(); + expect(mockMarkFailed).toHaveBeenCalledWith(42, expect.stringContaining('channel_not_found')); + expect(mockMarkDone).not.toHaveBeenCalled(); + }); + + /** A missing channel is settled; retrying it only delays the queue. */ + it('should not retry a permanent Slack failure', async () => { + mockRequestJson.mockResolvedValue({ ok: false, error: 'channel_not_found' }); + + await postEvent(reactionEvent()); + await drain(); + + expect(mockRequestJson).toHaveBeenCalledTimes(1); + }); + + it('should retry a rate-limited Slack failure until the attempt cap', async () => { + mockRequestJson.mockResolvedValue({ ok: false, error: 'ratelimited' }); + + await postEvent(reactionEvent()); + await drain(); + + expect(mockRequestJson).toHaveBeenCalledTimes(2); + }); +}); diff --git a/backend/tests/slack.normalizer.test.ts b/backend/tests/slack.normalizer.test.ts new file mode 100644 index 0000000..b579986 --- /dev/null +++ b/backend/tests/slack.normalizer.test.ts @@ -0,0 +1,245 @@ +import { describe, it, expect, afterEach } from 'vitest'; +import { + NormalizationError, + SLACK_PENDING_TEXT, + isNoteInputArray, + normalizeSlack, +} from '../config/normalizers.js'; +import type { NoteInput } from '../types/domain.js'; + +const sourceMeta = (input: NoteInput): Record => { + expect(input.sourceMeta).toBeDefined(); + return input.sourceMeta as Record; +}; + +const slackPayload = ( + event: Record = {}, + envelope: Record = {} +): Record => ({ + type: 'event_callback', + event_id: 'Ev08K1QR2X', + team_id: 'T0123', + event: { + type: 'reaction_added', + user: 'U_REACTOR', + reaction: 'pushpin', + item_user: 'U_AUTHOR', + item: { type: 'message', channel: 'C0123', ts: '1700000000.000100' }, + event_ts: '1700000001.000200', + ...event, + }, + ...envelope, +}); + +const item = (overrides: Record = {}): Record => ({ + type: 'message', + channel: 'C0123', + ts: '1700000000.000100', + ...overrides, +}); + +const savedReaction = process.env.SLACK_CAPTURE_REACTION; + +const setCaptureReaction = (value: string | undefined): void => { + if (value === undefined) { + delete process.env.SLACK_CAPTURE_REACTION; + return; + } + process.env.SLACK_CAPTURE_REACTION = value; +}; + +afterEach(() => { + setCaptureReaction(savedReaction); +}); + +describe('normalizeSlack', () => { + it('should map a captured reaction to a single placeholder note', () => { + const result = normalizeSlack(slackPayload()); + + expect(result.externalId).toBe('Ev08K1QR2X'); + expect(result.notes).toHaveLength(1); + expect(result.notes[0].id).toBe('slack_Ev08K1QR2X'); + expect(result.notes[0].text).toBe(SLACK_PENDING_TEXT); + expect(result.notes[0].author).toBe('U_AUTHOR'); + }); + + it('should export the placeholder text used for the pending message body', () => { + expect(SLACK_PENDING_TEXT).toBe('(pending Slack message text)'); + }); + + it('should attach provenance plus the identifiers the enrichment hook needs', () => { + const meta = sourceMeta(normalizeSlack(slackPayload()).notes[0]); + + expect(meta).toMatchObject({ + provider: 'slack', + externalId: 'Ev08K1QR2X', + needsMessageText: true, + channelId: 'C0123', + messageTs: '1700000000.000100', + teamId: 'T0123', + reaction: 'pushpin', + reactedBy: 'U_REACTOR', + authorUserId: 'U_AUTHOR', + }); + expect(typeof meta.receivedAt).toBe('string'); + }); + + it('should mark needsMessageText as exactly true', () => { + const meta = sourceMeta(normalizeSlack(slackPayload()).notes[0]); + + expect(meta.needsMessageText).toBe(true); + }); + + it('should record an ISO receivedAt timestamp in provenance', () => { + const receivedAt = String(sourceMeta(normalizeSlack(slackPayload()).notes[0]).receivedAt); + + expect(new Date(receivedAt).toISOString()).toBe(receivedAt); + }); + + it('should fall back to the reacting user when item_user is absent', () => { + expect(normalizeSlack(slackPayload({ item_user: undefined })).notes[0].author).toBe( + 'U_REACTOR' + ); + expect(normalizeSlack(slackPayload({ item_user: '' })).notes[0].author).toBe('U_REACTOR'); + }); + + it('should fall back to an unknown author when neither user is present', () => { + const result = normalizeSlack(slackPayload({ item_user: undefined, user: undefined })); + + expect(result.notes[0].author).toBe('unknown'); + expect(sourceMeta(result.notes[0]).reactedBy).toBeUndefined(); + expect(sourceMeta(result.notes[0]).authorUserId).toBeUndefined(); + }); + + it('should omit teamId when the envelope carries none', () => { + const payload = slackPayload({}, { team_id: undefined }); + + expect(sourceMeta(normalizeSlack(payload).notes[0]).teamId).toBeUndefined(); + }); + + it('should return zero notes for a reaction other than the configured one', () => { + expect(normalizeSlack(slackPayload({ reaction: 'eyes' }))).toEqual({ + externalId: 'Ev08K1QR2X', + notes: [], + }); + }); + + it('should honour a custom capture reaction from the environment', () => { + setCaptureReaction('thumbsup'); + + expect(normalizeSlack(slackPayload({ reaction: 'thumbsup' })).notes).toHaveLength(1); + expect(normalizeSlack(slackPayload({ reaction: 'pushpin' })).notes).toHaveLength(0); + }); + + it('should read the capture reaction at call time rather than at module load', () => { + setCaptureReaction('eyes'); + expect(normalizeSlack(slackPayload({ reaction: 'eyes' })).notes).toHaveLength(1); + + setCaptureReaction('rocket'); + expect(normalizeSlack(slackPayload({ reaction: 'eyes' })).notes).toHaveLength(0); + expect(normalizeSlack(slackPayload({ reaction: 'rocket' })).notes).toHaveLength(1); + }); + + it('should match a reaction carrying a skin-tone modifier', () => { + setCaptureReaction('thumbsup'); + const result = normalizeSlack(slackPayload({ reaction: 'thumbsup::skin-tone-3' })); + + expect(result.notes).toHaveLength(1); + expect(sourceMeta(result.notes[0]).reaction).toBe('thumbsup::skin-tone-3'); + }); + + it('should fall back to pushpin when no capture reaction is configured', () => { + setCaptureReaction(undefined); + + expect(normalizeSlack(slackPayload()).notes).toHaveLength(1); + expect(normalizeSlack(slackPayload({ reaction: 'pushpin::skin-tone-5' })).notes).toHaveLength( + 1 + ); + }); + + it('should return zero notes when the reaction is missing or not a string', () => { + expect(normalizeSlack(slackPayload({ reaction: undefined })).notes).toHaveLength(0); + expect(normalizeSlack(slackPayload({ reaction: 7 })).notes).toHaveLength(0); + expect(normalizeSlack(slackPayload({ reaction: '' })).notes).toHaveLength(0); + }); + + it('should return zero notes for an event type we do not capture', () => { + expect(normalizeSlack(slackPayload({ type: 'message' }))).toEqual({ + externalId: 'Ev08K1QR2X', + notes: [], + }); + expect(normalizeSlack(slackPayload({ type: 'reaction_removed' })).notes).toHaveLength(0); + }); + + it('should return zero notes when the event is missing or not an object', () => { + expect(normalizeSlack({ type: 'event_callback', event_id: 'Ev1' })).toEqual({ + externalId: 'Ev1', + notes: [], + }); + expect(normalizeSlack({ type: 'event_callback', event_id: 'Ev1', event: 'nope' })).toEqual({ + externalId: 'Ev1', + notes: [], + }); + }); + + it('should return zero notes when the reacted item is not a message', () => { + expect(normalizeSlack(slackPayload({ item: item({ type: 'file' }) }))).toEqual({ + externalId: 'Ev08K1QR2X', + notes: [], + }); + }); + + it('should return zero notes when the item channel is missing', () => { + expect(normalizeSlack(slackPayload({ item: item({ channel: undefined }) })).notes).toHaveLength( + 0 + ); + expect(normalizeSlack(slackPayload({ item: item({ channel: '' }) })).notes).toHaveLength(0); + }); + + it('should return zero notes when the item timestamp is missing', () => { + expect(normalizeSlack(slackPayload({ item: item({ ts: undefined }) })).notes).toHaveLength(0); + expect(normalizeSlack(slackPayload({ item: item({ ts: 1700000000 }) })).notes).toHaveLength(0); + }); + + it('should return zero notes when the item itself is missing', () => { + expect(normalizeSlack(slackPayload({ item: undefined })).notes).toHaveLength(0); + }); + + it('should throw when event_id is missing or empty', () => { + expect(() => normalizeSlack(slackPayload({}, { event_id: undefined }))).toThrow( + NormalizationError + ); + expect(() => normalizeSlack(slackPayload({}, { event_id: '' }))).toThrow(/missing event_id/); + expect(() => normalizeSlack(slackPayload({}, { event_id: 42 }))).toThrow(NormalizationError); + }); + + it('should throw for an envelope type other than event_callback', () => { + expect(() => normalizeSlack(slackPayload({}, { type: 'url_verification' }))).toThrow( + NormalizationError + ); + expect(() => normalizeSlack(slackPayload({}, { type: undefined }))).toThrow( + /not an event_callback/ + ); + }); + + it('should throw for a non-object payload', () => { + expect(() => normalizeSlack(null)).toThrow(/not an object/); + expect(() => normalizeSlack(undefined)).toThrow(NormalizationError); + expect(() => normalizeSlack('event_callback')).toThrow(NormalizationError); + expect(() => normalizeSlack([slackPayload()])).toThrow(/not an object/); + }); + + it('should truncate a generated note id to 64 characters', () => { + const longId = 'E'.repeat(400); + const result = normalizeSlack(slackPayload({}, { event_id: longId })); + + expect(result.externalId).toBe(longId); + expect(result.notes[0].id).toHaveLength(64); + expect(result.notes[0].id).toBe(`slack_${longId}`.slice(0, 64)); + expect(sourceMeta(result.notes[0]).externalId).toBe(longId); + }); + + it('should produce notes accepted by the note input guard', () => { + expect(isNoteInputArray(normalizeSlack(slackPayload()).notes)).toBe(true); + }); +}); diff --git a/backend/tests/slack.routes.test.ts b/backend/tests/slack.routes.test.ts new file mode 100644 index 0000000..fbecd15 --- /dev/null +++ b/backend/tests/slack.routes.test.ts @@ -0,0 +1,402 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import request from 'supertest'; +import { createHmac } from 'node:crypto'; +import express, { type NextFunction, type Request, type Response } from 'express'; +import { configureQueue, drain } from '../lib/queue.js'; + +vi.mock('../db/ingest_events.dao.js', () => ({ + recordDelivery: vi.fn(), + markProcessing: vi.fn(), + markDone: vi.fn(), + markFailed: vi.fn(), + resetStaleProcessing: vi.fn(), +})); + +vi.mock('../db/notes.dao.js', () => ({ + createNotes: vi.fn(), + getAllNotes: vi.fn(), + streamAllNotes: vi.fn(), +})); + +import { + recordDelivery, + markDone, + markFailed, + markProcessing, +} from '../db/ingest_events.dao.js'; +import { createNotes } from '../db/notes.dao.js'; +import slackRouter from '../routes/slack.routes.js'; + +const mockRecordDelivery = vi.mocked(recordDelivery); +const mockCreateNotes = vi.mocked(createNotes); +const mockMarkDone = vi.mocked(markDone); +const mockMarkFailed = vi.mocked(markFailed); +const mockMarkProcessing = vi.mocked(markProcessing); + +const SECRET = 'slack-test-secret'; + +// Mirrors app.ts: the raw parser must claim the body so the bytes Slack signed +// survive to verifySignature. +const app = express(); +app.use('/v1/slack', express.raw({ type: '*/*', limit: '2mb' }), slackRouter); +app.use((err: Error, _req: Request, res: Response, _next: NextFunction) => { + console.error(`Unhandled error: ${err.message}`); + if (res.headersSent) return; + res.status(500).json({ error: 'Internal server error' }); +}); + +const FIELDS: Record = { + command: '/sticky', + text: 'Deploys are scary', + user_id: 'U123', + user_name: 'jane', + team_id: 'T999', + channel_id: 'C555', + channel_name: 'retro', + trigger_id: 'trg_1', +}; + +const encode = (fields: Record): string => + new URLSearchParams(fields).toString(); + +const sign = (body: string, timestamp: string, secret = SECRET): string => + `v0=${createHmac('sha256', secret).update(`v0:${timestamp}:${body}`).digest('hex')}`; + +const now = (): string => Math.floor(Date.now() / 1000).toString(); + +const postRaw = (body: string, signature: string, timestamp: string) => + request(app) + .post('/v1/slack/commands') + .set('Content-Type', 'application/x-www-form-urlencoded') + .set('x-slack-request-timestamp', timestamp) + .set('x-slack-signature', signature) + .send(body); + +/** Signs whatever it sends, so the happy path is the default. */ +const postCommand = (overrides: Record = {}) => { + const fields: Record = { ...FIELDS }; + for (const [key, value] of Object.entries(overrides)) { + if (value === undefined) { + delete fields[key]; + } else { + fields[key] = value; + } + } + const body = encode(fields); + const timestamp = now(); + return postRaw(body, sign(body, timestamp), timestamp); +}; + +describe('POST /v1/slack/commands', () => { + const originalSecret = process.env.SLACK_SIGNING_SECRET; + + beforeEach(() => { + vi.clearAllMocks(); + process.env.SLACK_SIGNING_SECRET = SECRET; + configureQueue({ baseDelayMs: 0, maxAttempts: 2 }); + mockRecordDelivery.mockResolvedValue(42); + mockCreateNotes.mockResolvedValue([]); + }); + + afterEach(async () => { + await drain(); + if (originalSecret === undefined) { + delete process.env.SLACK_SIGNING_SECRET; + } else { + process.env.SLACK_SIGNING_SECRET = originalSecret; + } + }); + + it('should confirm a valid command with ephemeral text', async () => { + const res = await postCommand(); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ + response_type: 'ephemeral', + text: 'Added to Kongruity: "Deploys are scary"', + }); + }); + + it('should create the note behind the ack', async () => { + await postCommand(); + await drain(); + + expect(mockCreateNotes).toHaveBeenCalledOnce(); + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes).toHaveLength(1); + expect(notes[0].id).toBe('slack_trg_1'); + expect(notes[0].text).toBe('Deploys are scary'); + expect(notes[0].author).toBe('jane'); + }); + + it('should record provenance from the slash command fields', async () => { + await postCommand(); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].sourceMeta).toMatchObject({ + provider: 'slack', + externalId: 'trg_1', + channelId: 'C555', + channelName: 'retro', + authorHandle: 'jane', + authorUserId: 'U123', + teamId: 'T999', + via: 'slash-command', + }); + expect(typeof notes[0].sourceMeta?.receivedAt).toBe('string'); + }); + + it('should omit provenance keys whose source field is absent', async () => { + await postCommand({ channel_name: undefined, team_id: undefined }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + const meta = notes[0].sourceMeta ?? {}; + expect('channelName' in meta).toBe(false); + expect('teamId' in meta).toBe(false); + }); + + it('should respond before the insert finishes', async () => { + let finishInsert: () => void = () => {}; + mockCreateNotes.mockImplementation( + () => new Promise((resolve) => { + finishInsert = () => resolve([]); + }) + ); + + const res = await postCommand(); + + expect(res.status).toBe(200); + expect(mockMarkDone).not.toHaveBeenCalled(); + + finishInsert(); + await drain(); + + expect(mockMarkDone).toHaveBeenCalledWith(42); + }); + + it('should mark the delivery done once the insert succeeds', async () => { + await postCommand(); + await drain(); + + expect(mockMarkProcessing).toHaveBeenCalledWith(42); + expect(mockMarkDone).toHaveBeenCalledWith(42); + expect(mockMarkFailed).not.toHaveBeenCalled(); + }); + + it('should mark the delivery failed when the insert keeps failing', async () => { + mockCreateNotes.mockRejectedValue(new Error('connection refused')); + + await postCommand(); + await drain(); + + expect(mockMarkFailed).toHaveBeenCalledWith(42, 'connection refused'); + expect(mockMarkDone).not.toHaveBeenCalled(); + }); + + it('should apply cleanSlackText to the stored note', async () => { + await postCommand({ + text: 'Ping <@U9|dan> in <#C1|ops> about & ship', + }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].text).toBe('Ping @dan in #ops about the doc & ship'); + }); + + it('should echo the cleaned text rather than the raw mrkdwn', async () => { + const res = await postCommand({ text: 'read ' }); + + expect(res.body.text).toBe('Added to Kongruity: "read the doc"'); + }); + + it('should truncate a long echo so it does not flood the channel', async () => { + const long = 'a'.repeat(400); + + const res = await postCommand({ text: long }); + + expect(res.status).toBe(200); + expect(res.body.text).toContain('…'); + expect(res.body.text.length).toBeLessThan(160); + }); + + it('should store the whole note even when the echo is truncated', async () => { + const long = 'a'.repeat(400); + + await postCommand({ text: long }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].text).toBe(long); + }); + + it('should fall back to a synthetic external id when trigger_id is absent', async () => { + await postCommand({ trigger_id: undefined }); + + expect(mockRecordDelivery).toHaveBeenCalledOnce(); + const [input] = mockRecordDelivery.mock.calls[0]; + expect(input.provider).toBe('slack'); + expect(input.externalId).toMatch(/^T999:U123:\d+$/); + }); + + it('should truncate the note id to the 64 character column width', async () => { + await postCommand({ trigger_id: 'trg_'.padEnd(120, 'x') }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].id).toHaveLength(64); + expect(notes[0].id.startsWith('slack_trg_x')).toBe(true); + }); + + it('should drop a duplicate trigger_id without enqueueing work', async () => { + mockRecordDelivery.mockResolvedValue(null); + + const res = await postCommand(); + await drain(); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ response_type: 'ephemeral', text: 'Already captured.' }); + expect(mockCreateNotes).not.toHaveBeenCalled(); + expect(mockMarkProcessing).not.toHaveBeenCalled(); + }); + + it('should return usage when the text is empty', async () => { + const res = await postCommand({ text: '' }); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ + response_type: 'ephemeral', + text: 'Usage: /sticky ', + }); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should return usage when the text is only whitespace', async () => { + const res = await postCommand({ text: ' ' }); + + expect(res.status).toBe(200); + expect(res.body.text).toBe('Usage: /sticky '); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + /** + * Markup can be non-empty before cleaning and empty after it. Recording the + * delivery first would consume the trigger_id that dedups a Slack retry. + */ + it('should return usage without recording a delivery when the text cleans away to nothing', async () => { + const res = await postCommand({ text: '<>' }); + + expect(res.status).toBe(200); + expect(res.body.text).toBe('Usage: /sticky '); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + expect(mockCreateNotes).not.toHaveBeenCalled(); + }); + + it('should reject an unrecognized command name', async () => { + const res = await postCommand({ command: '/todo' }); + + expect(res.status).toBe(200); + expect(res.body).toEqual({ + response_type: 'ephemeral', + text: 'Unknown command /todo.', + }); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should fall back to the user id when no user_name is sent', async () => { + await postCommand({ user_name: undefined }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].author).toBe('U123'); + }); + + it('should fall back to "unknown" when the sender is unidentified', async () => { + await postCommand({ user_name: undefined, user_id: undefined }); + await drain(); + + const [notes] = mockCreateNotes.mock.calls[0]; + expect(notes[0].author).toBe('unknown'); + }); + + it('should reject a signature computed with the wrong secret', async () => { + const body = encode(FIELDS); + const timestamp = now(); + + const res = await postRaw(body, sign(body, timestamp, 'wrong-secret'), timestamp); + + expect(res.status).toBe(401); + expect(res.body).toEqual({ error: 'Invalid signature' }); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should reject a body altered after signing', async () => { + const body = encode(FIELDS); + const timestamp = now(); + const signature = sign(body, timestamp); + const tampered = body.replace('scary', 'great'); + + const res = await postRaw(tampered, signature, timestamp); + + expect(res.status).toBe(401); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should reject a request with no signature header', async () => { + const body = encode(FIELDS); + + const res = await request(app) + .post('/v1/slack/commands') + .set('Content-Type', 'application/x-www-form-urlencoded') + .set('x-slack-request-timestamp', now()) + .send(body); + + expect(res.status).toBe(401); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should reject a replayed request whose timestamp is outside the tolerance', async () => { + const body = encode(FIELDS); + const stale = (Math.floor(Date.now() / 1000) - 3600).toString(); + + const res = await postRaw(body, sign(body, stale), stale); + + expect(res.status).toBe(401); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should return 500 when no signing secret is configured', async () => { + delete process.env.SLACK_SIGNING_SECRET; + + const res = await postCommand(); + + expect(res.status).toBe(500); + expect(res.body).toEqual({ error: 'Slack is not configured' }); + expect(mockRecordDelivery).not.toHaveBeenCalled(); + }); + + it('should return 500 when recording the delivery fails', async () => { + mockRecordDelivery.mockRejectedValue(new Error('connection refused')); + + const res = await postCommand(); + + expect(res.status).toBe(500); + expect(mockCreateNotes).not.toHaveBeenCalled(); + }); + + it('should answer every user-facing outcome with ephemeral JSON', async () => { + const responses = await Promise.all([ + postCommand(), + postCommand({ text: '' }), + postCommand({ command: '/todo' }), + ]); + + for (const res of responses) { + expect(res.status).toBe(200); + expect(res.headers['content-type']).toMatch(/application\/json/); + expect(res.body.response_type).toBe('ephemeral'); + expect(typeof res.body.text).toBe('string'); + } + }); +}); diff --git a/backend/tests/slack.service.test.ts b/backend/tests/slack.service.test.ts new file mode 100644 index 0000000..9e51550 --- /dev/null +++ b/backend/tests/slack.service.test.ts @@ -0,0 +1,650 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; + +const { mockRequestJson } = vi.hoisted(() => ({ mockRequestJson: vi.fn() })); + +// The real HttpRequestError is kept so its propagation through callSlack can be +// asserted against the actual class rather than a stand-in. +vi.mock('../lib/httpClient.js', async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, requestJson: mockRequestJson }; +}); + +import { HttpRequestError } from '../lib/httpClient.js'; +import { + callSlack, + authTest, + getUserDisplayName, + getMessageText, + enrichSlackDelivery, + isSlackApiError, + SlackApiError, + resetSlackCaches, +} from '../services/slack.service.js'; +import type { NormalizedDelivery } from '../types/integration.js'; + +const API_BASE = 'https://slack.com/api/'; + +/** Answers each Slack method from a fixture; an unrouted method is a test bug. */ +const route = (routes: Record): void => { + mockRequestJson.mockImplementation((url: unknown) => { + const method = String(url).slice(API_BASE.length); + if (!(method in routes)) { + return Promise.reject(new Error(`unexpected Slack method "${method}"`)); + } + return Promise.resolve(routes[method]); + }); +}; + +const formOf = (callIndex: number): URLSearchParams => { + const [, init] = mockRequestJson.mock.calls[callIndex] as [string, RequestInit]; + return new URLSearchParams(String(init.body)); +}; + +const caught = async (promise: Promise): Promise => + promise.then(() => null).catch((err: unknown) => err); + +const historyOf = (message: Record | null): Record => ({ + ok: true, + messages: message === null ? [] : [message], +}); + +const delivery = (notes: NormalizedDelivery['notes']): NormalizedDelivery => ({ + externalId: 'evt_1', + notes, +}); + +describe('slack.service', () => { + let originalToken: string | undefined; + + beforeEach(() => { + vi.clearAllMocks(); + resetSlackCaches(); + originalToken = process.env.SLACK_BOT_TOKEN; + process.env.SLACK_BOT_TOKEN = 'xoxb-test-token'; + }); + + afterEach(() => { + if (originalToken === undefined) delete process.env.SLACK_BOT_TOKEN; + else process.env.SLACK_BOT_TOKEN = originalToken; + }); + + describe('callSlack', () => { + it('should post form-encoded params to the named method with a bearer token', async () => { + route({ 'users.info': { ok: true, user: { name: 'ada' } } }); + + await callSlack('users.info', { user: 'U1', include_locale: 'false' }); + + const [url, init] = mockRequestJson.mock.calls[0] as [string, RequestInit]; + expect(url).toBe('https://slack.com/api/users.info'); + expect(init.method).toBe('POST'); + expect(init.headers).toMatchObject({ + authorization: 'Bearer xoxb-test-token', + 'content-type': 'application/x-www-form-urlencoded; charset=utf-8', + }); + expect(formOf(0).get('user')).toBe('U1'); + expect(formOf(0).get('include_locale')).toBe('false'); + }); + + it('should send an empty body when no params are given', async () => { + route({ 'auth.test': { ok: true } }); + + await callSlack('auth.test'); + + const [, init] = mockRequestJson.mock.calls[0] as [string, RequestInit]; + expect(init.body).toBe(''); + }); + + it('should resolve the parsed envelope when ok is true', async () => { + route({ 'auth.test': { ok: true, team: 'Acme' } }); + + await expect(callSlack('auth.test')).resolves.toEqual({ ok: true, team: 'Acme' }); + }); + + it('should throw a SlackApiError for an HTTP 200 carrying ok false', async () => { + route({ 'users.info': { ok: false, error: 'invalid_auth' } }); + + const error = await caught(callSlack('users.info', { user: 'U1' })); + + expect(isSlackApiError(error)).toBe(true); + expect(error).toBeInstanceOf(SlackApiError); + expect(isSlackApiError(error) && error.slackError).toBe('invalid_auth'); + expect(isSlackApiError(error) && error.method).toBe('users.info'); + expect(error instanceof Error ? error.message : '').toContain('invalid_auth'); + }); + + it('should classify every known non-retryable error string as permanent', async () => { + const permanent = [ + 'invalid_auth', + 'not_authed', + 'account_inactive', + 'token_revoked', + 'token_expired', + 'no_permission', + 'missing_scope', + 'channel_not_found', + 'not_in_channel', + 'message_not_found', + 'user_not_found', + 'invalid_arguments', + 'invalid_form_data', + 'is_archived', + ] as const; + + for (const slackError of permanent) { + route({ 'conversations.history': { ok: false, error: slackError } }); + + const error = await caught(callSlack('conversations.history')); + + expect(isSlackApiError(error) && error.kind, slackError).toBe('permanent'); + } + }); + + it('should classify a known transient error string as retryable', async () => { + route({ 'conversations.history': { ok: false, error: 'ratelimited' } }); + + const error = await caught(callSlack('conversations.history')); + + expect(isSlackApiError(error) && error.kind).toBe('retryable'); + }); + + it('should classify an unrecognized error string as retryable', async () => { + route({ 'conversations.history': { ok: false, error: 'some_future_slack_error' } }); + + const error = await caught(callSlack('conversations.history')); + + expect(isSlackApiError(error) && error.slackError).toBe('some_future_slack_error'); + expect(isSlackApiError(error) && error.kind).toBe('retryable'); + }); + + it('should report unknown_error when the failure body names no error', async () => { + route({ 'auth.test': { ok: false } }); + + const error = await caught(callSlack('auth.test')); + + expect(isSlackApiError(error) && error.slackError).toBe('unknown_error'); + expect(isSlackApiError(error) && error.kind).toBe('retryable'); + }); + + it('should report unknown_error when the body is not an object', async () => { + route({ 'auth.test': 'maintenance' }); + + const error = await caught(callSlack('auth.test')); + + expect(isSlackApiError(error) && error.slackError).toBe('unknown_error'); + }); + + it('should throw a permanent no_token error without making a request', async () => { + delete process.env.SLACK_BOT_TOKEN; + + const error = await caught(callSlack('auth.test')); + + expect(isSlackApiError(error) && error.slackError).toBe('no_token'); + expect(isSlackApiError(error) && error.kind).toBe('permanent'); + expect(mockRequestJson).not.toHaveBeenCalled(); + }); + + it('should treat an empty or whitespace token as absent', async () => { + process.env.SLACK_BOT_TOKEN = ''; + expect(isSlackApiError(await caught(callSlack('auth.test')))).toBe(true); + + process.env.SLACK_BOT_TOKEN = ' '; + expect(isSlackApiError(await caught(callSlack('auth.test')))).toBe(true); + + expect(mockRequestJson).not.toHaveBeenCalled(); + }); + + it('should read the token at call time rather than at module load', async () => { + process.env.SLACK_BOT_TOKEN = 'xoxb-rotated'; + route({ 'auth.test': { ok: true } }); + + await callSlack('auth.test'); + + const [, init] = mockRequestJson.mock.calls[0] as [string, RequestInit]; + expect(init.headers).toMatchObject({ authorization: 'Bearer xoxb-rotated' }); + }); + + it('should let an HttpRequestError propagate unwrapped', async () => { + const transport = new HttpRequestError(`${API_BASE}auth.test`, 'timeout', 3); + mockRequestJson.mockRejectedValue(transport); + + const error = await caught(callSlack('auth.test')); + + expect(error).toBe(transport); + expect(isSlackApiError(error)).toBe(false); + }); + + it('should distinguish a Slack failure from an unrelated error', () => { + expect(isSlackApiError(new TypeError('bad call'))).toBe(false); + expect(isSlackApiError('invalid_auth')).toBe(false); + expect(isSlackApiError(null)).toBe(false); + }); + }); + + describe('authTest', () => { + it('should map the identity fields of a successful auth.test', async () => { + route({ + 'auth.test': { ok: true, team_id: 'T1', team: 'Acme', user_id: 'U_BOT', url: 'x' }, + }); + + await expect(authTest()).resolves.toEqual({ + teamId: 'T1', + teamName: 'Acme', + botUserId: 'U_BOT', + }); + }); + + it('should reject with a SlackApiError when the credential is rejected', async () => { + route({ 'auth.test': { ok: false, error: 'token_revoked' } }); + + const error = await caught(authTest()); + + expect(isSlackApiError(error) && error.kind).toBe('permanent'); + }); + }); + + describe('getUserDisplayName', () => { + it('should request users.info for the given user', async () => { + route({ 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } } }); + + await getUserDisplayName('U1'); + + const [url] = mockRequestJson.mock.calls[0] as [string, RequestInit]; + expect(url).toBe('https://slack.com/api/users.info'); + expect(formOf(0).get('user')).toBe('U1'); + }); + + it('should prefer profile.display_name above every other name', async () => { + route({ + 'users.info': { + ok: true, + user: { + name: 'ada.l', + real_name: 'Ada L', + profile: { display_name: 'ada', real_name: 'Ada Lovelace' }, + }, + }, + }); + + await expect(getUserDisplayName('U1')).resolves.toBe('ada'); + }); + + it('should fall back to profile.real_name when display_name is empty', async () => { + route({ + 'users.info': { + ok: true, + user: { + name: 'ada.l', + real_name: 'Ada L', + profile: { display_name: '', real_name: 'Ada Lovelace' }, + }, + }, + }); + + await expect(getUserDisplayName('U1')).resolves.toBe('Ada Lovelace'); + }); + + it('should fall back to the top-level real_name when the profile has neither', async () => { + route({ + 'users.info': { ok: true, user: { name: 'ada.l', real_name: 'Ada L', profile: {} } }, + }); + + await expect(getUserDisplayName('U1')).resolves.toBe('Ada L'); + }); + + it('should fall back to the account name when no real name is set', async () => { + route({ 'users.info': { ok: true, user: { name: 'ada.l' } } }); + + await expect(getUserDisplayName('U1')).resolves.toBe('ada.l'); + }); + + it('should fall back to the raw user id when the profile carries no name', async () => { + route({ 'users.info': { ok: true, user: { profile: {} } } }); + + await expect(getUserDisplayName('U1')).resolves.toBe('U1'); + }); + + it('should memoize a resolved name and not request it twice', async () => { + route({ 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } } }); + + expect(await getUserDisplayName('U1')).toBe('ada'); + expect(await getUserDisplayName('U1')).toBe('ada'); + + expect(mockRequestJson).toHaveBeenCalledTimes(1); + }); + + it('should memoize each user separately', async () => { + mockRequestJson.mockImplementation((_url: unknown, init?: RequestInit) => { + const user = new URLSearchParams(String(init?.body)).get('user'); + return Promise.resolve({ ok: true, user: { profile: { display_name: `name-${user}` } } }); + }); + + expect(await getUserDisplayName('U1')).toBe('name-U1'); + expect(await getUserDisplayName('U2')).toBe('name-U2'); + expect(mockRequestJson).toHaveBeenCalledTimes(2); + }); + + it('should clear memoized names on resetSlackCaches', async () => { + route({ 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } } }); + + await getUserDisplayName('U1'); + resetSlackCaches(); + await getUserDisplayName('U1'); + + expect(mockRequestJson).toHaveBeenCalledTimes(2); + }); + + it('should degrade to the raw user id on a permanent error', async () => { + route({ 'users.info': { ok: false, error: 'user_not_found' } }); + + await expect(getUserDisplayName('U_GONE')).resolves.toBe('U_GONE'); + }); + + it('should degrade to the raw user id when the token is missing', async () => { + delete process.env.SLACK_BOT_TOKEN; + + await expect(getUserDisplayName('U1')).resolves.toBe('U1'); + expect(mockRequestJson).not.toHaveBeenCalled(); + }); + + it('should rethrow a retryable error so the queue can retry', async () => { + route({ 'users.info': { ok: false, error: 'ratelimited' } }); + + const error = await caught(getUserDisplayName('U1')); + + expect(isSlackApiError(error) && error.kind).toBe('retryable'); + }); + + it('should rethrow a transport error unwrapped', async () => { + const transport = new HttpRequestError(`${API_BASE}users.info`, 'network', 3); + mockRequestJson.mockRejectedValue(transport); + + expect(await caught(getUserDisplayName('U1'))).toBe(transport); + }); + }); + + describe('getMessageText', () => { + it('should read a single message from conversations.history by timestamp', async () => { + route({ 'conversations.history': historyOf({ text: 'ship it', user: 'U1' }) }); + + const result = await getMessageText('C1', '1700000000.000100'); + + expect(result).toEqual({ text: 'ship it', userId: 'U1' }); + const [url] = mockRequestJson.mock.calls[0] as [string, RequestInit]; + expect(url).toBe('https://slack.com/api/conversations.history'); + + const form = formOf(0); + expect(form.get('channel')).toBe('C1'); + expect(form.get('latest')).toBe('1700000000.000100'); + expect(form.get('oldest')).toBe('1700000000.000100'); + expect(form.get('inclusive')).toBe('true'); + expect(form.get('limit')).toBe('1'); + }); + + it('should clean Slack mrkdwn out of the returned text', async () => { + route({ + 'conversations.history': historyOf({ + text: 'ask <@U9|ada> about & ship ', + }), + }); + + await expect(getMessageText('C1', '1.1')).resolves.toEqual({ + text: 'ask @ada about the doc & ship', + }); + }); + + it('should omit userId when the message carries no user', async () => { + route({ 'conversations.history': historyOf({ text: 'from a bot' }) }); + + const result = await getMessageText('C1', '1.1'); + + expect(result).toEqual({ text: 'from a bot' }); + expect(result === null ? true : 'userId' in result).toBe(false); + }); + + it('should fall back to conversations.replies for a threaded reply', async () => { + route({ + 'conversations.history': historyOf(null), + 'conversations.replies': { + ok: true, + messages: [{ ts: '1.1', text: 'thread parent' }, { ts: '2.2', text: 'the reply' }], + }, + }); + + const result = await getMessageText('C1', '2.2'); + + expect(result).toEqual({ text: 'the reply' }); + const [url] = mockRequestJson.mock.calls[1] as [string, RequestInit]; + expect(url).toBe('https://slack.com/api/conversations.replies'); + + const form = formOf(1); + expect(form.get('channel')).toBe('C1'); + expect(form.get('ts')).toBe('2.2'); + expect(form.get('limit')).toBe('1'); + expect(form.get('inclusive')).toBe('true'); + }); + + it('should not call conversations.replies when history already answered', async () => { + route({ 'conversations.history': historyOf({ text: 'ship it' }) }); + + await getMessageText('C1', '1.1'); + + expect(mockRequestJson).toHaveBeenCalledTimes(1); + }); + + it('should return null when neither history nor replies yields a message', async () => { + route({ + 'conversations.history': historyOf(null), + 'conversations.replies': { ok: true, messages: [] }, + }); + + await expect(getMessageText('C1', '1.1')).resolves.toBeNull(); + expect(mockRequestJson).toHaveBeenCalledTimes(2); + }); + + it('should return null when the messages field is missing entirely', async () => { + route({ + 'conversations.history': { ok: true }, + 'conversations.replies': { ok: true }, + }); + + await expect(getMessageText('C1', '1.1')).resolves.toBeNull(); + }); + + it('should return null for a file-only message whose text is empty', async () => { + route({ 'conversations.history': historyOf({ text: '', user: 'U1' }) }); + + await expect(getMessageText('C1', '1.1')).resolves.toBeNull(); + }); + + it('should return null when the text is only markup that cleans away', async () => { + route({ 'conversations.history': historyOf({ text: ' ', user: 'U1' }) }); + + await expect(getMessageText('C1', '1.1')).resolves.toBeNull(); + }); + + it('should reject with a SlackApiError when the channel is unreadable', async () => { + route({ 'conversations.history': { ok: false, error: 'not_in_channel' } }); + + const error = await caught(getMessageText('C1', '1.1')); + + expect(isSlackApiError(error) && error.kind).toBe('permanent'); + }); + }); + + describe('enrichSlackDelivery', () => { + const taggedNote = (overrides: Record = {}) => ({ + id: 'slack_C1_1.1', + text: '(pending message text)', + author: 'unknown', + sourceMeta: { + provider: 'slack', + externalId: 'evt_1', + receivedAt: '2026-01-01T00:00:00.000Z', + needsMessageText: true, + channelId: 'C1', + messageTs: '1.1', + ...overrides, + }, + }); + + it('should replace placeholder text and resolve the author', async () => { + route({ + 'conversations.history': historyOf({ text: 'ship the thing', user: 'U_AUTHOR' }), + 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } }, + }); + + const result = await enrichSlackDelivery(delivery([taggedNote()]), {}); + + expect(result.notes).toHaveLength(1); + expect(result.notes[0]?.text).toBe('ship the thing'); + expect(result.notes[0]?.author).toBe('ada'); + expect(result.notes[0]?.sourceMeta?.authorHandle).toBe('ada'); + }); + + it('should strip the needsMessageText marker from the resulting sourceMeta', async () => { + route({ + 'conversations.history': historyOf({ text: 'ship it', user: 'U_AUTHOR' }), + 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } }, + }); + + const result = await enrichSlackDelivery(delivery([taggedNote()]), {}); + + expect(result.notes[0]?.sourceMeta).not.toHaveProperty('needsMessageText'); + expect(result.notes[0]?.sourceMeta).toMatchObject({ + provider: 'slack', + externalId: 'evt_1', + channelId: 'C1', + messageTs: '1.1', + }); + }); + + it('should prefer an explicit authorUserId over the message author', async () => { + mockRequestJson.mockImplementation((url: unknown, init?: RequestInit) => { + if (String(url).endsWith('conversations.history')) { + return Promise.resolve(historyOf({ text: 'ship it', user: 'U_MESSAGE' })); + } + const user = new URLSearchParams(String(init?.body)).get('user'); + return Promise.resolve({ ok: true, user: { profile: { display_name: `name-${user}` } } }); + }); + + const result = await enrichSlackDelivery( + delivery([taggedNote({ authorUserId: 'U_REACTOR' })]), + {} + ); + + expect(result.notes[0]?.author).toBe('name-U_REACTOR'); + }); + + it('should keep the existing author when no user id can be resolved', async () => { + route({ 'conversations.history': historyOf({ text: 'from a bot' }) }); + + const result = await enrichSlackDelivery(delivery([taggedNote()]), {}); + + expect(result.notes[0]?.text).toBe('from a bot'); + expect(result.notes[0]?.author).toBe('unknown'); + expect(mockRequestJson).toHaveBeenCalledTimes(1); + }); + + it('should drop a note whose message cannot be fetched', async () => { + route({ + 'conversations.history': historyOf(null), + 'conversations.replies': { ok: true, messages: [] }, + }); + + const result = await enrichSlackDelivery(delivery([taggedNote()]), {}); + + expect(result.notes).toEqual([]); + expect(result.externalId).toBe('evt_1'); + }); + + it('should drop a tagged note that carries no channel or timestamp', async () => { + route({}); + + const result = await enrichSlackDelivery( + delivery([taggedNote({ channelId: undefined, messageTs: undefined })]), + {} + ); + + expect(result.notes).toEqual([]); + expect(mockRequestJson).not.toHaveBeenCalled(); + }); + + it('should pass an untagged note through untouched', async () => { + route({}); + const plain = { + id: 'slack_evt_2', + text: 'already complete', + author: 'Ada Lovelace', + sourceMeta: { provider: 'slack', externalId: 'evt_2', authorHandle: 'ada' }, + }; + + const result = await enrichSlackDelivery(delivery([plain]), {}); + + expect(result.notes[0]).toBe(plain); + expect(mockRequestJson).not.toHaveBeenCalled(); + }); + + it('should pass a note through when it has no sourceMeta at all', async () => { + route({}); + const plain = { id: 'n1', text: 'manual note', author: 'ada' }; + + const result = await enrichSlackDelivery(delivery([plain]), {}); + + expect(result.notes[0]).toBe(plain); + }); + + it('should enrich tagged notes while leaving untagged ones in place', async () => { + route({ + 'conversations.history': historyOf({ text: 'fetched', user: 'U_AUTHOR' }), + 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } }, + }); + const plain = { id: 'n_plain', text: 'untouched', author: 'someone' }; + + const result = await enrichSlackDelivery(delivery([plain, taggedNote()]), {}); + + expect(result.notes).toHaveLength(2); + expect(result.notes[0]).toBe(plain); + expect(result.notes[1]?.text).toBe('fetched'); + }); + + it('should not mutate the delivery or the notes it was given', async () => { + route({ + 'conversations.history': historyOf({ text: 'ship it', user: 'U_AUTHOR' }), + 'users.info': { ok: true, user: { profile: { display_name: 'ada' } } }, + }); + const input = delivery([taggedNote()]); + const snapshot = structuredClone(input); + + const result = await enrichSlackDelivery(input, {}); + + expect(input).toEqual(snapshot); + expect(result).not.toBe(input); + expect(result.notes).not.toBe(input.notes); + }); + + it('should return an empty delivery unchanged in shape', async () => { + route({}); + + await expect(enrichSlackDelivery(delivery([]), {})).resolves.toEqual({ + externalId: 'evt_1', + notes: [], + }); + }); + + it('should propagate a retryable failure so the queue retries the delivery', async () => { + route({ 'conversations.history': { ok: false, error: 'ratelimited' } }); + + const error = await caught(enrichSlackDelivery(delivery([taggedNote()]), {})); + + expect(isSlackApiError(error) && error.kind).toBe('retryable'); + }); + + it('should accept a payload of any shape without needing it', async () => { + route({ 'conversations.history': historyOf({ text: 'ship it' }) }); + + await expect( + enrichSlackDelivery(delivery([taggedNote()]), undefined) + ).resolves.toMatchObject({ notes: [{ text: 'ship it' }] }); + }); + }); +}); diff --git a/backend/tests/slackText.test.ts b/backend/tests/slackText.test.ts new file mode 100644 index 0000000..d964da3 --- /dev/null +++ b/backend/tests/slackText.test.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from 'vitest'; +import { cleanSlackText } from '../lib/slackText.js'; + +describe('cleanSlackText', () => { + it('should leave plain text untouched', () => { + expect(cleanSlackText('ship the thing')).toBe('ship the thing'); + }); + + it('should keep a bare user mention readable when no label is supplied', () => { + expect(cleanSlackText('ping <@U0123> about it')).toBe('ping @U0123 about it'); + }); + + it('should prefer the label on a user mention', () => { + expect(cleanSlackText('ping <@U0123|kevin> about it')).toBe('ping @kevin about it'); + }); + + it('should render a channel reference by name', () => { + expect(cleanSlackText('see <#C0123|general>')).toBe('see #general'); + }); + + it('should keep a channel reference without a label', () => { + expect(cleanSlackText('see <#C0123>')).toBe('see #C0123'); + }); + + it('should convert broadcast mentions', () => { + expect(cleanSlackText(' heads up')).toBe('@here heads up'); + expect(cleanSlackText(' heads up')).toBe('@channel heads up'); + }); + + it('should replace a labelled link with its label', () => { + expect(cleanSlackText('read ')).toBe('read the docs'); + }); + + it('should keep the url when a link has no label', () => { + expect(cleanSlackText('read ')).toBe('read https://example.com'); + }); + + it('should unwrap a mailto link', () => { + expect(cleanSlackText('mail ')).toBe('mail a@b.com'); + expect(cleanSlackText('mail ')).toBe('mail a@b.com'); + }); + + it('should unescape html entities', () => { + expect(cleanSlackText('tabs & spaces')).toBe('tabs & spaces'); + }); + + /** + * Slack escapes a literal angle bracket so it is not read as markup. If + * entities were unescaped first, this would be parsed as a link and the + * user's text would silently disappear. + */ + it('should not reparse an escaped angle bracket as markup', () => { + expect(cleanSlackText('if a <b> then stop')).toBe('if a then stop'); + }); + + it('should handle several references in one message', () => { + expect( + cleanSlackText('<@U1|amy> moved <#C1|ops> to ') + ).toBe('@amy moved #ops to the wiki'); + }); + + it('should trim surrounding whitespace and trailing spaces on each line', () => { + expect(cleanSlackText(' first line \n second ')).toBe('first line\n second'); + }); + + it('should return an empty string for whitespace-only input', () => { + expect(cleanSlackText(' \n ')).toBe(''); + }); + + it('should leave an empty angle-bracket pair alone rather than throwing', () => { + expect(cleanSlackText('a <> b')).toBe('a b'); + }); +}); -- 2.43.0 From 17746e9268f2b3558ce85f4aa017dbdccfc62528 Mon Sep 17 00:00:00 2001 From: KS Jannette Date: Sat, 22 Aug 2026 00:52:55 -0400 Subject: [PATCH 2/2] buildout Slack integration --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 93b4678..3b4a67b 100644 --- a/README.md +++ b/README.md @@ -140,7 +140,7 @@ DATABASE_URL=postgresql://:@localhost:5432/kongruity EOF ``` -Replace placeholder values with your actual keys and database credentials. +Replace placeholder values with your keys and database credentials. #### Integration variables @@ -179,7 +179,7 @@ EOF `SLACK_SIGNING_SECRET` is already in the integration variables above. There is no OAuth flow here on purpose: a single workspace reading credentials from the environment is far less machinery than an install dance, and the OAuth endpoints in the provider registry stay unused until multi-workspace support is actually needed. -### 2. Expose your local backend +### 2. Expose backend Slack only delivers to a public HTTPS URL, so a tunnel is required for local development. -- 2.43.0