Test Cycles

The Test Cycles vertical is a complete QA management system layered onto the existing Hackorda app. Different vertical from the quiz platform: it lives under /app/test-cycles/* and /app/admin/{organizations,products,test-cycles}/*, and its own DB tables live alongside the quiz schema.

This document is the canonical reference for what shipped, where the code lives, and how the pieces fit. The legacy docs (features.md, api-routes.md, database-schema.md) describe the quiz system only — they do not yet cover this vertical.

Mental model

Organization
 └── Product (has URL — where QA navigates to test)
      └── Product version (e.g. v0.1.0)
           └── Test Cycle (planned → active → review → closed/cancelled)
                ├── Cycle Documents (Notion-style markdown docs)
                ├── Testers (per-cycle members: tester | lead | observer)
                ├── Test Runs (a tester's session)
                │    └── Run pack (attachments scoped to the run)
                └── Issues (filed during runs)
                     ├── Inline markdown body w/ embedded media
                     ├── Comments (markdown w/ embedded media)
                     ├── Attachments (issue evidence pack)
                     └── AI suggestions (title / severity / bug type)

A user has a system role (ROLES.QA = 4 displayed as "Tester" in UI) and zero or more per-cycle roles (testers.role). System role gates global access; per-cycle role gates which cycles they can read/write inside.

Three actors, three flows

Admin

• Creates organizations, products, product versions
• Promotes users to system-level Tester via Profile → Actions
• Creates test cycles, assigns members (tester / lead / observer)
• Authors cycle documents (handbooks, runbooks)
• Triages issues, approves payouts, marks paid (Kaspi / bank / cash)
• Closes the cycle → AI generates a closing-report doc

Cycle lead (testers.role='lead')

• Same write access as admin within their cycle
• Can edit docs, retry AI agents, approve payouts
• Cannot assign other admins or change system roles

Tester (testers.role='tester')

• Reads docs, starts test runs, files issues with screenshots/videos
• Drops session recordings into the run pack
• Sees AI suggestions on their issues but can't apply them on others'
• Receives payouts when admin approves their bugs

Feature surfaces

1. Cycle Documents

Multiple markdown docs per cycle, with kind = doc | brief | runbook | report. Notion-style two-pane layout: left rail of doc titles, right pane renders the selected doc. Admin/lead can create / inline-edit / delete; testers and observers read-only.

The Hackorda Onboarding cycle's two seed docs (Onboarding handbook, Test cases) are sourced from docs/qa/*.md files in this repo. The seed (seed:tc-onboarding) does an upsert by (test_cycle_id, slug) on every container start, so editing the markdown in a PR auto-syncs to production.

Files:

• Schema: src/db/schema.ts → cycleDocuments table + CYCLE_DOCUMENT_KINDS
• Migration: src/db/migrations/0020_spooky_starbolt.sql
• API: src/app/api/test-cycles/[id]/documents/{route.ts,[docId]/route.ts}
• UI: src/components/test-cycles/CycleDocsPanel.tsx
• Hooks: src/hooks/test-cycles/documents.ts
• Seed: src/db/seed-hackorda-qa-cycle.ts (upsertCycleDocs)

2. Inline media in markdown

Any MarkdownEditor with an uploadContext prop accepts drag-drop, clipboard paste, and click-to-upload. Files go to DO Spaces with a public-read ACL, and the markdown renderer routes by URL extension:

• .png .jpg .webp .avif .svg → <img loading="lazy">
• .mp4 .webm .mov .m4v .ogv → <video controls preload="metadata">
• Other → fallback to a download link

Wired into: cycle docs edit, cycle brief side panel, issue create description, issue edit description, issue comments.

Files:

• Renderer: src/lib/markdown.ts (renderMarkdown extended with ![alt](../url))
• Editor: src/components/test-cycles/MarkdownEditor.tsx
• Read view: src/components/test-cycles/MarkdownView.tsx
• Upload endpoint: src/app/api/test-cycles/[id]/attachments/route.ts

3. Run packs

Each test run can have its own attachment bundle — screen recordings, console logs, HAR files. Expand a run card → drop zone + tile gallery (image thumbs, inline video players, file rows). Lazy-loaded: only fetches when the user expands a card, so the cycle page doesn't fan out N requests.

Files:

• Schema: attachments.target_type='test_run' (varchar accepts the new value; no migration needed)
• API: GET /api/test-cycles/:id/runs/:runId returns attachments[]
• UI: src/components/test-cycles/RunPackPanel.tsx
• Hook: src/hooks/test-cycles/runs.ts → useTestRun(cycleId, runId)

4. Doc-level attachments

Files attached to a doc (PDF specs, JSON sample payloads) but not embedded in body. Same DO Spaces pipeline. Image thumbnails / video players / file rows in a 3-column gallery.

Files:

• Schema: attachments.target_type='cycle_document'
• UI: DocAttachmentSection inside src/components/test-cycles/CycleDocsPanel.tsx

5. AI agents

Three fire-and-forget agents, all pinned to Claude Sonnet 4 (claude-sonnet-4-20250514). All write provenance to a single ai_runs table for cost and debug tracking.

Why fire-and-forget, not pg-boss: we run on long-lived Docker containers, so void run...(...) from API handlers stays alive past the response. Every attempt logs to ai_runs so even process restarts mid-call leave breadcrumbs.

Why tool use, not free-form JSON: intake calls submit_intake_analysis with a typed schema, so output validity is structural. Severity / bug_type constrained to actual db/schema.ts enum values.

Vision via URL: DO Spaces serves public CDN URLs, so we pass { type: 'image', source: { type: 'url', url } } directly — no base64 transcoding. Capped at 3 images per intake call.

Files:

• Lib: src/lib/ai/{client.ts,intake.ts,run-summary.ts,cycle-close.ts}
• Schema: src/db/schema.ts → aiRuns, AI_RUN_KINDS, AI_RUN_STATUSES, IssueAiSuggestions
• Migrations: 0021_tense_exiles.sql (ai_runs + issue cols), 0023_funny_luke_cage.sql (test_runs.ai_summary cols)
• UI: src/components/test-cycles/AiSuggestionsCard.tsx
• Retry endpoint: src/app/api/test-cycles/[id]/issues/[issueId]/intake/route.ts

Operational:

• Enable: set ANTHROPIC_API_KEY on the droplet
• Disable: unset → isAiEnabled() short-circuits everywhere
• Monitor: select kind, status, cost_usd_cents, latency_ms from ai_runs order by created_at desc
• See docs/deployment.md for full operational notes

6. Graceful AI failure UX

Anthropic errors are categorised into 8 friendly modes:

The AiSuggestionsCard renders each as a destructive-accented card with a Retry button (admin/lead only — testers can't burn budget).

Files:

• Categoriser: src/lib/ai/intake.ts → categoriseError() + AI_FAILURE_REASONS
• API: GET /api/test-cycles/:id/issues/:issueId returns aiLatestFailure
• UI: src/components/test-cycles/AiSuggestionsCard.tsx → failure state

7. Tester role + product URL

• ROLES.QA = 4 (existed in DB) is now selectable from Profile → Actions → Set as Tester. UI label is "Tester" everywhere; internal constant stays ROLES.QA for backward compat
• products.url (new column, migration 0022_conscious_eternals.sql) — where QA navigates to test. Surfaced as a tinted "Product under test" callout on the cycle Brief tab + a small Globe link under the cycle header
• Discipline taxonomy split: legacy web → web_app (interactive) and website (marketing). Old web rows render gracefully via legacyDisciplineLabel until edited

Files:

• Helpers: src/lib/profile-utils.ts → getRoleName, getRoleBadgeVariant
• Profile UI: src/components/profile/ProfileHeader.tsx
• Admin user list: src/app/app/admin/all-users/columns.tsx
• Admin product UI: src/app/app/admin/products/page.tsx

Database tables added

All migrations are additive — no destructive changes to legacy quiz schema.

Polymorphic table:

attachments
  target_type ∈ {test_cycle, issue, issue_comment, cycle_document, test_run}
  target_id   uuid (refers to the row in the corresponding table)
  s3_key      DO Spaces key
  media_kind  ∈ {image, video, file}

The polymorphism keeps one upload pipeline (POST /api/test-cycles/:id/attachments) for every attachment context.

API surface added

All new routes are scoped under /api/test-cycles/* (member or admin access) and /api/admin/{organizations,products,test-cycles}/* (admin only).

Admin

GET    /api/admin/organizations
POST   /api/admin/organizations
GET    /api/admin/organizations/:id
PATCH  /api/admin/organizations/:id
DELETE /api/admin/organizations/:id

GET    /api/admin/products              ?organizationId=
POST   /api/admin/products
GET    /api/admin/products/:id
PATCH  /api/admin/products/:id
DELETE /api/admin/products/:id
GET    /api/admin/products/:id/versions
POST   /api/admin/products/:id/versions

GET    /api/admin/test-cycles
POST   /api/admin/test-cycles
GET    /api/admin/test-cycles/:id
PATCH  /api/admin/test-cycles/:id        # status transitions trigger cycle-close agent
DELETE /api/admin/test-cycles/:id
POST   /api/admin/test-cycles/:id/testers
DELETE /api/admin/test-cycles/:id/testers/:testerId
PATCH  /api/admin/users/:userId/role     # for Tester role assignment

Member-scoped

GET    /api/test-cycles                                  # cycles I'm in
GET    /api/test-cycles/:id
POST   /api/test-cycles/:id/attachments                  # polymorphic upload

GET    /api/test-cycles/:id/runs
POST   /api/test-cycles/:id/runs                         # start run
GET    /api/test-cycles/:id/runs/:runId                  # includes pack attachments
PATCH  /api/test-cycles/:id/runs/:runId                  # status=completed triggers run-summary agent

GET    /api/test-cycles/:id/issues
POST   /api/test-cycles/:id/issues                       # non-draft triggers intake agent
GET    /api/test-cycles/:id/issues/:issueId              # includes aiSuggestions, aiLatestFailure
PATCH  /api/test-cycles/:id/issues/:issueId
GET    /api/test-cycles/:id/issues/:issueId/comments
POST   /api/test-cycles/:id/issues/:issueId/comments
POST   /api/test-cycles/:id/issues/:issueId/intake       # admin/lead retry intake agent
GET    /api/test-cycles/:id/issues/:issueId/payments
POST   /api/test-cycles/:id/issues/:issueId/payments

GET    /api/test-cycles/:id/documents
POST   /api/test-cycles/:id/documents                    # admin/lead only
GET    /api/test-cycles/:id/documents/:docId
PATCH  /api/test-cycles/:id/documents/:docId             # admin/lead only
DELETE /api/test-cycles/:id/documents/:docId             # admin/lead only

GET    /api/test-cycles/issues                           # cross-cycle issues view

Auth

• Most routes use checkTestCycleAccess(cycleId) — admin or member
• Doc write routes additionally check isAdmin || role === 'lead'
• Admin routes use requireAdmin / requireAdminForRoute middleware

UI surfaces added

/app/test-cycles                    My cycles list
/app/test-cycles/:id                Cycle detail (Brief / Docs / Issues / My runs / Testers tabs)
/app/test-cycles/:id/issues/new
/app/test-cycles/:id/issues/:id
/app/test-cycles/:id/issues/:id/edit
/app/issues             Cross-cycle "All issues" view

/app/admin/organizations            Org CRUD
/app/admin/products                 Product CRUD (with URL + discipline)
/app/admin/test-cycles              Cycle CRUD
/app/admin/test-cycles/:id          Admin cycle detail
/app/admin/test-cycles/:id/edit

Sidebar QASectionSidebar for the test-cycles vertical wraps the routes via src/app/app/test-cycles/layout.tsx.

Cost notes

Per-cycle AI cost for a typical 50-issue cycle:

• 50 intake calls × ~$0.01 = $0.50
• 5 run summaries × ~$0.005 = $0.025
• 1 cycle report × ~$0.05 = $0.05

Total ≈ $0.60 per cycle. Logged in ai_runs.cost_usd_cents. We do not yet have a hard per-cycle budget cap (deferred — cost_usd_cents is tracked, just not enforced).

Backlog (not yet built)

• Duplicate detection — embeddings + pgvector for "this looks like #42" suggestions. Or in-context Claude compare with no infra.
• AI cost cap per cycle — hard limit on ai_runs.cost_usd_cents sum
• Issue attachment lightbox — click-to-zoom for image attachments
• Linear/Jira outbound sync — external_* columns already on issues; needs the "Send to Linear" button + integration creds vault
• Email notifications for issue events
• Block-based editing — markdown for v1; ProseMirror later

Quick links to deeper docs

• Deployment — droplet ops, GHCR, AI key setup
• Onboarding — QA-side handbook (also auto-seeded into the Hackorda Onboarding cycle)
• Hackorda Test Cases — 52 test cases for the cycle (also auto-seeded)