Feature Matrix

Single source of truth for "what does the QA platform actually do today?" and "what events will we capture once we wire telemetry?".

📋 Flow specs live in Flows. Each canonical journey in §2 below has a dedicated F-NN-name.md flow file with steps, acceptance criteria, manual QA checklist, automated test stubs, and code references. QA engineers should hand-verify and write Playwright specs against those files. This matrix is the index; the flows directory is the plan.

Companion docs:

• Flows — file-per-flow QA specs (16 flows)
• Test Cycles — architectural reference

Update this file whenever a feature lands. Update the corresponding flow file in docs/flows/ whenever its behavior changes.

Scope. Test Cycles vertical only. The legacy quiz / events / leaderboard features are documented in features.md, api-routes.md, database-schema.md.

1. Actors

Two-axis access model: system role (global gate) × per-cycle role (per-cycle gate). ROLES.QA is purely a UI label — the actual permission gate is whether the user has a testers row on the cycle.

2. Canonical end-to-end flows

These are the reference scenarios to instrument. Every feature below is a step inside one of these flows. Each one has a dedicated flow doc in Flows — open it for steps, acceptance criteria, manual checklist, and test stubs.

Flow A — Cycle goes live

admin: create org → create product (+ URL) → create version
    → create test cycle (planned)
    → seed cycle docs (brief, runbook)
    → assign testers (or set "open to self-join")
    → flip status: planned → active   ← cycle.status_changed event
notification: every member gets "cycle is now active"

Flow B — Tester files a bug

tester: open active cycle → click "Start Run"   ← run.started
    → captures environment (UA, OS, browser, screen)
    → reproduces bug → click "+ File Issue"
    → fills title/description/severity, drops screenshot/video
    → submit                                       ← issue.filed
    → AI intake fires (fire-and-forget)            ← ai.intake_completed
    → leads + admins get bell notification
tester: end run → "Complete"                       ← run.completed
    → AI run summary fires (fire-and-forget)       ← ai.run_summary_completed

Flow C — Admin triages

admin: open /app/admin/triage (cross-cycle queue)
    → sees one card at a time with screenshots inline
    → optionally reclassify severity (recomputes payout amount)
    → optionally add comment / "Request info"      ← issue.info_requested
        (reporter gets bell + comment thread)
        (reporter replies → auto re-enters queue)  ← issue.info_provided
    → Approve  → payout_status='approved'          ← issue.approved
    → Reject   → status='rejected'                 ← issue.rejected
    → Skip     → next card (no DB write)
notification: reporter gets bell on approve/reject

Flow D — Verification gate (when active)

[gate active = cycle override OR product default = true]
admin: marks issue triaged → dev fixes → admin marks 'verified'
    → only then can it be marked paid                ← issue.verified
[gate inactive] approval is enough; verification step skipped.

Flow E — Period batch payout (the preferred settlement path)

admin: open /app/admin/payouts → "Period batch" panel
    → pick range (Today / Last 7 days / This month / Last month / custom)
    → live dry-run preview shows N issues, K testers, totals/currency
    → enter method (Kaspi/bank/cash/other), reference, notes
    → confirm → POST /api/admin/payouts/run-batch (dryRun=false)
        → creates payout_batches row                  ← payout.batch_run
        → inserts N issue_payouts rows (with batch_id)
        → flips N issues to paid
        → fans out ISSUE_PAID notifications
notification: each reporter sees "Paid: $X via Kaspi · ref ABC"

Flow F — Tester earnings dashboard

tester: /app/test-cycles/balance
    → 5 columns: Awaiting triage · Pending verification ·
                 Available · Total paid · Reports
    → by-cycle list + full issue history table
    → can export own earnings as CSV               ← balance.csv_exported

Flow G — Linear export (engineering handoff)

admin: open issue detail or triage → "Export to Linear"
    → single OR group mode (multi-select)
    → preview Linear team + project + status
    → confirm → creates Linear issue(s) via GraphQL
        → stores external_id / external_url on issues  ← issue.exported_to_linear

Flow H — Cycle close (with AI report)

admin: cycle status → 'closed'
    → AI cycle-report agent fires                   ← ai.cycle_report_completed
    → new cycle_documents row with kind='report'
    → cycle.status_changed notification fan-out

3. Feature registry

Status legend: ✅ shipped · 🟡 partial / behind flag · 🔲 planned · 🚫 deferred.

3.1 Identity & access

3.2 Catalog (orgs, products, versions, cycles)

3.3 Cycle execution (tester flow)

3.4 Triage & evaluation (admin/lead flow)

3.5 Payouts (admin flow)

3.6 Tester earnings (tester flow)

3.7 Notifications

3.8 AI agents (Anthropic)

3.9 Integrations

3.10 Cross-cutting

4. Event taxonomy (proposed)

When we wire telemetry, we'll capture events using the domain.action convention (lowercase, snake_case). The "Suggested event" column above is the draft registry. Capture rules below.

4.1 Naming

<domain>.<action>           e.g.  issue.filed, payout.batch_run
<domain>.<action>_failed    e.g.  ai.intake_failed

Domains we use today: auth, user, org, product, cycle, cycle_doc, run, issue, issue_comment, attachment, payout, balance, notification, ai, integration, admin_dashboard.

4.2 Required properties on every event

Whatever the event, we capture:

• event — name
• actor_id — users.id
• actor_role — system role (admin / qa / student)
• cycle_id — when scoped to a cycle (most events)
• org_id, product_id — denormalised when present
• issue_id — when scoped to an issue
• created_at — server timestamp
• metadata — jsonb with action-specific payload (severity, currency, batch_id, …)

4.3 Where to instrument

We don't have a telemetry sink yet. Three options when we wire it:

1. Append-only events table (cheapest, one migration, queryable in Postgres). Drop a db.insert(events).values(...) next to every existing console.log at the eight sites listed below.
2. PostHog / Plausible / Mixpanel via a src/lib/track.ts client wrapper.
3. Both — DB for forensics, vendor for dashboards.

Recommended: option 1 first (no vendor, no key, no SLA), promote to option 3 when we want product analytics.

4.4 Eight existing log sites that should become events

(grep "console.log" src/app/api/admin src/app/api/test-cycles — all of these already log a structured one-liner; replace console.log with track(...) plus the line.)

That's the path of least resistance — those eight lines + a thin track() helper give us 80% of the analytic value with one PR.

5. Gaps — what's left to build

Bucketed by impact.

5.1 Operational (no code, your turn)

• ☐ Smoke-test the period batch flow end-to-end on production (file → approve → verify → batch → tester sees Paid + bell)
• ☐ Set Anthropic API key at /app/admin/integrations (frees us from env vars)
• ☐ Set Linear API key at /app/admin/integrations
• ☐ Phone-test the new mobile triage overlay (#175)

5.2 High-value code

• ☐ Past batches view — drill into a payout_batches row, list issues paid, regenerate receipt, void if needed
• ☐ Telemetry / event sink — wire the 8 log sites above to a real events table (1 migration + 1 helper)
• ☐ Cycle issues list polish — payout-status badge column, attachment count, severity filter chips, group-by-status
• ☐ PWA manifest — installable on phone (low effort, big win for tester ergonomics)
• ☐ Mobile audit on issue detail (same overlay treatment as triage if needed)

5.3 Bigger workstreams

• ☐ Telegram bot fan-out for notifications
• ☐ Multi-tenant Linear (per-org config)
• ☐ Linear → Hackorda webhook (pull external_status back)
• ☐ Cycle close report as downloadable PDF
• ☐ Email notifications
• ☐ Playwright Phase 4 (visual snapshot tests)

5.4 Deferred from original plan

• ☐ Encrypted credentials store (creds currently live in plain organization_integrations.config)
• ☐ pgvector embeddings for issue dedup
• ☐ Test cases / test plans (explicitly out-of-scope from v1; cycle docs cover the immediate need)

6. How to update this doc

When you ship a feature:

1. Add a row to the right table in §3 (use the same shape).
2. Suggest an event name in the rightmost column even if you don't wire it.
3. If the feature defines a new end-to-end flow, add it to §2.
4. If the feature exposes a new gap, add it to §5.

When you wire telemetry:

1. Pick events from §4 (don't invent new names — change this doc first).
2. Update the rightmost column to ✅ once instrumented.

Drift check. If this doc and docs/test-cycles.md disagree, both are wrong. test-cycles.md is the architectural reference; this doc is the product-shaped view. They should describe the same code, just at different angles.

Last updated: 2026-05-08 (after #176 — period batch payouts).