Readme

Versioned record of every manual + automated test scenario for the app. The catalog is the entry point — both for humans walking through a release and for whichever agent is helping verify a change.

Why this exists

We've shipped a lot of money-adjacent flows (verifier bounties, reporter payouts, feature cycles) and most of them only have happy-path E2E coverage. The Vitest suite (added in #266) covers pure logic; this catalog covers stateful flows — the things that need a click trail through real UIs to verify.

Long term it converts into automation: each scenario file has an automated_test: field; once a Playwright spec lands, the scenario gets stamped automatically by CI instead of by hand.

Layout

docs/qa/
├── README.md                       ← this file
├── verifier-bounty/
│   ├── INDEX.md                    ← feature overview + scenario list
│   ├── 001-tester-files-bug.md
│   ├── 002-engineer-fixes-bug.md
│   └── ...
├── feature-cycles/
│   ├── INDEX.md
│   └── ...
└── <other-features>/

One feature per subdirectory. One scenario per file. The scenario ID (VERBO-001, FEAT-001, etc.) is short, durable, and stable across file moves — never re-number, only deprecate.

Scenario file shape

---
id: VERBO-001
feature: verifier-bounty
title: Tester files a bug on an active cycle
status: untested        # untested | passing | failing | blocked | deprecated
last_verified: null     # ISO-8601 UTC timestamp, or null
verified_by: null       # handle (e.g. adilet), or null
automated_test: null    # path to Playwright/Vitest spec, or null
---

## Setup
- Preconditions, test accounts, seed data, etc.

## Steps
1. Numbered, terse, specific to the UI.

## Expected
- Observable outcomes. One bullet per outcome.

## Notes
- Anything contextual: gotchas, related PRs, deferred edge cases.

## Verification log
- 2026-05-27T23:50:00Z — adilet — PASS — first verification post-Phase 4
- 2026-05-25T14:00:00Z — adilet — PASS — initial walkthrough

The Verification log is append-only. The frontmatter last_verified / verified_by mirror the most recent log entry — they're there for machine consumption (audits, dashboards), the log is for humans.

Workflow

Manual verification session

1. Pick a scenario (catalog or follow the next_up pointer in INDEX.md).
2. Walk through it.
3. Tell the agent the result:
   • pass — straightforward
   • fail: <what happened> — agent files an issue (or notes in the log)
   • blocked: <why> — couldn't get to the assertion (missing seed data, unrelated bug, etc.)
4. Agent updates the frontmatter + appends a log entry + offers the next scenario.

Adding a new scenario

1. Pick the right feature directory (or create a new one with its own INDEX.md).
2. Next sequential number: NNN-kebab-slug.md.
3. Copy the frontmatter shape; status starts at untested.
4. Add a link to the feature's INDEX.md queue.

Bridging to automation

When a Playwright spec covers the scenario, set the automated_test: field to the spec path. A future scripts/audit-qa.ts will:

• Verify the spec file exists
• Update last_verified from the CI run timestamp
• Flag drift (scenario marked passing but the spec is failing)

Until then, automated tests just get the automated_test: field as a pointer — they don't auto-update timestamps yet.

Status values

Staleness threshold

A passing scenario whose last_verified is older than 90 days is considered stale — scripts/audit-qa.ts (future) will surface these weekly. Re-run a stale scenario or convert it to automation.

Reference: existing Playwright specs (already covered)

These scenarios have automation today, but no catalog entry yet — they'll get one each as we walk through the bounty flow:

• tests/e2e/flows/self-join.spec.ts
• tests/e2e/flows/triage/reclassify.spec.ts
• tests/e2e/flows/testing/file-issue.spec.ts
• tests/e2e/flows/payouts/batch-pay.spec.ts
• tests/e2e/flows/payouts/balance-reflects-payment.spec.ts
• tests/e2e/flows/payouts/csv-export.spec.ts