Require Playwright E2E tests in every verification step

- New skill e2e-testing: on-demand Playwright setup (config, test:e2e script), acceptance-criteria-driven specs (one per feature + app-shell smoke), blueprint-driven semantic locators, and the shared verification checklist
- /add-feature Step 7 and /execute-milestones Step 2.6 now write/update E2E specs and run npm run test:e2e alongside test/lint/typecheck; milestone gates and completion criteria include E2E
- /generate-app Step 5 sets up Playwright and creates the app-shell smoke spec for the web target (integration_test / UI automation noted for flutter/winui3)
- steering tasklist template, milestone-planning Definition of Done, milestone-execution task slicing/gates, development-guidelines pointers, and implementation-validator checks updated
- CLAUDE.md, README, and settings.json updated

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-02 12:26:15 +02:00
parent 33b33c4bab
commit 1b67d1f15e
12 changed files with 166 additions and 21 deletions

View File

@ -53,6 +53,7 @@ Verify that the implemented code meets the following criteria:
- [ ] Is the coverage target met?
- [ ] Are edge cases tested?
- [ ] Are the tests named appropriately?
- [ ] Are there Playwright E2E tests covering the feature's acceptance criteria (e2e/*.spec.ts)?
**Evaluation Criteria**:
- ✅ Sufficient: Coverage of 80% or more, covering the main cases
@ -213,6 +214,11 @@ npm test
npm run test:coverage
```
### Run E2E Tests (Playwright)
```bash
npm run test:e2e
```
### Build Check
```bash
npm run build

View File

@ -121,15 +121,19 @@ If any of the following situations occur while the implementation loop is runnin
**Once this step completes successfully, never stop; immediately proceed to Step 7.**
## Step 7: Run Automated Tests
## Step 7: Run Automated Tests (including Playwright E2E)
1. Run the following commands in order and confirm that all tests pass.
1. Write or update the feature's end-to-end tests following the **e2e-testing skill** (`Skill('e2e-testing')`):
- Create/update `e2e/[feature-name].spec.ts` covering the feature's acceptance criteria.
- If Playwright is not yet set up in the repository, set it up first as described in the skill.
2. Run the following commands in order and confirm that all of them pass.
```bash
Bash('npm test')
Bash('npm run lint')
Bash('npm run typecheck')
Bash('npm run test:e2e')
```
2. If any command produces an error, analyze the problem, generate and apply a fix, and then run this step again.
3. If any command produces an error, analyze the problem, generate and apply a fix, and then run this step again. Never skip or delete a failing test to make verification pass.
**Once this step completes successfully, never stop; immediately proceed to Step 8.**
@ -164,7 +168,7 @@ If any of the following situations occur while the implementation loop is runnin
This workflow completes automatically once all of the following conditions are met.
- Step 5: All tasks in `tasklist.md` are complete (`[x]` or skipped for a valid reason).
- Step 6: The `implementation-validator` subagent's validation passes.
- Step 7: The `test`, `lint`, and `typecheck` commands all succeed without errors.
- Step 7: The `test`, `lint`, `typecheck`, and `test:e2e` (Playwright) commands all succeed without errors, and the feature has E2E tests covering its acceptance criteria.
- Step 8: Handover notes are recorded in `tasklist.md`, all six persistent documents in `docs/` have been reconciled with the feature (updates applied, or a skip reason noted for each), and the feature is checked off in its phase file (if `docs/milestones/` exists).
Until these completion criteria are met, continue to think autonomously, solve problems, and carry on the work.

View File

@ -103,15 +103,19 @@ Use the `Task` tool to launch the `implementation-validator` subagent:
- `description`: "Milestone implementation validation"
- `prompt`: "Please validate the quality of all changes implemented for milestone `[NN] [name]`. The target files are `[list of implemented file paths]`. Focus on coding standards, error handling, testability, consistency with existing patterns, and whether every feature's acceptance criteria in `.steering/[dir]/requirements.md` are met."
#### 2.6 Run Automated Tests
#### 2.6 Run Automated Tests (including Playwright E2E)
1. Write or update the milestone's end-to-end tests following the **e2e-testing skill** (`Skill('e2e-testing')`): one `e2e/[feature-name].spec.ts` per feature in the milestone, covering its acceptance criteria. Set up Playwright first (as described in the skill) if it is missing.
2. Run:
```bash
Bash('npm test')
Bash('npm run lint')
Bash('npm run typecheck')
Bash('npm run test:e2e')
```
If any command fails, analyze, fix, and re-run until all pass.
If any command fails, analyze, fix, and re-run until all pass. Never skip or delete a failing test to make verification pass.
#### 2.7 Retrospective and Document Reconciliation
@ -124,7 +128,7 @@ Apply `/add-feature` Step 8 for the milestone:
#### 2.8 Milestone Gate — then Continue
The milestone is done only when: all tasks are `[x]`, validation passed, tests/lint/typecheck are green, the six documents are reconciled, and the milestone status is updated in the phase file and the roadmap.
The milestone is done only when: all tasks are `[x]`, validation passed, tests/lint/typecheck/E2E (Playwright) are green, the six documents are reconciled, and the milestone status is updated in the phase file and the roadmap.
**Once the gate passes, never stop; immediately start Step 2 for the next milestone in the queue.** If the gate genuinely cannot be passed (external blocker), record the blocker in the milestone's `tasklist.md`, the phase file, and `roadmap.md`, then stop and report — this is the only permitted early stop.
@ -138,7 +142,7 @@ The milestone is done only when: all tasks are `[x]`, validation passed, tests/l
- Every milestone in the specified phase file has status `Completed`.
- The phase's status is `Completed` in both the phase file and `docs/milestones/roadmap.md`.
- Each executed milestone has its own `.steering/[date]-milestone-[NN]-[name]/` directory with all tasks `[x]`.
- Tests, lint, and typecheck pass on the final state.
- Tests, lint, typecheck, and Playwright E2E tests pass on the final state, with E2E coverage for every implemented feature's acceptance criteria.
- The six persistent documents reflect everything implemented.
Completion message:

View File

@ -117,10 +117,13 @@ Apply the requirements for the selected target:
If new directories/files were created, update `docs/repository-structure.md` so it reflects the generated layout.
### Step 5: Verify
### Step 5: Verify (including Playwright E2E)
- If the target is `web` and the repo has `npm test`, `npm run lint`, `npm run typecheck`, run them and fix any errors caused by the generated code.
- For `flutter` / `winui3`, run the platform's available static checks if the toolchain is present; otherwise note what should be checked manually.
- If the target is `web`:
- Run `npm test`, `npm run lint`, `npm run typecheck` (where present) and fix any errors caused by the generated code.
- Set up Playwright following the **e2e-testing skill** (`Skill('e2e-testing')`) if it is not set up yet, and create the app-shell smoke spec `e2e/app-shell.spec.ts` (shell renders, every blueprint route navigates, primary navigation works).
- Run `npm run test:e2e` and fix any failures caused by the generated code.
- For `flutter` / `winui3`, run the platform's available static checks and UI/integration tests if the toolchain is present (`integration_test` for Flutter, the platform's UI automation for WinUI 3); otherwise note what should be checked manually.
## Completion Criteria
@ -140,6 +143,7 @@ Generated:
✅ Routes / navigation (from ui-blueprint.json)
✅ Screens (one per blueprint screen)
✅ Reusable components
✅ Playwright E2E smoke spec (e2e/app-shell.spec.ts, web target)
Next steps:
- Run the app and verify the shell renders

View File

@ -12,6 +12,7 @@
"Skill(platform-ui-generation)",
"Skill(milestone-planning)",
"Skill(milestone-execution)",
"Skill(e2e-testing)",
"Skill(steering)"
],
"deny": [],

View File

@ -90,6 +90,7 @@ When creating development guidelines: ./template.md
### When Designing Tests
- "Test Strategy" in ./guides/process.md (pyramid, coverage)
- "Test Code" in ./guides/implementation.md (implementation patterns)
- E2E tests: the **e2e-testing skill** (Playwright — E2E is part of every verification step)
### When Preparing a Release
- "Git Workflow Rules" in ./guides/process.md (policy for merging into main)

View File

@ -0,0 +1,118 @@
---
name: e2e-testing
description: Guide for writing and running Playwright end-to-end tests as part of every verification step. Use when verifying a feature, milestone, or generated app shell, or when setting up Playwright in the repository.
allowed-tools: Read, Write, Edit, Bash
---
# E2E Testing Skill (Playwright)
This skill defines how end-to-end tests are written and run in this project. **Every verification step includes E2E tests**: a feature, milestone, or app shell is not "verified" until its Playwright tests pass alongside unit tests, lint, and typecheck.
## Core Principles
1. **E2E is part of verification, not an extra**: the standard verification sequence is `npm test``npm run lint``npm run typecheck``npm run test:e2e`. All four must pass.
2. **Tests are derived from acceptance criteria**: every feature's acceptance criteria (from its phase file or steering `requirements.md`) map to E2E assertions. If a criterion cannot be asserted in an E2E test, note why in the tasklist.
3. **Blueprint-driven selectors**: screens, routes, and actions come from `docs/design/ui-blueprint.json`. Tests navigate by the blueprint's routes and interact via the blueprint's actions.
4. **One spec file per feature** plus one smoke spec for the app shell.
5. **Web-first**: Playwright covers the `web` target. For `flutter` use `integration_test`, for `winui3` use the platform's UI automation — apply the same acceptance-criteria mapping; note in the tasklist when Playwright does not apply.
## Setup (run once, on demand)
If Playwright is not yet set up in the repository (no `@playwright/test` in `package.json`), set it up before the first E2E run:
```bash
npm install -D @playwright/test
npx playwright install --with-deps chromium
```
Create `playwright.config.ts`:
```typescript
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
retries: process.env.CI ? 2 : 0,
reporter: [['list'], ['html', { open: 'never' }]],
use: {
baseURL: process.env.E2E_BASE_URL ?? 'http://localhost:3000',
trace: 'retain-on-failure',
screenshot: 'only-on-failure',
},
webServer: {
command: 'npm run dev', // adjust to the project's start command
url: 'http://localhost:3000', // adjust to the project's port
reuseExistingServer: !process.env.CI,
},
});
```
Add the script to `package.json`:
```json
"test:e2e": "playwright test"
```
Adjust `baseURL`, `webServer.command`, and the port to the project's actual dev server (see `docs/architecture.md`). Add `e2e/` artifacts (`playwright-report/`, `test-results/`) to `.gitignore`.
## Test Layout
```
e2e/
├── app-shell.spec.ts # smoke: shell renders, every blueprint route navigates
├── [feature-name].spec.ts # one spec per feature
└── helpers/ # shared fixtures/utilities (only when duplicated 3+ times)
```
## Writing Feature Specs
For each feature, create `e2e/[kebab-case-feature-name].spec.ts`:
- One `test.describe` block per feature; one `test` per acceptance criterion (or a tight group of criteria).
- Navigate using the route of the affected screen from `ui-blueprint.json`.
- Prefer semantic locators: `getByRole`, `getByLabel`, `getByText` — matching the blueprint's component types and labels. Avoid CSS/XPath selectors tied to implementation details.
- No fixed waits (`waitForTimeout`); rely on Playwright's auto-waiting and explicit `expect(...).toBeVisible()` style assertions.
- Keep tests independent: each test sets up its own state and does not depend on execution order.
Example:
```typescript
import { test, expect } from '@playwright/test';
test.describe('Edit user profile', () => {
test('saves a changed display name', async ({ page }) => {
await page.goto('/profile');
await page.getByLabel('Display name').fill('New Name');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Profile updated')).toBeVisible();
});
});
```
## Writing the App-Shell Smoke Spec
Created by `/generate-app` (web target). `e2e/app-shell.spec.ts` must verify:
- The app starts and the entry screen renders.
- Every route in `ui-blueprint.json` is reachable and renders its screen title.
- Primary navigation (from the blueprint's nav component) works.
## Running
```bash
npm run test:e2e
```
- If a test fails, analyze the trace/screenshot in `test-results/`, fix the code (or the test if the spec was wrong against the acceptance criteria), and re-run until green.
- Never delete or skip a failing test to make verification pass; a skipped test requires a written reason in the tasklist, same as a struck task.
## Verification Checklist
Used by `/add-feature` Step 7, `/execute-milestones` Step 2.6, and `/generate-app` Step 5:
- [ ] Playwright is set up (`test:e2e` script exists) — set it up via this skill if missing.
- [ ] Every implemented feature has `e2e/[feature].spec.ts` covering its acceptance criteria.
- [ ] The app-shell smoke spec still passes.
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, and `npm run test:e2e` all pass.
- [ ] No test was skipped or deleted without a written reason.

View File

@ -46,14 +46,15 @@ Tasks are **grouped by feature**, in the implementation order chosen in `design.
- [ ] [Task 1: data/model layer]
- [ ] [Task 2: logic/service layer]
- [ ] [Task 3: UI per affected screen]
- [ ] [Task 4: tests for the acceptance criteria]
- [ ] [Task 4: unit tests + Playwright E2E spec for the acceptance criteria]
## Feature: Log in / log out
- [ ] ...
## Milestone Verification
- [ ] All acceptance criteria in requirements.md confirmed
- [ ] npm test / lint / typecheck pass
- [ ] E2E specs exist for every feature (e2e/[feature].spec.ts, per the e2e-testing skill)
- [ ] npm test / lint / typecheck / test:e2e pass
## Post-Implementation Retrospective
_Filled at the end: completion date, plan vs. actual, lessons learned._
@ -62,8 +63,8 @@ _Filled at the end: completion date, plan vs. actual, lessons learned._
## How to Divide a Feature into Tasks
- 38 tasks per feature; each completable in **one implementation-loop iteration**.
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests.
- Every feature's section ends with a task that verifies its acceptance criteria.
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests (unit + Playwright E2E).
- Every feature's section ends with a task that verifies its acceptance criteria via the feature's E2E spec (see the e2e-testing skill).
- Tasks that unblock several features go into a "Shared Foundations" section at the top — never duplicated per feature.
## Gate Criteria Between Milestones
@ -72,7 +73,8 @@ A milestone passes its gate only when **all** of the following hold. Only then m
- [ ] Every task in the milestone's `tasklist.md` is `[x]` (or struck with a technical reason).
- [ ] The `implementation-validator` subagent's validation passed.
- [ ] `npm test`, `npm run lint`, `npm run typecheck` all succeed.
- [ ] Every feature of the milestone has a Playwright E2E spec covering its acceptance criteria.
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` all succeed.
- [ ] The six persistent documents in `docs/` are reconciled with the milestone's changes.
- [ ] In the phase file: every feature of the milestone is checked off, and the milestone status is `Completed` with Completion Notes.
- [ ] `roadmap.md` points to the next milestone (or, after the phase's last milestone, the phase is `Completed` and the roadmap points to the next phase file).

View File

@ -59,7 +59,8 @@ Use this structure for each `docs/milestones/phase[N]-milestones.md`. A phase fi
### Definition of Done
- [ ] All features above are checked off
- [ ] `npm test`, `npm run lint`, `npm run typecheck` pass
- [ ] Every feature has a Playwright E2E spec covering its acceptance criteria
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` pass
- [ ] Persistent documents in `docs/` are reconciled with all features
- [ ] [Milestone-specific criterion, e.g. "demo flow X→Y→Z works end-to-end"]

View File

@ -59,6 +59,9 @@ When skipping, always state the reason clearly:
- [ ] `npm run lint`
- [ ] Confirm that there are no type errors
- [ ] `npm run typecheck`
- [ ] Confirm that the E2E tests pass (Playwright — see the e2e-testing skill)
- [ ] E2E specs exist for this work's acceptance criteria
- [ ] `npm run test:e2e`
- [ ] Confirm that the build succeeds
- [ ] `npm run build`