Require Playwright E2E tests in every verification step
- New skill e2e-testing: on-demand Playwright setup (config, test:e2e script), acceptance-criteria-driven specs (one per feature + app-shell smoke), blueprint-driven semantic locators, and the shared verification checklist - /add-feature Step 7 and /execute-milestones Step 2.6 now write/update E2E specs and run npm run test:e2e alongside test/lint/typecheck; milestone gates and completion criteria include E2E - /generate-app Step 5 sets up Playwright and creates the app-shell smoke spec for the web target (integration_test / UI automation noted for flutter/winui3) - steering tasklist template, milestone-planning Definition of Done, milestone-execution task slicing/gates, development-guidelines pointers, and implementation-validator checks updated - CLAUDE.md, README, and settings.json updated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@ -90,6 +90,7 @@ When creating development guidelines: ./template.md
|
||||
### When Designing Tests
|
||||
- "Test Strategy" in ./guides/process.md (pyramid, coverage)
|
||||
- "Test Code" in ./guides/implementation.md (implementation patterns)
|
||||
- E2E tests: the **e2e-testing skill** (Playwright — E2E is part of every verification step)
|
||||
|
||||
### When Preparing a Release
|
||||
- "Git Workflow Rules" in ./guides/process.md (policy for merging into main)
|
||||
|
||||
118
.claude/skills/e2e-testing/SKILL.md
Normal file
118
.claude/skills/e2e-testing/SKILL.md
Normal file
@ -0,0 +1,118 @@
|
||||
---
|
||||
name: e2e-testing
|
||||
description: Guide for writing and running Playwright end-to-end tests as part of every verification step. Use when verifying a feature, milestone, or generated app shell, or when setting up Playwright in the repository.
|
||||
allowed-tools: Read, Write, Edit, Bash
|
||||
---
|
||||
|
||||
# E2E Testing Skill (Playwright)
|
||||
|
||||
This skill defines how end-to-end tests are written and run in this project. **Every verification step includes E2E tests**: a feature, milestone, or app shell is not "verified" until its Playwright tests pass alongside unit tests, lint, and typecheck.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **E2E is part of verification, not an extra**: the standard verification sequence is `npm test` → `npm run lint` → `npm run typecheck` → `npm run test:e2e`. All four must pass.
|
||||
2. **Tests are derived from acceptance criteria**: every feature's acceptance criteria (from its phase file or steering `requirements.md`) map to E2E assertions. If a criterion cannot be asserted in an E2E test, note why in the tasklist.
|
||||
3. **Blueprint-driven selectors**: screens, routes, and actions come from `docs/design/ui-blueprint.json`. Tests navigate by the blueprint's routes and interact via the blueprint's actions.
|
||||
4. **One spec file per feature** plus one smoke spec for the app shell.
|
||||
5. **Web-first**: Playwright covers the `web` target. For `flutter` use `integration_test`, for `winui3` use the platform's UI automation — apply the same acceptance-criteria mapping; note in the tasklist when Playwright does not apply.
|
||||
|
||||
## Setup (run once, on demand)
|
||||
|
||||
If Playwright is not yet set up in the repository (no `@playwright/test` in `package.json`), set it up before the first E2E run:
|
||||
|
||||
```bash
|
||||
npm install -D @playwright/test
|
||||
npx playwright install --with-deps chromium
|
||||
```
|
||||
|
||||
Create `playwright.config.ts`:
|
||||
|
||||
```typescript
|
||||
import { defineConfig } from '@playwright/test';
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './e2e',
|
||||
fullyParallel: true,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
reporter: [['list'], ['html', { open: 'never' }]],
|
||||
use: {
|
||||
baseURL: process.env.E2E_BASE_URL ?? 'http://localhost:3000',
|
||||
trace: 'retain-on-failure',
|
||||
screenshot: 'only-on-failure',
|
||||
},
|
||||
webServer: {
|
||||
command: 'npm run dev', // adjust to the project's start command
|
||||
url: 'http://localhost:3000', // adjust to the project's port
|
||||
reuseExistingServer: !process.env.CI,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Add the script to `package.json`:
|
||||
|
||||
```json
|
||||
"test:e2e": "playwright test"
|
||||
```
|
||||
|
||||
Adjust `baseURL`, `webServer.command`, and the port to the project's actual dev server (see `docs/architecture.md`). Add `e2e/` artifacts (`playwright-report/`, `test-results/`) to `.gitignore`.
|
||||
|
||||
## Test Layout
|
||||
|
||||
```
|
||||
e2e/
|
||||
├── app-shell.spec.ts # smoke: shell renders, every blueprint route navigates
|
||||
├── [feature-name].spec.ts # one spec per feature
|
||||
└── helpers/ # shared fixtures/utilities (only when duplicated 3+ times)
|
||||
```
|
||||
|
||||
## Writing Feature Specs
|
||||
|
||||
For each feature, create `e2e/[kebab-case-feature-name].spec.ts`:
|
||||
|
||||
- One `test.describe` block per feature; one `test` per acceptance criterion (or a tight group of criteria).
|
||||
- Navigate using the route of the affected screen from `ui-blueprint.json`.
|
||||
- Prefer semantic locators: `getByRole`, `getByLabel`, `getByText` — matching the blueprint's component types and labels. Avoid CSS/XPath selectors tied to implementation details.
|
||||
- No fixed waits (`waitForTimeout`); rely on Playwright's auto-waiting and explicit `expect(...).toBeVisible()` style assertions.
|
||||
- Keep tests independent: each test sets up its own state and does not depend on execution order.
|
||||
|
||||
Example:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Edit user profile', () => {
|
||||
test('saves a changed display name', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
await page.getByLabel('Display name').fill('New Name');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
await expect(page.getByText('Profile updated')).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Writing the App-Shell Smoke Spec
|
||||
|
||||
Created by `/generate-app` (web target). `e2e/app-shell.spec.ts` must verify:
|
||||
|
||||
- The app starts and the entry screen renders.
|
||||
- Every route in `ui-blueprint.json` is reachable and renders its screen title.
|
||||
- Primary navigation (from the blueprint's nav component) works.
|
||||
|
||||
## Running
|
||||
|
||||
```bash
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
- If a test fails, analyze the trace/screenshot in `test-results/`, fix the code (or the test if the spec was wrong against the acceptance criteria), and re-run until green.
|
||||
- Never delete or skip a failing test to make verification pass; a skipped test requires a written reason in the tasklist, same as a struck task.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Used by `/add-feature` Step 7, `/execute-milestones` Step 2.6, and `/generate-app` Step 5:
|
||||
|
||||
- [ ] Playwright is set up (`test:e2e` script exists) — set it up via this skill if missing.
|
||||
- [ ] Every implemented feature has `e2e/[feature].spec.ts` covering its acceptance criteria.
|
||||
- [ ] The app-shell smoke spec still passes.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, and `npm run test:e2e` all pass.
|
||||
- [ ] No test was skipped or deleted without a written reason.
|
||||
@ -46,14 +46,15 @@ Tasks are **grouped by feature**, in the implementation order chosen in `design.
|
||||
- [ ] [Task 1: data/model layer]
|
||||
- [ ] [Task 2: logic/service layer]
|
||||
- [ ] [Task 3: UI per affected screen]
|
||||
- [ ] [Task 4: tests for the acceptance criteria]
|
||||
- [ ] [Task 4: unit tests + Playwright E2E spec for the acceptance criteria]
|
||||
|
||||
## Feature: Log in / log out
|
||||
- [ ] ...
|
||||
|
||||
## Milestone Verification
|
||||
- [ ] All acceptance criteria in requirements.md confirmed
|
||||
- [ ] npm test / lint / typecheck pass
|
||||
- [ ] E2E specs exist for every feature (e2e/[feature].spec.ts, per the e2e-testing skill)
|
||||
- [ ] npm test / lint / typecheck / test:e2e pass
|
||||
|
||||
## Post-Implementation Retrospective
|
||||
_Filled at the end: completion date, plan vs. actual, lessons learned._
|
||||
@ -62,8 +63,8 @@ _Filled at the end: completion date, plan vs. actual, lessons learned._
|
||||
## How to Divide a Feature into Tasks
|
||||
|
||||
- 3–8 tasks per feature; each completable in **one implementation-loop iteration**.
|
||||
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests.
|
||||
- Every feature's section ends with a task that verifies its acceptance criteria.
|
||||
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests (unit + Playwright E2E).
|
||||
- Every feature's section ends with a task that verifies its acceptance criteria via the feature's E2E spec (see the e2e-testing skill).
|
||||
- Tasks that unblock several features go into a "Shared Foundations" section at the top — never duplicated per feature.
|
||||
|
||||
## Gate Criteria Between Milestones
|
||||
@ -72,7 +73,8 @@ A milestone passes its gate only when **all** of the following hold. Only then m
|
||||
|
||||
- [ ] Every task in the milestone's `tasklist.md` is `[x]` (or struck with a technical reason).
|
||||
- [ ] The `implementation-validator` subagent's validation passed.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck` all succeed.
|
||||
- [ ] Every feature of the milestone has a Playwright E2E spec covering its acceptance criteria.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` all succeed.
|
||||
- [ ] The six persistent documents in `docs/` are reconciled with the milestone's changes.
|
||||
- [ ] In the phase file: every feature of the milestone is checked off, and the milestone status is `Completed` with Completion Notes.
|
||||
- [ ] `roadmap.md` points to the next milestone (or, after the phase's last milestone, the phase is `Completed` and the roadmap points to the next phase file).
|
||||
|
||||
@ -59,7 +59,8 @@ Use this structure for each `docs/milestones/phase[N]-milestones.md`. A phase fi
|
||||
### Definition of Done
|
||||
|
||||
- [ ] All features above are checked off
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck` pass
|
||||
- [ ] Every feature has a Playwright E2E spec covering its acceptance criteria
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` pass
|
||||
- [ ] Persistent documents in `docs/` are reconciled with all features
|
||||
- [ ] [Milestone-specific criterion, e.g. "demo flow X→Y→Z works end-to-end"]
|
||||
|
||||
|
||||
@ -59,6 +59,9 @@ When skipping, always state the reason clearly:
|
||||
- [ ] `npm run lint`
|
||||
- [ ] Confirm that there are no type errors
|
||||
- [ ] `npm run typecheck`
|
||||
- [ ] Confirm that the E2E tests pass (Playwright — see the e2e-testing skill)
|
||||
- [ ] E2E specs exist for this work's acceptance criteria
|
||||
- [ ] `npm run test:e2e`
|
||||
- [ ] Confirm that the build succeeds
|
||||
- [ ] `npm run build`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user