- New skill e2e-testing: on-demand Playwright setup (config, test:e2e script), acceptance-criteria-driven specs (one per feature + app-shell smoke), blueprint-driven semantic locators, and the shared verification checklist - /add-feature Step 7 and /execute-milestones Step 2.6 now write/update E2E specs and run npm run test:e2e alongside test/lint/typecheck; milestone gates and completion criteria include E2E - /generate-app Step 5 sets up Playwright and creates the app-shell smoke spec for the web target (integration_test / UI automation noted for flutter/winui3) - steering tasklist template, milestone-planning Definition of Done, milestone-execution task slicing/gates, development-guidelines pointers, and implementation-validator checks updated - CLAUDE.md, README, and settings.json updated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5.1 KiB
| name | description | allowed-tools |
|---|---|---|
| e2e-testing | Guide for writing and running Playwright end-to-end tests as part of every verification step. Use when verifying a feature, milestone, or generated app shell, or when setting up Playwright in the repository. | Read, Write, Edit, Bash |
E2E Testing Skill (Playwright)
This skill defines how end-to-end tests are written and run in this project. Every verification step includes E2E tests: a feature, milestone, or app shell is not "verified" until its Playwright tests pass alongside unit tests, lint, and typecheck.
Core Principles
- E2E is part of verification, not an extra: the standard verification sequence is
npm test→npm run lint→npm run typecheck→npm run test:e2e. All four must pass. - Tests are derived from acceptance criteria: every feature's acceptance criteria (from its phase file or steering
requirements.md) map to E2E assertions. If a criterion cannot be asserted in an E2E test, note why in the tasklist. - Blueprint-driven selectors: screens, routes, and actions come from
docs/design/ui-blueprint.json. Tests navigate by the blueprint's routes and interact via the blueprint's actions. - One spec file per feature plus one smoke spec for the app shell.
- Web-first: Playwright covers the
webtarget. Forflutteruseintegration_test, forwinui3use the platform's UI automation — apply the same acceptance-criteria mapping; note in the tasklist when Playwright does not apply.
Setup (run once, on demand)
If Playwright is not yet set up in the repository (no @playwright/test in package.json), set it up before the first E2E run:
npm install -D @playwright/test
npx playwright install --with-deps chromium
Create playwright.config.ts:
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
retries: process.env.CI ? 2 : 0,
reporter: [['list'], ['html', { open: 'never' }]],
use: {
baseURL: process.env.E2E_BASE_URL ?? 'http://localhost:3000',
trace: 'retain-on-failure',
screenshot: 'only-on-failure',
},
webServer: {
command: 'npm run dev', // adjust to the project's start command
url: 'http://localhost:3000', // adjust to the project's port
reuseExistingServer: !process.env.CI,
},
});
Add the script to package.json:
"test:e2e": "playwright test"
Adjust baseURL, webServer.command, and the port to the project's actual dev server (see docs/architecture.md). Add e2e/ artifacts (playwright-report/, test-results/) to .gitignore.
Test Layout
e2e/
├── app-shell.spec.ts # smoke: shell renders, every blueprint route navigates
├── [feature-name].spec.ts # one spec per feature
└── helpers/ # shared fixtures/utilities (only when duplicated 3+ times)
Writing Feature Specs
For each feature, create e2e/[kebab-case-feature-name].spec.ts:
- One
test.describeblock per feature; onetestper acceptance criterion (or a tight group of criteria). - Navigate using the route of the affected screen from
ui-blueprint.json. - Prefer semantic locators:
getByRole,getByLabel,getByText— matching the blueprint's component types and labels. Avoid CSS/XPath selectors tied to implementation details. - No fixed waits (
waitForTimeout); rely on Playwright's auto-waiting and explicitexpect(...).toBeVisible()style assertions. - Keep tests independent: each test sets up its own state and does not depend on execution order.
Example:
import { test, expect } from '@playwright/test';
test.describe('Edit user profile', () => {
test('saves a changed display name', async ({ page }) => {
await page.goto('/profile');
await page.getByLabel('Display name').fill('New Name');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByText('Profile updated')).toBeVisible();
});
});
Writing the App-Shell Smoke Spec
Created by /generate-app (web target). e2e/app-shell.spec.ts must verify:
- The app starts and the entry screen renders.
- Every route in
ui-blueprint.jsonis reachable and renders its screen title. - Primary navigation (from the blueprint's nav component) works.
Running
npm run test:e2e
- If a test fails, analyze the trace/screenshot in
test-results/, fix the code (or the test if the spec was wrong against the acceptance criteria), and re-run until green. - Never delete or skip a failing test to make verification pass; a skipped test requires a written reason in the tasklist, same as a struck task.
Verification Checklist
Used by /add-feature Step 7, /execute-milestones Step 2.6, and /generate-app Step 5:
- Playwright is set up (
test:e2escript exists) — set it up via this skill if missing. - Every implemented feature has
e2e/[feature].spec.tscovering its acceptance criteria. - The app-shell smoke spec still passes.
npm test,npm run lint,npm run typecheck, andnpm run test:e2eall pass.- No test was skipped or deleted without a written reason.