Compare commits
1 Commits
phase-file
...
e2e-verifi
| Author | SHA1 | Date | |
|---|---|---|---|
| 1b67d1f15e |
@ -53,6 +53,7 @@ Verify that the implemented code meets the following criteria:
|
||||
- [ ] Is the coverage target met?
|
||||
- [ ] Are edge cases tested?
|
||||
- [ ] Are the tests named appropriately?
|
||||
- [ ] Are there Playwright E2E tests covering the feature's acceptance criteria (e2e/*.spec.ts)?
|
||||
|
||||
**Evaluation Criteria**:
|
||||
- ✅ Sufficient: Coverage of 80% or more, covering the main cases
|
||||
@ -213,6 +214,11 @@ npm test
|
||||
npm run test:coverage
|
||||
```
|
||||
|
||||
### Run E2E Tests (Playwright)
|
||||
```bash
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
### Build Check
|
||||
```bash
|
||||
npm run build
|
||||
|
||||
@ -121,15 +121,19 @@ If any of the following situations occur while the implementation loop is runnin
|
||||
|
||||
**Once this step completes successfully, never stop; immediately proceed to Step 7.**
|
||||
|
||||
## Step 7: Run Automated Tests
|
||||
## Step 7: Run Automated Tests (including Playwright E2E)
|
||||
|
||||
1. Run the following commands in order and confirm that all tests pass.
|
||||
1. Write or update the feature's end-to-end tests following the **e2e-testing skill** (`Skill('e2e-testing')`):
|
||||
- Create/update `e2e/[feature-name].spec.ts` covering the feature's acceptance criteria.
|
||||
- If Playwright is not yet set up in the repository, set it up first as described in the skill.
|
||||
2. Run the following commands in order and confirm that all of them pass.
|
||||
```bash
|
||||
Bash('npm test')
|
||||
Bash('npm run lint')
|
||||
Bash('npm run typecheck')
|
||||
Bash('npm run test:e2e')
|
||||
```
|
||||
2. If any command produces an error, analyze the problem, generate and apply a fix, and then run this step again.
|
||||
3. If any command produces an error, analyze the problem, generate and apply a fix, and then run this step again. Never skip or delete a failing test to make verification pass.
|
||||
|
||||
**Once this step completes successfully, never stop; immediately proceed to Step 8.**
|
||||
|
||||
@ -164,7 +168,7 @@ If any of the following situations occur while the implementation loop is runnin
|
||||
This workflow completes automatically once all of the following conditions are met.
|
||||
- Step 5: All tasks in `tasklist.md` are complete (`[x]` or skipped for a valid reason).
|
||||
- Step 6: The `implementation-validator` subagent's validation passes.
|
||||
- Step 7: The `test`, `lint`, and `typecheck` commands all succeed without errors.
|
||||
- Step 7: The `test`, `lint`, `typecheck`, and `test:e2e` (Playwright) commands all succeed without errors, and the feature has E2E tests covering its acceptance criteria.
|
||||
- Step 8: Handover notes are recorded in `tasklist.md`, all six persistent documents in `docs/` have been reconciled with the feature (updates applied, or a skip reason noted for each), and the feature is checked off in its phase file (if `docs/milestones/` exists).
|
||||
|
||||
Until these completion criteria are met, continue to think autonomously, solve problems, and carry on the work.
|
||||
|
||||
@ -103,15 +103,19 @@ Use the `Task` tool to launch the `implementation-validator` subagent:
|
||||
- `description`: "Milestone implementation validation"
|
||||
- `prompt`: "Please validate the quality of all changes implemented for milestone `[NN] [name]`. The target files are `[list of implemented file paths]`. Focus on coding standards, error handling, testability, consistency with existing patterns, and whether every feature's acceptance criteria in `.steering/[dir]/requirements.md` are met."
|
||||
|
||||
#### 2.6 Run Automated Tests
|
||||
#### 2.6 Run Automated Tests (including Playwright E2E)
|
||||
|
||||
1. Write or update the milestone's end-to-end tests following the **e2e-testing skill** (`Skill('e2e-testing')`): one `e2e/[feature-name].spec.ts` per feature in the milestone, covering its acceptance criteria. Set up Playwright first (as described in the skill) if it is missing.
|
||||
2. Run:
|
||||
|
||||
```bash
|
||||
Bash('npm test')
|
||||
Bash('npm run lint')
|
||||
Bash('npm run typecheck')
|
||||
Bash('npm run test:e2e')
|
||||
```
|
||||
|
||||
If any command fails, analyze, fix, and re-run until all pass.
|
||||
If any command fails, analyze, fix, and re-run until all pass. Never skip or delete a failing test to make verification pass.
|
||||
|
||||
#### 2.7 Retrospective and Document Reconciliation
|
||||
|
||||
@ -124,7 +128,7 @@ Apply `/add-feature` Step 8 for the milestone:
|
||||
|
||||
#### 2.8 Milestone Gate — then Continue
|
||||
|
||||
The milestone is done only when: all tasks are `[x]`, validation passed, tests/lint/typecheck are green, the six documents are reconciled, and the milestone status is updated in the phase file and the roadmap.
|
||||
The milestone is done only when: all tasks are `[x]`, validation passed, tests/lint/typecheck/E2E (Playwright) are green, the six documents are reconciled, and the milestone status is updated in the phase file and the roadmap.
|
||||
|
||||
**Once the gate passes, never stop; immediately start Step 2 for the next milestone in the queue.** If the gate genuinely cannot be passed (external blocker), record the blocker in the milestone's `tasklist.md`, the phase file, and `roadmap.md`, then stop and report — this is the only permitted early stop.
|
||||
|
||||
@ -138,7 +142,7 @@ The milestone is done only when: all tasks are `[x]`, validation passed, tests/l
|
||||
- Every milestone in the specified phase file has status `Completed`.
|
||||
- The phase's status is `Completed` in both the phase file and `docs/milestones/roadmap.md`.
|
||||
- Each executed milestone has its own `.steering/[date]-milestone-[NN]-[name]/` directory with all tasks `[x]`.
|
||||
- Tests, lint, and typecheck pass on the final state.
|
||||
- Tests, lint, typecheck, and Playwright E2E tests pass on the final state, with E2E coverage for every implemented feature's acceptance criteria.
|
||||
- The six persistent documents reflect everything implemented.
|
||||
|
||||
Completion message:
|
||||
|
||||
@ -117,10 +117,13 @@ Apply the requirements for the selected target:
|
||||
|
||||
If new directories/files were created, update `docs/repository-structure.md` so it reflects the generated layout.
|
||||
|
||||
### Step 5: Verify
|
||||
### Step 5: Verify (including Playwright E2E)
|
||||
|
||||
- If the target is `web` and the repo has `npm test`, `npm run lint`, `npm run typecheck`, run them and fix any errors caused by the generated code.
|
||||
- For `flutter` / `winui3`, run the platform's available static checks if the toolchain is present; otherwise note what should be checked manually.
|
||||
- If the target is `web`:
|
||||
- Run `npm test`, `npm run lint`, `npm run typecheck` (where present) and fix any errors caused by the generated code.
|
||||
- Set up Playwright following the **e2e-testing skill** (`Skill('e2e-testing')`) if it is not set up yet, and create the app-shell smoke spec `e2e/app-shell.spec.ts` (shell renders, every blueprint route navigates, primary navigation works).
|
||||
- Run `npm run test:e2e` and fix any failures caused by the generated code.
|
||||
- For `flutter` / `winui3`, run the platform's available static checks and UI/integration tests if the toolchain is present (`integration_test` for Flutter, the platform's UI automation for WinUI 3); otherwise note what should be checked manually.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
@ -140,6 +143,7 @@ Generated:
|
||||
✅ Routes / navigation (from ui-blueprint.json)
|
||||
✅ Screens (one per blueprint screen)
|
||||
✅ Reusable components
|
||||
✅ Playwright E2E smoke spec (e2e/app-shell.spec.ts, web target)
|
||||
|
||||
Next steps:
|
||||
- Run the app and verify the shell renders
|
||||
|
||||
@ -12,6 +12,7 @@
|
||||
"Skill(platform-ui-generation)",
|
||||
"Skill(milestone-planning)",
|
||||
"Skill(milestone-execution)",
|
||||
"Skill(e2e-testing)",
|
||||
"Skill(steering)"
|
||||
],
|
||||
"deny": [],
|
||||
|
||||
@ -90,6 +90,7 @@ When creating development guidelines: ./template.md
|
||||
### When Designing Tests
|
||||
- "Test Strategy" in ./guides/process.md (pyramid, coverage)
|
||||
- "Test Code" in ./guides/implementation.md (implementation patterns)
|
||||
- E2E tests: the **e2e-testing skill** (Playwright — E2E is part of every verification step)
|
||||
|
||||
### When Preparing a Release
|
||||
- "Git Workflow Rules" in ./guides/process.md (policy for merging into main)
|
||||
|
||||
118
.claude/skills/e2e-testing/SKILL.md
Normal file
118
.claude/skills/e2e-testing/SKILL.md
Normal file
@ -0,0 +1,118 @@
|
||||
---
|
||||
name: e2e-testing
|
||||
description: Guide for writing and running Playwright end-to-end tests as part of every verification step. Use when verifying a feature, milestone, or generated app shell, or when setting up Playwright in the repository.
|
||||
allowed-tools: Read, Write, Edit, Bash
|
||||
---
|
||||
|
||||
# E2E Testing Skill (Playwright)
|
||||
|
||||
This skill defines how end-to-end tests are written and run in this project. **Every verification step includes E2E tests**: a feature, milestone, or app shell is not "verified" until its Playwright tests pass alongside unit tests, lint, and typecheck.
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **E2E is part of verification, not an extra**: the standard verification sequence is `npm test` → `npm run lint` → `npm run typecheck` → `npm run test:e2e`. All four must pass.
|
||||
2. **Tests are derived from acceptance criteria**: every feature's acceptance criteria (from its phase file or steering `requirements.md`) map to E2E assertions. If a criterion cannot be asserted in an E2E test, note why in the tasklist.
|
||||
3. **Blueprint-driven selectors**: screens, routes, and actions come from `docs/design/ui-blueprint.json`. Tests navigate by the blueprint's routes and interact via the blueprint's actions.
|
||||
4. **One spec file per feature** plus one smoke spec for the app shell.
|
||||
5. **Web-first**: Playwright covers the `web` target. For `flutter` use `integration_test`, for `winui3` use the platform's UI automation — apply the same acceptance-criteria mapping; note in the tasklist when Playwright does not apply.
|
||||
|
||||
## Setup (run once, on demand)
|
||||
|
||||
If Playwright is not yet set up in the repository (no `@playwright/test` in `package.json`), set it up before the first E2E run:
|
||||
|
||||
```bash
|
||||
npm install -D @playwright/test
|
||||
npx playwright install --with-deps chromium
|
||||
```
|
||||
|
||||
Create `playwright.config.ts`:
|
||||
|
||||
```typescript
|
||||
import { defineConfig } from '@playwright/test';
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './e2e',
|
||||
fullyParallel: true,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
reporter: [['list'], ['html', { open: 'never' }]],
|
||||
use: {
|
||||
baseURL: process.env.E2E_BASE_URL ?? 'http://localhost:3000',
|
||||
trace: 'retain-on-failure',
|
||||
screenshot: 'only-on-failure',
|
||||
},
|
||||
webServer: {
|
||||
command: 'npm run dev', // adjust to the project's start command
|
||||
url: 'http://localhost:3000', // adjust to the project's port
|
||||
reuseExistingServer: !process.env.CI,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Add the script to `package.json`:
|
||||
|
||||
```json
|
||||
"test:e2e": "playwright test"
|
||||
```
|
||||
|
||||
Adjust `baseURL`, `webServer.command`, and the port to the project's actual dev server (see `docs/architecture.md`). Add `e2e/` artifacts (`playwright-report/`, `test-results/`) to `.gitignore`.
|
||||
|
||||
## Test Layout
|
||||
|
||||
```
|
||||
e2e/
|
||||
├── app-shell.spec.ts # smoke: shell renders, every blueprint route navigates
|
||||
├── [feature-name].spec.ts # one spec per feature
|
||||
└── helpers/ # shared fixtures/utilities (only when duplicated 3+ times)
|
||||
```
|
||||
|
||||
## Writing Feature Specs
|
||||
|
||||
For each feature, create `e2e/[kebab-case-feature-name].spec.ts`:
|
||||
|
||||
- One `test.describe` block per feature; one `test` per acceptance criterion (or a tight group of criteria).
|
||||
- Navigate using the route of the affected screen from `ui-blueprint.json`.
|
||||
- Prefer semantic locators: `getByRole`, `getByLabel`, `getByText` — matching the blueprint's component types and labels. Avoid CSS/XPath selectors tied to implementation details.
|
||||
- No fixed waits (`waitForTimeout`); rely on Playwright's auto-waiting and explicit `expect(...).toBeVisible()` style assertions.
|
||||
- Keep tests independent: each test sets up its own state and does not depend on execution order.
|
||||
|
||||
Example:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Edit user profile', () => {
|
||||
test('saves a changed display name', async ({ page }) => {
|
||||
await page.goto('/profile');
|
||||
await page.getByLabel('Display name').fill('New Name');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
await expect(page.getByText('Profile updated')).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Writing the App-Shell Smoke Spec
|
||||
|
||||
Created by `/generate-app` (web target). `e2e/app-shell.spec.ts` must verify:
|
||||
|
||||
- The app starts and the entry screen renders.
|
||||
- Every route in `ui-blueprint.json` is reachable and renders its screen title.
|
||||
- Primary navigation (from the blueprint's nav component) works.
|
||||
|
||||
## Running
|
||||
|
||||
```bash
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
- If a test fails, analyze the trace/screenshot in `test-results/`, fix the code (or the test if the spec was wrong against the acceptance criteria), and re-run until green.
|
||||
- Never delete or skip a failing test to make verification pass; a skipped test requires a written reason in the tasklist, same as a struck task.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Used by `/add-feature` Step 7, `/execute-milestones` Step 2.6, and `/generate-app` Step 5:
|
||||
|
||||
- [ ] Playwright is set up (`test:e2e` script exists) — set it up via this skill if missing.
|
||||
- [ ] Every implemented feature has `e2e/[feature].spec.ts` covering its acceptance criteria.
|
||||
- [ ] The app-shell smoke spec still passes.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, and `npm run test:e2e` all pass.
|
||||
- [ ] No test was skipped or deleted without a written reason.
|
||||
@ -46,14 +46,15 @@ Tasks are **grouped by feature**, in the implementation order chosen in `design.
|
||||
- [ ] [Task 1: data/model layer]
|
||||
- [ ] [Task 2: logic/service layer]
|
||||
- [ ] [Task 3: UI per affected screen]
|
||||
- [ ] [Task 4: tests for the acceptance criteria]
|
||||
- [ ] [Task 4: unit tests + Playwright E2E spec for the acceptance criteria]
|
||||
|
||||
## Feature: Log in / log out
|
||||
- [ ] ...
|
||||
|
||||
## Milestone Verification
|
||||
- [ ] All acceptance criteria in requirements.md confirmed
|
||||
- [ ] npm test / lint / typecheck pass
|
||||
- [ ] E2E specs exist for every feature (e2e/[feature].spec.ts, per the e2e-testing skill)
|
||||
- [ ] npm test / lint / typecheck / test:e2e pass
|
||||
|
||||
## Post-Implementation Retrospective
|
||||
_Filled at the end: completion date, plan vs. actual, lessons learned._
|
||||
@ -62,8 +63,8 @@ _Filled at the end: completion date, plan vs. actual, lessons learned._
|
||||
## How to Divide a Feature into Tasks
|
||||
|
||||
- 3–8 tasks per feature; each completable in **one implementation-loop iteration**.
|
||||
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests.
|
||||
- Every feature's section ends with a task that verifies its acceptance criteria.
|
||||
- Slice along the natural layers the feature touches: data/model → logic/service → UI (one task per affected screen) → tests (unit + Playwright E2E).
|
||||
- Every feature's section ends with a task that verifies its acceptance criteria via the feature's E2E spec (see the e2e-testing skill).
|
||||
- Tasks that unblock several features go into a "Shared Foundations" section at the top — never duplicated per feature.
|
||||
|
||||
## Gate Criteria Between Milestones
|
||||
@ -72,7 +73,8 @@ A milestone passes its gate only when **all** of the following hold. Only then m
|
||||
|
||||
- [ ] Every task in the milestone's `tasklist.md` is `[x]` (or struck with a technical reason).
|
||||
- [ ] The `implementation-validator` subagent's validation passed.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck` all succeed.
|
||||
- [ ] Every feature of the milestone has a Playwright E2E spec covering its acceptance criteria.
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` all succeed.
|
||||
- [ ] The six persistent documents in `docs/` are reconciled with the milestone's changes.
|
||||
- [ ] In the phase file: every feature of the milestone is checked off, and the milestone status is `Completed` with Completion Notes.
|
||||
- [ ] `roadmap.md` points to the next milestone (or, after the phase's last milestone, the phase is `Completed` and the roadmap points to the next phase file).
|
||||
|
||||
@ -59,7 +59,8 @@ Use this structure for each `docs/milestones/phase[N]-milestones.md`. A phase fi
|
||||
### Definition of Done
|
||||
|
||||
- [ ] All features above are checked off
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck` pass
|
||||
- [ ] Every feature has a Playwright E2E spec covering its acceptance criteria
|
||||
- [ ] `npm test`, `npm run lint`, `npm run typecheck`, `npm run test:e2e` pass
|
||||
- [ ] Persistent documents in `docs/` are reconciled with all features
|
||||
- [ ] [Milestone-specific criterion, e.g. "demo flow X→Y→Z works end-to-end"]
|
||||
|
||||
|
||||
@ -59,6 +59,9 @@ When skipping, always state the reason clearly:
|
||||
- [ ] `npm run lint`
|
||||
- [ ] Confirm that there are no type errors
|
||||
- [ ] `npm run typecheck`
|
||||
- [ ] Confirm that the E2E tests pass (Playwright — see the e2e-testing skill)
|
||||
- [ ] E2E specs exist for this work's acceptance criteria
|
||||
- [ ] `npm run test:e2e`
|
||||
- [ ] Confirm that the build succeeds
|
||||
- [ ] `npm run build`
|
||||
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
1. **Document creation**: Define "what to build" in persistent documents (`docs/`)
|
||||
2. **Work planning**: Plan "what to do this time" in steering files (`.steering/`)
|
||||
3. **Implementation**: Implement according to tasklist.md and update progress as you go
|
||||
4. **Verification**: Testing and operation checks
|
||||
4. **Verification**: Testing and operation checks — unit tests, lint, typecheck, and Playwright E2E tests (see the e2e-testing skill)
|
||||
5. **Update**: Reconcile all `docs/` documents with the implemented feature (see `/add-feature` Step 8)
|
||||
|
||||
### Important Rules
|
||||
@ -103,7 +103,7 @@ Define "what to do this time" for a specific development task:
|
||||
|
||||
- `.claude/agents/` - Subagent definitions (doc-reviewer, implementation-validator)
|
||||
- `.claude/commands/` - Slash commands (setup-project, define-design, generate-app, update-design, plan-milestones, execute-milestones, add-feature, review-docs)
|
||||
- `.claude/skills/` - Task-specific skills (prd-writing, functional-design, architecture-design, repository-structure, development-guidelines, glossary-creation, ui-design, design-tokens, platform-ui-generation, milestone-planning, milestone-execution, steering)
|
||||
- `.claude/skills/` - Task-specific skills (prd-writing, functional-design, architecture-design, repository-structure, development-guidelines, glossary-creation, ui-design, design-tokens, platform-ui-generation, milestone-planning, milestone-execution, e2e-testing, steering)
|
||||
|
||||
## Development Process
|
||||
|
||||
|
||||
@ -68,7 +68,7 @@ A fully autonomous loop, one feature per run. Each run:
|
||||
2. Reads the feature's entry in its phase file and the design spec first — and updates the design first if the feature changes the UI
|
||||
3. Implements every task in `tasklist.md`, checking items off as it goes
|
||||
4. Validates quality with the `implementation-validator` subagent
|
||||
5. Runs `npm test`, `npm run lint`, `npm run typecheck` until green
|
||||
5. Writes Playwright E2E tests for the feature's acceptance criteria, then runs `npm test`, `npm run lint`, `npm run typecheck`, and `npm run test:e2e` until green
|
||||
6. Records a retrospective, reconciles all six `docs/` documents, and checks the feature off in its phase file
|
||||
|
||||
### Step 8 — Repeat and maintain
|
||||
@ -78,7 +78,8 @@ A fully autonomous loop, one feature per run. Each run:
|
||||
- Document quality checks → `/review-docs <path>`
|
||||
|
||||
**A feature is "done" when**: all tasks in its tasklist are `[x]`, validation passes,
|
||||
test/lint/typecheck succeed, and the persistent documents are back in sync with the code.
|
||||
test/lint/typecheck and the Playwright E2E suite succeed, and the persistent documents are back
|
||||
in sync with the code.
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user