From af5579b24b39b0cca56b472e02efe07d3d6ebbd7 Mon Sep 17 00:00:00 2001 From: Dayowe Date: Mon, 14 Sep 2026 10:34:24 +0200 Subject: [PATCH] Streamline evidence verification and validation setup; bump plugin to 0.1.4 --- .../.codex-plugin/plugin.json | 2 +- .../skills/implementer/SKILL.md | 3 ++- .../skills/orchestrator/SKILL.md | 14 ++++++++------ .../staged-implementation/skills/planner/SKILL.md | 4 ++-- .../skills/validator/SKILL.md | 7 ++++--- 5 files changed, 17 insertions(+), 13 deletions(-) diff --git a/plugins/staged-implementation/.codex-plugin/plugin.json b/plugins/staged-implementation/.codex-plugin/plugin.json index f568a77..5fbc8e5 100644 --- a/plugins/staged-implementation/.codex-plugin/plugin.json +++ b/plugins/staged-implementation/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "staged-implementation", - "version": "0.1.3", + "version": "0.1.4", "description": "Planner, implementer, validator, and orchestrator skills for staged implementation workflows.", "author": { "name": "Local developer" diff --git a/plugins/staged-implementation/skills/implementer/SKILL.md b/plugins/staged-implementation/skills/implementer/SKILL.md index c59b130..2b23fd3 100644 --- a/plugins/staged-implementation/skills/implementer/SKILL.md +++ b/plugins/staged-implementation/skills/implementer/SKILL.md @@ -54,6 +54,7 @@ The prompt should define scope, non-goals, requirements, invariants, validation, - Never revert unrelated dirty changes. 5. Validate. + - Before expensive checks, preflight working directory/paths, tool versions, generated-input prerequisites and relevant file-type/symlink handling; reuse verified setup and recheck changed or uncertain assumptions. Derive intended assertions from frozen contracts and use source for implementation facts; investigate disagreement rather than making tests mirror a defect. Reuse applicable harness/verification helpers and candidate input records; create only the smallest missing helper when needed. Preflight does not replace behavioral tests. - Run the validation commands requested in the prompt when feasible. - Run targeted additional checks only when they directly reduce risk for the edited surface. - For corrections, identify affected behavior/consumers and rerun affected checks. Reuse evidence only when the assignment permits it and relevant inputs still match; retain original limits. Rerun if applicability is uncertain. Never skip a required fresh check or broaden into unrelated matrices/harnesses. @@ -70,7 +71,7 @@ The prompt should define scope, non-goals, requirements, invariants, validation, 7. Report results. - Summarize changed files and behavior. - - Distinguish fresh validation from verified reused evidence; give results/artifact paths instead of repeating full logs. Retain raw evidence and inspect failures/unexpected output. + - Distinguish fresh validation from verified reused evidence; give concise differences, counts, failures, skipped cases/limits and artifact paths instead of repeating unchanged inventories or full logs. Retain raw evidence and prior candidate provenance; inspect failures/unexpected output. Briefly explain recurring setup failures or repeated checks in this report. Apply process improvements prospectively without repackaging historical evidence. - Hand off owned temporary paths, active processes, retained evidence/recovery needs and disposable candidates to the parent. Do not remove a build/check-out needed by validation or a later gate merely because implementation ended. - List blockers, ambiguities, or residual risk. - Include the requested contract verification matrix for contract-heavy chunks. diff --git a/plugins/staged-implementation/skills/orchestrator/SKILL.md b/plugins/staged-implementation/skills/orchestrator/SKILL.md index b5d5477..b591fdc 100644 --- a/plugins/staged-implementation/skills/orchestrator/SKILL.md +++ b/plugins/staged-implementation/skills/orchestrator/SKILL.md @@ -98,7 +98,7 @@ For each authorized ready chunk: 8. Accept the chunk. - Confirm required validation passed under the current recorded acceptance contract; handle explicit user-approved exceptions under **Commit Rules**, never as fabricated PASS evidence. - - Recheck that the candidate source/diff and relevant build still match the reviewed/validated identity; changes invalidate affected evidence until reconciled and revalidated. + - Establish that candidate source/diff and relevant build still match the reviewed/validated identity, reusing verified continuity under the provenance rule below where applicable; changes invalidate affected evidence until reconciled and revalidated. - Confirm no out-of-scope work remains. - Apply the explicit commit policy. - Use the chunk's proposed commit message when acceptable; otherwise write a one-line commit message with the chunk ID prefix when one exists. @@ -212,17 +212,19 @@ When invoking a validator pass: - do not ask the validator to edit production code or commit - review the validator's evidence before accepting the chunk -Use lightweight provenance sufficient to identify what was tested, such as a scoped diff/input digest and build identifier; do not require exhaustive repository/dependency hashing by default. If relevant inputs change during a pass, stop affected validation, preserve the old evidence with its limits, and establish the corrected candidate before revalidation. Non-overlapping work is safe only when it cannot alter those inputs, target or results. +Use lightweight provenance sufficient to identify what was tested, such as a scoped diff/input digest and build identifier; do not require exhaustive repository/dependency hashing by default. Establish identity at validation entry. Verification can cover adjacent reporting, acceptance and commit boundaries when controlled ownership and applicable change checks demonstrate unchanged source, relevant inputs and build; a prior agent's claim alone is insufficient. Do not automatically rescan full inventories at every boundary. Mutation, interruption that makes continuity uncertain, or other identity uncertainty requires renewed verification. If relevant inputs change during a pass, stop affected validation, preserve the old evidence with its limits, and establish the corrected candidate before revalidation. Non-overlapping work is safe only when it cannot alter those inputs, target or results. ## Review and Validation Effort Keep roles distinct: implementer self-audit; parent actual-diff/integration/contract review; validator independent checks of assigned behavior/risks. Do not automatically add a general reviewer for each correction. Reuse a validator for bounded corrections while context remains valid; a changed mechanism, disputed finding, new risk or explicit gate may require a fresh challenge. All mandated independent reviews remain required. +Reuse reliable verification helpers and a canonical candidate-scoped input record where useful; verify their coverage/applicability rather than rebuilding them per worker. Keep prior candidate provenance intact when inputs change, and keep concurrent candidates separate. Create only the smallest missing helper when justified, not a mandatory evidence framework. Before expensive checks, preflight relevant paths, tool versions, generated prerequisites and file-type/symlink handling. Recheck changed or uncertain setup; preflight does not replace behavioral assertions. Validators independently verify inputs and expected outcomes against the frozen contract, not an unexamined helper or previous verdict. + For corrections, put a short impact note in the existing prompt/review: changed behavior/files, affected consumers, reruns and proposed evidence reuse. Reuse only after verifying relevant source, harness, dependencies, build configuration and environment match, preserving original limits. A commit ID alone is insufficient. Rerun if applicability is uncertain; shared owners/styles may affect many consumers. Required fresh checks cannot be skipped. Use automation for repeatable regression/mechanical checks and browser inspection for relevant visual/focus behavior and discrepancies. Do not automatically repeat the full matrix with every tool. Retain each required case at its assigned evidence layer/gate. Matching builds/evidence satisfy a requirement only when it permits reuse; later platform/device checks remain pending until observed. Never silently rewrite acceptance to avoid a blocker. -Report concise results, failures and evidence paths; retain raw logs/artifacts and inspect unexpected output. Link evidence instead of duplicating logs/galleries across reviews. Efficiency does not authorize model changes, missed cases, weaker contracts or unsafe rollback. +Report concise differences, counts, failures, skipped/not-tested obligations, evidence limits and artifact paths; retain raw logs/artifacts and inspect unexpected output. Link any existing canonical input record and detailed results instead of regenerating/reproducing unchanged inventories, hashes or packages. Apply process improvements prospectively; do not reorganize historical evidence merely to match a new convention. Briefly record causes of recurring setup failures or reasons for repeated checks in the existing report, not a new tracking system. Efficiency does not authorize model changes, missed cases, weaker contracts or unsafe rollback. ## Persistent Workspaces @@ -252,7 +254,7 @@ Before releasing an evidence-bearing workspace, preserve the required artifacts Use the run's bounded cleanup scope, carried into each assignment. Check each exact path is run-owned and within that scope, with no active process/validator, retained artifact or pending consumer depending on it. Release owned processes/handles before removal, respecting pauses and operational authority. Do not traverse symlinks/mounts into unrelated locations, clear `/tmp` broadly, delete by wildcard/prefix, or prune shared caches, files predating the run, user files or resources belonging to other runs. Worker resources may be cleaned by the parent after a recorded ownership handoff, subject to the same authorization and release checks; resources still owned by another worker remain protected. Use the owning tool's safe lifecycle for managed resources such as Git worktrees, without forced removal of uncommitted work. If ownership, retention or authority is unclear, keep the resource and ask with concrete paths/reasons. -Perform eligible cleanup at the lifecycle boundaries above, including run completion, rather than waiting for disk pressure. Record removed paths, retained paths with reasons/revisit conditions, and resulting headroom after substantial cleanup. If cleanup is blocked or a pause does not permit it, preserve the inventory for handoff; do not silently forget resources or relax acceptance to recover space. Cleanup beyond the run-owned scope requires separate authority. +Perform eligible cleanup at the lifecycle boundaries above, including run completion, rather than waiting for disk pressure. After accepted integration, explicitly resolve redundant candidate/build copies: verify persistent source/evidence preservation, rollback needs and consumer release, then remove eligible copies or record a specific retention reason and release condition. An evidence archive alone does not establish disposability; uncommitted work remains protected. Record removed paths, retained paths with reasons/revisit conditions, and resulting headroom after substantial cleanup. If cleanup is blocked or a pause does not permit it, preserve the inventory for handoff; do not silently forget resources or relax acceptance to recover space. Cleanup beyond the run-owned scope requires separate authority. ## Review Rules @@ -316,8 +318,8 @@ If the user explicitly approves a validation exception, first record its exact s Before committing: -- inspect the diff -- recheck the exact changes to be committed and relevant build against the accepted candidate identity; exclude held/unrelated work and resolve any affected evidence mismatch before committing +- inspect the exact staged diff against the accepted scope, including the absence of held/unrelated work +- establish that staged content and relevant build match the accepted candidate identity; verified continuity may cover adjacent boundaries without another full inventory/hash scan or test run, but never replaces staged-diff inspection. Renew verification when identity is uncertain and resolve affected evidence mismatches before committing - confirm the reviewed chunk is acceptable - confirm required validation passed under the current recorded acceptance contract - confirm no unrelated changes are included diff --git a/plugins/staged-implementation/skills/planner/SKILL.md b/plugins/staged-implementation/skills/planner/SKILL.md index c9b04a6..f1bbad8 100644 --- a/plugins/staged-implementation/skills/planner/SKILL.md +++ b/plugins/staged-implementation/skills/planner/SKILL.md @@ -122,9 +122,9 @@ Give each artifact one job: plan = architecture/rationale; checklist = work/depe Design validation alongside chunk boundaries. Assign each obligation to the first gate that needs it: local implementation, integrated behavior, or actual platform/device/release. Record its target and prerequisites. Do not require later release evidence before a local chunk unless correctness or safe activation depends on it. An unavailable mandatory check remains pending at its assigned gate; emulation/local success cannot pass that gate. -Prefer existing tests/harnesses and the smallest validation surface that credibly proves the contract. Expand for shared-owner impact or newly found risk. Do not prescribe every appearance × viewport × state combination, another harness or another general review without a coverage need. Retain required integration/independent review and any explicitly mandated matrix or fresh run unless expressly amended. +Prefer existing tests/harnesses and the smallest validation surface that credibly proves the contract. Plan a cheap setup preflight before expensive validation: working directory/paths, tool versions, generated-input prerequisites and relevant file-type/symlink handling; subsequently recheck changed or uncertain assumptions. Intended test outcomes come from frozen contracts; source establishes implementation facts, not permission to copy a defect into the expected result. Investigate disagreement. Expand for shared-owner impact or newly found risk. Do not prescribe every appearance × viewport × state combination, another harness or another general review without a coverage need. Retain required integration/independent review and any explicitly mandated matrix or fresh run unless expressly amended. -Specify evidence reuse conditions: relevant source, harness, dependencies, configuration and environment must match, retaining original limits. Changed inputs require affected checks; uncertain applicability requires a rerun. A correction need not repeat unrelated builds/screenshots/reviews, but efficiency cannot weaken an acceptance case. +Specify evidence reuse conditions: relevant source, harness, dependencies, configuration and environment must match, retaining original limits. Prefer one applicable verification helper and canonical input record per candidate over duplicate evidence packages; neither a helper nor a manifest is mandatory for a small chunk. Preserve prior candidate records when inputs change. Require trustworthy identity at handoff/reporting/acceptance/commit, not a full scan at each boundary: verified continuity can cover adjacent boundaries, while mutation or uncertainty requires renewed verification. Independent validators still verify scope/applicability and assigned behavior. A correction need not repeat unrelated builds/screenshots/reviews, but efficiency cannot weaken an acceptance case. Plan persistent implementation storage from the first edit, including backing Git metadata. Use the existing checkout for safe sequential work; when isolation is needed, prefer the project's established persistent worktree location and reuse the same workspace for same-chunk corrections. If no suitable convention or authorized location exists, propose one dedicated location and ask once before creation; never automatically create sibling directories or fall back to `/tmp`. For nested worktrees, require the paths to be ignored and untracked in the containing repository and protected from broad cleanup. Record the location in the handoff. Required evidence/recovery artifacts also need persistent destinations from creation; a pause-time copy or a transcript is not a durability strategy. This does not authorize premature commits or change acceptance gates. diff --git a/plugins/staged-implementation/skills/validator/SKILL.md b/plugins/staged-implementation/skills/validator/SKILL.md index 6f77088..4a87f9e 100644 --- a/plugins/staged-implementation/skills/validator/SKILL.md +++ b/plugins/staged-implementation/skills/validator/SKILL.md @@ -53,13 +53,14 @@ Follow the assigned acceptance level: local implementation, integration, or actu - Read project instructions first. - Read the assigned requirements, relevant plan/checklist sections, prompt, findings and changed-surface context; expand when dependencies or applicability are unclear, not to repeat unrelated planning audits. - Identify the exact behavior and contracts to validate. - - Verify the candidate's baseline/scoped changes, relevant test/build inputs and build/artifact provenance. Confirm writes to that scope and replacement of the target are held for the pass; do not assume a URL or HEAD alone identifies what is being tested. + - Independently verify the candidate's baseline/scoped changes, relevant test/build inputs and build/artifact provenance. Reuse applicable verification helpers and a canonical candidate input record without trusting their scope or a previous verdict unexamined; do not rebuild evidence machinery per worker. Confirm writes to that scope and replacement of the target are held for the pass; do not assume a URL or HEAD alone identifies what is being tested. 2. Create a validation plan. - List the validation items. - Separate happy paths, negative paths, edge cases, and contract checks. - Identify tools to use: build/test commands, API calls, browser tools, logs, screenshots, device/runtime checks. - State any preconditions, such as running server, seeded data, credentials, hardware, or env vars. + - Before expensive execution, cheaply verify working directory/paths, tool versions, generated-input prerequisites and relevant file-type/symlink handling; recheck changed or uncertain setup thereafter. Intended outcomes come from frozen contracts; source establishes implementation facts, not an oracle that may copy defects into assertions. Investigate disagreement. Reuse established harnesses and add only the smallest missing helper when justified; preflight is not behavioral validation. - Verify any disposable build/test checkout has a complete persistent source candidate, including uncommitted/new files and required local inputs. Put required evidence/recovery artifacts in persistent destinations as produced, not only when reporting; `/tmp` is for reproducible scratch. Do not edit the candidate to repair a storage problem; report it to the parent. After interrupted-work recovery, verify actual candidate identity and rerun affected checks; surviving transcripts or Git metadata do not establish complete recovery. - Record owned temporary paths/profiles and their consumers as created. Check target-filesystem headroom before large build/browser installs, copies or captures; hold unsafe allocations. Reuse compatible resources without changing the frozen candidate or breaking isolation. - Receive cleanup scope/retention and allocation restrictions explicitly. Coordinate large starts with the parent, account for combined remaining peak usage per filesystem, and recheck after substantial allocations. Defaults unless explicitly overridden: below 5 GiB free report/serialize large allocations; 2 GiB or less pause write-heavy work and report. Keep projected free space above the critical reserve; without a parent, account for known competing allocations and serialize when uncertain. @@ -74,13 +75,13 @@ Follow the assigned acceptance level: local implementation, integration, or actu 4. Capture evidence. - Record commands or tools used. - - Summarize results/failures and artifact paths; retain full logs and inspect unexpected output. Distinguish newly run checks from verified reused evidence with its original limitations. + - Summarize differences, counts, failures, skipped/not-tested cases, limits and artifact paths; retain full logs and inspect unexpected output. Link applicable canonical input records rather than regenerating/reproducing unchanged inventories. Preserve prior candidate provenance and distinguish newly run checks from verified reused evidence. Briefly explain recurring setup failures or repeated checks in the existing report; apply improvements prospectively without repackaging historical evidence. - Capture screenshots only when they support the verdict. - Note relevant console errors, network failures, logs, or API responses. - Do not print secrets, tokens, passwords, or sensitive env values. 5. Report verdict. - - Recheck candidate identity before reporting. If relevant inputs changed during validation, stop the affected checks, notify the parent and qualify old evidence; do not claim it validates the changed candidate. Resume affected validation only after a stable candidate is established. + - Establish candidate identity remains valid before reporting. Entry verification may cover this adjacent boundary when controlled ownership and applicable change checks demonstrate unchanged relevant source/inputs/build; do not automatically repeat full scans. Mutation, interruption that makes continuity uncertain or other identity uncertainty requires renewed verification. If relevant inputs changed, stop affected checks, notify the parent and qualify old evidence; resume only after a stable candidate is established. Prior-agent assurances alone do not establish continuity. - Mark each validation item as pass, fail, blocked, or not tested. - Explain failures with concrete observed behavior. - State residual risk and untested areas.