Skip to main content

Module 5: Evidence & Validation

Module 4 walked the loop from classification through a draft PR. This module is about the step most teams skip when they build their own agent workflows: making the agent prove its work before anyone reviews it.

The seed infographics call this "absence of proof is failure." That phrase is vivid, but it maps to something concrete: for web-UI projects, ADW's Phase B validation step (generalized from the Mars project's /spec-validate pattern) automatically produces evidence artifacts — a chaptered browser walkthrough, before/after screenshots, and a written verdict file named RESULTS.md. If that evidence is missing, the work does not get promoted toward a mergeable state, full stop.

How AI pipelines catch hallucinations (12 min)

Why evidence, not just "it compiled"​

A maker agent (the one that wrote the implementation) is a poor judge of its own work — it's the same model, the same context, the same blind spots that produced the code in the first place. That's why ADW puts an independent adversarial verifier between the maker and the commit step: the maker–checker gate covered in Module 4. Evidence generation is the next layer on top of that: instead of trusting a verifier's opinion alone, the pipeline captures artifacts a human can actually look at.

Grounding this claim

This is real mechanics, not illustrative framing: the Playwright validator and RESULTS.md verdict pattern originated in the Mars work-order system's /spec-validate command and has been generalized for reuse across web-UI Wasteology projects. The always-on gate underneath all of this is still the draft PR + human merge — evidence generation doesn't replace human review, it makes that review faster and better-informed.

What a validation pass produces​

For a web-UI project, an automated validation pass typically produces:

  1. A chaptered walkthrough — a browser recording broken into labeled chapters, one per test scenario, so a reviewer can jump straight to the part that matters instead of watching a raw screen capture.
  2. Before/after screenshots — paired captures showing the state of the UI before the change and after, per scenario.
  3. RESULTS.md — a written verdict: which scenarios passed, which failed, and why. This is the artifact the promotion gate actually checks for.

Wastey says

Notice what's not in that diagram: an auto-merge arrow. Evidence makes the human reviewer's job easier — it never replaces the human reviewer. The draft PR and the merge button are still where a person signs off.

The "absence of proof is failure" rule, precisely​

The ADW safety gate process
The ADW safety gate process

Read the infographic's headline claim carefully and pair it with the real rule: if a project is configured for automated validation and the expected evidence artifact (a RESULTS.md file, or the recording it references) is absent or incomplete, the work is not promoted. It stays exactly where it is — implemented, but not advanced toward merge — until either the evidence is produced or a human intervenes directly. There is no path where missing evidence is quietly waved through.

This matters because it closes a common failure mode in less careful agentic setups: an agent that claims success in its own commit message or chat output, with nothing independently checkable behind the claim. ADW's evidence gate forces the claim to come with a receipt.

Illustrative vs. mechanical

Some seed material frames this with dramatic language ("neuro-symbolic", "deterministic sandbox physically unreachable from prod"). Treat phrasing like that as illustrative color. What you can rely on precisely: draft-PR + human merge (always on), the independent maker-checker verifier (Module 4), and — for web-UI projects — the Playwright-generated RESULTS.md evidence gate described above.

What a good RESULTS.md looks like​

A RESULTS.md verdict is short, scenario-oriented, and honest about failures. A reasonable shape:

# Validation Results — <feature/change name>

## Scenarios

| # | Scenario | Result | Notes |
|---|----------|--------|-------|
| 1 | User logs in with valid credentials | PASS | Screenshot: 01-before.png / 01-after.png |
| 2 | User logs in with expired token | PASS | Redirects to re-auth, chapter 2 in walkthrough |
| 3 | Dashboard loads under 2s | FAIL | Observed 3.4s — see chapter 3 |

## Verdict

2/3 scenarios pass. Scenario 3 needs follow-up before this is safe to merge.

## Artifacts

- Chaptered walkthrough: `walkthrough.mp4`
- Screenshots: `screenshots/`

A verdict that reports a failure honestly is more valuable than one that glosses over it — the whole point of the gate is that a human can trust what it says.

Hands-on lab​

Produce your own evidence package for a small change (real or practice) in a project you can run locally:

  1. Pick a change with a visible effect in a UI (or, if you don't have a web UI handy, a CLI command whose before/after output you can capture).
  2. Run the change through the Module 4 loop far enough to have an implementation to validate — it does not need to be a full feature.
  3. Manually reconstruct what the automated validator would produce:
    • Capture a "before" screenshot or terminal output.
    • Capture an "after" screenshot or terminal output.
    • Write a RESULTS.md file following the shape above, with a real scenario table and an honest verdict (a partial pass is a fine outcome for this exercise).
  4. Save your RESULTS.md and screenshots together in one folder, as if you were handing it to a reviewer who has never seen the change.
  5. Re-read your own RESULTS.md as if you were the reviewer. Would you merge it? What's missing?

If your project already has Playwright configured, you can instead trigger a real validation pass and inspect the RESULTS.md it produces — compare it against the shape above.

Explore the evidence package​

What a validation pass producesInteractive

Click any step to see what happens and who's responsible.

▸Why is absence of proof treated as failure, instead of trusting the maker agent's own claim of success?

Knowledge check​

What does the "absence of proof is failure" rule actually mean in ADW?
For web-UI projects, what does the Playwright validation pass produce?
Does evidence generation replace the human merge gate?