What "Done" Actually Means

By Charlin Joe · 25 July 20260 views
What "Done" Actually Means

What "Done" Actually Means

The developer pasted the last test result into the team channel. All green. 83% coverage. PR open against main.

"Done," they typed.

They were not done.


"Done" in most AI-assisted workflows means: the AI finished generating, the tests pass, and the code is on a branch. That's three things, and they're all about the code. None of them are about the feature.

The feature might be done. Or it might have the wrong button size, a missing confirmation email, and a GDPR deletion handler that handles users but not guests. The test suite confirmed none of those things, because the test suite doesn't know what you agreed to build.

The requirements file does.

This is Step 6 of the ANN SDLC — the human verification step, the second gate, the one that closes the gap between "the tests pass" and "the feature is correct."


Here's what Step 6 looks like in practice.

The developer sits down with the requirements file. Not the code. Not the diff. The requirements file. They go through every acceptance criterion one at a time and answer a single question for each one: can I verify this is true right now?

For guest checkout, the first criterion is: "Continue as guest" is displayed at the same visual weight as "Sign in."

The developer opens the app. They look at the checkout screen. They see that "Sign in" is an ElevatedButton — full width, filled background — and "Continue as guest" is a TextButton underneath it. Different widget types. Different visual weight. The criterion fails.

This took thirty seconds to find. It would have taken thirty seconds to find at any point during development. It would also have taken thirty seconds to find in the Gate 3 review, or in QA, or when the product owner saw the screen for the first time. The difference is what it costs to fix it at each of those points.

At Step 6, the fix takes twenty minutes. The PR hasn't opened yet. The reviewer has seen nothing yet. The schedule hasn't slipped yet.

After PR merge, the fix costs a new PR, a new review cycle, and an explanation for why the initial implementation was wrong. After production deployment, it costs a hotfix deployment and a client conversation. The discovery itself is free. The timing of the discovery is expensive.


The second thing Step 6 catches is the ADR gap.

After checking each requirement, the developer goes through the ADR consequences. Not the full ADR — just the consequences section, which lists every constraint the implementation must satisfy because of the architectural decision.

ADR-012 has three consequences:

  1. All order queries must handle userId == null explicitly.
  2. Index on guestEmail required for returns lookup by email.
  3. GDPR deletion requests must include guest order cleanup by email, not uid.

The developer checks each one against the implementation.

Consequence one: the query filter uses where('userId', isEqualTo: uid), which excludes null values by definition. Passes.

Consequence two: the Firestore index configuration includes guestEmail. Passes.

Consequence three: the GDPR deletion Cloud Function was updated to handle account deletion. The guest order cleanup by email is not there. The developer searches the codebase. It's nowhere.

The GDPR deletion handler for guest orders by email was never implemented. The AI built the user account deletion path — it was explicitly in the requirements — but the guest cleanup path was in the ADR consequences, not the requirements. It's a gap between two documents that nobody reconciled before generation ran.

This gap, discovered at Step 6, is a half-day fix: implement the email-based batch deletion, add a test, update the ADR to note that the consequence was implemented. Done before the PR opens.

Discovered after merge: a production bug. Guest orders persist through GDPR deletion requests. That's not a half-day fix. That's a compliance incident.


Step 6 is where the AI's structural blind spot gets caught.

The AI generates code from a context window. The context window contains the requirements, the ADR, and the implementation plan. What the context window does not contain is institutional knowledge — the understanding of what else in the codebase depends on the assumptions the AI is making.

The human doing Step 6 has that institutional knowledge. Or they don't, and they know who to ask.

For guest checkout, the human doing Step 6 knows — because they wrote some of the original order service — that there's a nightly reconciliation job that generates a financial report. That job groups by userId. Guest orders have userId: null. The developer checks whether the report handles null correctly. It doesn't explicitly — it relies on Firestore's sorting, which puts null values in an implementation-specific position. This isn't a show-stopper, but it's something to mention in the Gate 3 review notes so the senior engineer knows to look at it.

This is the texture of Step 6: it's not a checklist run by someone who wasn't involved. It's a judgment pass by someone who understands both the requirements and the codebase. It surfaces the gaps between what the AI was told to do and what the system actually needs.


There's a version of Step 6 that most teams try to skip: they let CI do it.

CI runs the test suite. The test suite passes. Therefore the feature is correct. Ship it.

The problem is that CI verifies the code against the tests, not against the requirements. If the test doesn't check that both buttons have equal visual weight, CI can't catch the visual weight mismatch. If the test doesn't check that GDPR deletion handles guest users by email, CI can't catch the missing handler. The test suite is only as good as the tests that were written — and the tests that were written reflect what the developer thought to test, not what the requirements say is necessary.

An 83% coverage gate is not a requirement verification. It's a coverage verification. Those are different things.

The correct relationship between CI and Step 6: CI is a necessary precondition. If CI is failing, Step 6 doesn't happen yet. If CI is passing, Step 6 is the next step — not the final step, the next step. The PR doesn't open until both are done.


"Done" in the ANN SDLC means something specific.

It means: the AI generated the implementation, CI is green, the developer verified every acceptance criterion against the running application, the developer verified every ADR consequence against the implementation, and the PR is ready for Gate 3 review.

Not "the tests pass." Not "it looks good." Done. Verifiably done — because there's a document that says what done looks like, and someone checked the implementation against it before calling it done.

The thirty seconds it takes to find the wrong button is the same at every stage. The cost of finding it is not.


Comments

No comments yet. Be the first!

Sign in to leave a comment.