Overview
AI programming agents can produce plausible code without evidence that it works. This series follows executable specifications from evaluation through training, candidate selection, local runtime checks, and human-centered review, showing how prompts express intent while tests expose selected behavior. The evidence supports bounded claims, not universal reliability: every contract has coverage, environment, and incentive blind spots.
Learning outcomes
- Prompts express intent, but execution exposes selected program behavior.
- A test supports only the claim defined by its inputs, environment, and decision rule.
- Training and inference reuse execution signals, but weak or toxic tests can teach the wrong behavior.
- Reliable coding agents need feedback at consequential decisions and human review for unresolved trade-offs.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
Why are longer prompts not enough for AI programming agents?
Longer prompts can clarify intent, but they cannot demonstrate that a program works; execution-backed checks expose selected behavior under defined conditions.
What does an executable specification show?
It shows how a candidate behaves on selected inputs, workloads, or runtime observations under a defined environment, not how it behaves on every possible execution.
How do executable tests affect coding-agent training and inference?
In training, tests can select trajectories or become rewards; in inference, they can repair a candidate or help choose among sampled patches. Their blind spots still shape what the system learns or selects.
When does a coding agent need local checks or human review?
Local checks matter when an intermediate decision can contaminate later work or a final test cannot localize failure; human review remains necessary for context-dependent scope, communication, and trade-offs.