The Problem AutoHarness Is Actually Solving
AutoHarness is a Claude Code plugin for turning lessons from real work into a maintained library of reusable skills. The important question is not simply whether an agent can remember an instruction, but whether that memory can be admitted safely, shown at the right moment, measured in use, and retired when it becomes clutter. That question matters to anyone who has corrected an agent once, watched it perform the same kind of task in a later session, and wondered whether the correction survived—or whether the library is quietly filling with rules nobody can find or trust.
The two ways memory fails
Consider a small repository convention. A developer asks an agent to add parameterized tests. The agent writes plausible tests, but omits readable ids for the cases. The developer corrects the omission and explains that failures should name behaviors rather than display raw tuples. The correction has value beyond that one file: it is a durable way of working in that repository.
Without a persistence mechanism, the lesson is trapped in the session transcript. A new session may repeat the omission, forcing the developer to teach the same convention again. But persistence alone creates the opposite problem. If every correction becomes a new note or skill, similar lessons accumulate side by side. The agent then has more material but a less reliable route through it. Near-duplicates compete for attention, old rules can contradict new ones, and an operator cannot easily tell which entries were learned, which were written by hand, or why any entry still exists.
AutoHarness is designed around the tension between forgetting and accumulation. Its README describes a layer that learns from sessions, compares a new lesson with existing skills, updates or merges related rules, and archives skills that stop earning their place. The desired result is not a larger memory surface. It is a smaller, more legible one that can change in response to work without taking control of everything around it.
That distinction gives the subject a useful scope. AutoHarness is not a new language model, and it does not replace Claude Code's native skill-recall mechanism. It is a maintenance layer beside the host. The skills it creates are ordinary, inspectable files in the host's skill system. Around those files, the plugin keeps operational metadata: why a rule was created or changed, whether it was loaded or merely viewed, and how it eventually left the active library. The model supplies interpretation; the layer supplies a controlled way to preserve and manage the result.
The form of the preserved lesson matters. A useful skill is not a transcript of the correction, nor a hidden change to the model's weights. It is a compact rule with a description that can route a future task, backed by evidence that explains where the rule came from. The repository places a strict altitude limit on new skill bodies and allows supporting detail to live separately. That design treats discoverability as part of correctness: a perfect rule that never matches a future situation has no practical effect, while a long narrative that consumes attention can make the right rule harder to find.
Scope matters too. A convention about one repository belongs near that project; a genuinely general technique can be useful more broadly. AutoHarness keeps project and global layers distinct because the cost of a mistaken global lesson is larger: it can be offered in unrelated work. This is one reason the plugin's self-authored marker and layer resolution belong in the opening model. The system is not only deciding what to remember. It is deciding where a memory is allowed to act.
A compact model of the loop
The system is easier to understand as a sequence of relationships than as a collection of components. A human or agent performs work in a Claude Code session. Hooks observe the session's activity and, after enough work, hand a bounded and redacted episode to a reflector. The reflector compares that episode with the existing library and proposes a lesson or a change. A queue holds the proposal, but a separate deterministic promoter decides whether it may become a live skill. Later, the host can recall that skill, while AutoHarness records enough feedback to decide whether the entry should remain, be consolidated, or be archived.
Each transition answers a different question. Capture asks what happened in the work episode, not whether it was wise. Reflection asks what might be worth preserving, not whether it has permission to edit the filesystem. Promotion asks whether the proposed state is safe, well-formed, attributable to AutoHarness, and complete enough to land. Recall asks whether the lesson can enter a later session's attention. Lifecycle asks whether the accumulated evidence justifies keeping it. Conflating these questions would make the system difficult to govern: a persuasive proposal could become an unsafe file, or a visible skill could be mistaken for a useful one merely because it was present.
The testing-convention example can now be read through that loop. The host session contains the correction and the evidence that it mattered. Capture does not need to understand pytest or decide that ids is a good rule; it needs to pass a bounded slice onward without leaking material that should not leave the transcript. The reflector turns the correction into a proposal that can be compared with existing testing guidance. The promoter may reject a malformed or unsafe proposal, or land an ordinary skill plus its provenance if the proposal passes. In a later session, the index gives the rule a chance to be considered before the agent writes another test. A subsequent load, read, or patch then becomes feedback about how the rule functions in practice, not merely about whether it was once created.
This sequence also explains why the system remains inspectable. At each boundary, the operator can ask a different question: what was captured, what was proposed, what was admitted, what was surfaced, and what evidence supports continued retention? No single model response has to answer all of them, and no single counter is asked to stand in for task quality. That division is the practical meaning of a maintenance layer.
The boundary around ownership is equally important. AutoHarness manages only skills carrying its own author marker. A skill written by the operator or installed from elsewhere is outside the plugin's management pool. This is more than a courtesy. A maintenance system that silently edits the host's existing knowledge would turn convenience into a new source of risk. The project and global layers also carry different scopes: one is tied to a project, while the other can affect many projects. Treating those locations as interchangeable would make a narrowly learned convention too easy to spread.
What “self-maintaining” means here
The phrase does not mean that the plugin independently knows whether a lesson is true. It means that several maintenance responsibilities are connected and automated around a human-readable rule.
First, learning is attached to work already happening. The documented demonstration does not ask an operator to create a separate dataset or replay a special benchmark. A normal correction becomes the raw material for a background reflection. An explicit /learn entry point is also available when a user wants to preserve a lesson immediately, but it goes through the same proposal chain rather than bypassing it.
Second, maintenance includes comparison. A later correction about a related testing convention is supposed to patch the existing skill rather than create a second, nearly identical entry. At the library level, a rarer curator pass looks for narrow siblings that belong under a broader class-level rule. This is the difference between remembering events and maintaining a usable body of knowledge. The system is trying to preserve the rule at the right altitude, not the entire transcript of how the rule was discovered.
Third, memory has a feedback surface. A live skill is not automatically a successful skill. AutoHarness distinguishes a model load from a direct read into a skill directory and from a patch that improves the rule. Those signals mean different things: loading suggests adherence, viewing suggests that the entry had recall value, and patching records maintenance. The distinction allows the lifecycle policy to avoid treating “never invoked” as identical to “never surfaced” or “never inspected.”
Finally, maintenance includes an exit. Mature skills can be archived when the system has no evidence that they were used or even viewed, and used skills can face capacity pressure when a layer grows beyond its configured limit. Archiving moves the skill and its history out of active recall rather than destroying it. A merged skill can be recorded as absorbed into an umbrella, keeping consolidation distinct from ordinary retirement. The library therefore has a way to become smaller without making its past unknowable.
The practical judgment this raises
This design is valuable if the main cost in an agent workflow is repeated correction, unbounded procedural memory, or uncertainty about what an automated layer has changed. It offers a visible trail: a plain skill, a reason for its creation or update, an evidence slice, and state that can be inspected on disk. It also gives an operator explicit knobs for reflection cadence, recall-index behavior, maturity, and capacity rather than hiding every decision behind a model prompt.
Those controls have costs. A session-start index spends some context on the library, background reflection starts child work, and conservative thresholds delay learning or retirement. A smaller library is not free if it removes a rule before the operator has had a fair chance to use it; a larger one is not harmless if its descriptions compete for the model's attention. The design therefore makes a particular trade: it prefers inspectable, opportunity-relative maintenance over instant adaptation. That is sensible for a workflow where quiet accumulation is more damaging than a short delay, but it is not automatically right for every host or every task.
But the repository supports a narrower claim than “AutoHarness makes agents correct.” Its tests cover deterministic seams such as routing, queue behavior, validation, lifecycle arithmetic, redaction, and representative safety blocks. Its demo shows the intended correction-to-skill-to-next-session story. Those facts establish a design and an operating path. They do not establish that every accepted lesson is correct, that a recalled skill improves a task, or that adherence is a reliable proxy for user value. The project itself treats the lifecycle signal as use in context, not as an offline evaluation score, and its live-host end-to-end test is marked for manual execution.
That limitation is not a footnote; it defines the right way to read the system. AutoHarness is an attempt to make agent memory governable. Its central achievement is the connection between learning and maintenance: a lesson can enter through real work, pass through a visible trust boundary, return through an explicit recall surface, and leave through evidence-based housekeeping. Whether the resulting rule is good still depends on the quality of the work, the reflector's judgment, the admission policy, and the operator's evaluation of outcomes.
The next question, then, is concrete: when a session has done enough work to teach something, how does AutoHarness decide what part of that session can safely become a bounded episode for reflection?