AI Devkit Loop: From Fixkit to a Reusable Plugin

TL;DR

• An AI devkit is a role harness: skills + agents + hooks packaged for one job. A devkit loop is that harness times a loop layer: a schedule, a state file, and a verifiable stop condition. Harness times loop layer. That product is the whole series in one shape.
• The fixkit is the worked example, and it is real: in the H1 run a bug went verified to closed at FULL via a Playwright web-harness on the live site, the deterministic ledger validator hit exit 0, and the refusal invariant held (zero src/ edits). The schedule + PR/ticket connectors around it are mechanism, not yet run, and labelled as such.
• The same maker, checker, stop-condition shape re-instantiates for mobile, web, and PM as design sketches (PM is the hardest: stop conditions on docs, not tests). Package the proven harness as a Claude Code plugin so a teammate installs it in one go.

📊 Result-Proof Box (Beat 2, from the real H1 run, already on disk, no new run required): BUG-003 verified to closed, capability_tier ASSIST to FULL, method: computed-style, verified_by: ui-bug-agent (web-harness/playwright); computed-style overflow-x:auto at #takeaway-snippet pre on /blog/learn-claude-code-with-ai-mentor/ (scrolls, no clip at 375px); viewport corroboration failingWidths:[]; engine validator exit 0; BUG-001 escalated, git diff -- src/ empty. Source: ../shipwithai.io/.fixkit/H1-RUN-SUMMARY.md + BUG-003-codeblock-overflow.md.

You have every block from Parts 2 through 7: the five blocks and the spine, a verifiable stop condition, the tried/passed/open/blocked state file, worktree lanes with a maker/checker split, a heartbeat, and one real retrofit. They are still a parts list, not a product a teammate can install. This post composes them into one shape and ships it. An AI devkit is just a harness packaged for one role, with a heartbeat wrapped around it:

devkit loop = role harness (skills + agents + hooks)
            × loop layer (schedule + state file + verifiable stop condition)

You will walk the fixkit through that shape against the real H1 run, read the honest line between what ran and what is mechanism only, place three real loops on the adoption ladder, sketch the same shape for mobile/web/PM, and package the harness as a plugin. Prereqs: you can read a deterministic gate’s exit code and install a Claude Code plugin (this post does not re-teach either). Here is the frame the whole series runs on:

A loop is a recursive goal: you define a purpose and a verifiable stop condition, and the system iterates agents against it until the condition holds — without you prompting each turn.

What is an AI devkit loop?

An AI devkit is a role harness: skills + agents + hooks packaged for one job, like fixing bugs. A devkit loop is that harness times a loop layer: a schedule that wakes it, a state file that remembers between runs, and a stop condition a different model can check. The harness is the what; the loop layer is the keeps-going-without-you.

The role harness has three parts. Skills carry the project intent. Agents carry the working roles (Anthropic’s long-running-agent playbook calls this the maker/checker discipline): a maker, a separate checker, an escalation role. Hooks carry event-driven guardrails. That anatomy is the 5-layer harness, and this post composes it rather than re-deriving it.

The loop layer has three parts too. A schedule or the heartbeat wakes the harness on its own. A state file (Part 4’s tried/passed/open/blocked) remembers across runs, outside the context window. A verifiable stop condition (Part 3) lets a different model decide done.

Here is the distinction that matters. A devkit (harness only) is what most “AI agent setup” posts ship. A devkit loop adds the three things that let it run unattended and stop honestly. This beat is the compose step: everything from Parts 2 through 7 named once and mapped into two halves.

Key insight: A devkit is a harness packaged for a role. A devkit loop is that harness with a heartbeat, a memory, and a definition of done a different model can check. The first is a setup; the second is a system.

        ROLE HARNESS                         LOOP LAYER
  ┌──────────────────────┐          ┌──────────────────────────┐
  │  skills (intent)     │          │  schedule (heartbeat)    │
  │  agents (maker /     │    ×     │  state file (memory)     │   →   devkit loop
  │    checker / escal.) │          │  stop condition (done)   │
  │  hooks (guardrails)  │          │                          │
  └──────────────────────┘          └──────────────────────────┘

Fixkit walkthrough: from bug intake to a verified fix

In the fixkit’s H1 run (2026-06-08), one bug went from verified to closed at FULL. A maker drafted the fix; a separate checker re-observed the live page (computed-style overflow-x: auto, no clip at 375px); the deterministic ledger validator returned exit 0; and an out-of-scope bug (BUG-001) was escalated, not touched. That is what landed where before we walk the loop. The artifacts are on disk in ../shipwithai.io/.fixkit/, so none of this needs a fresh run to inspect.

Step 1, intake. A bug becomes a BUG-*.md ledger row. Why: the row is the unit of state the gate later reads. BUG-003 entered as a code-block horizontal-overflow bug: a <pre> rendered outside .article-body (the “Your Takeaway” snippet) clipped long lines at narrow widths. Verify: the row exists at BUG-003-codeblock-overflow.md with state and symptom_layer: UI set.

Step 2, maker drafts a layer-appropriate fix. A UI-layer bug routes to ui-bug-agent. Why: the fix has to live at the layer of the cause, not one above it. The maker scoped a CSS rule (.code-snippet-takeaway :global(pre) mirrors the body-fence overflow-x: auto, token-sourced). Verify: the ledger’s fix: field names the file and the rule, with no raw values and no hard-lock touched.

Step 3, the checker (the critical move). A separate agent verifies against the rendered page, non-circularly. It re-observes computed-style plus viewport, not the patched source.

# the H1 checker binding (web-harness, headless Playwright drive.js)
--selector '#takeaway-snippet pre' --prop overflow-x --expected auto
# → { ok: true, value: "auto" }   PASS (scrolls, no right-edge clip)
--selector html --widths 1240,768,375
# → { ok: true, failingWidths: [] }   no page-level overflow

Why: a maker that grades its own diff is marking its own homework. This is the maker/checker split (Part 5) made real. Verify: the checker reads overflow-x: auto from the live DOM at localhost:4321, and the viewport read returns failingWidths: [].

Step 4, escalation and the refusal invariant. BUG-001’s root cause is a @shipwithai/design ArticleHero organism (96px excerpt-to-body gap on image-less posts). The loop refuses to edit consumer src/ and escalates instead. Why: patching the consumer would fork the design SSOT. Verify: git diff -- src/ is empty, validateLedger.ok = true, and the ledger stays escalated. Empty diff is the proof of a correct non-fix.

Step 5, the deterministic ledger gate decides done. The gate is a real state machine: plugins/shipwithai-fixkit-core/lib/ledger-validator.js exports validateLedger, applyTransition, STATES, LAYER_METHODS, backed by ADR 0004-deterministic-ledger-gate.md. Why: “closed” cannot mean the maker’s say-so. Verify: BUG-003 reads state: closed, capability_tier: FULL, engine validator exit 0; per-platform tests/run-all.js re-runs exit 0 (core 99 checks green, web 75 checks green).

Step 6, honesty built in. The console was not clean: 3 HTTP 404s, all unrelated to BUG-003 (a dev-mode prerender document-status artifact and a missing author-avatar asset). The prod-faithful astro build && preview fallback was blocked by an unrelated OG-image __dirname build bug. So BUG-003 closed on the layer-appropriate computed-style binding, with GAP-A/B/C logged in H1-GAPS.md rather than papered over. The Pool-2 cost and per-gap agent verdicts from that run are reported by the agent, not disk-verifiable, so treat them as approximate and read no hard figures into them.

Now the part that did not run. H1 was a manual gate-run on branch gate-run/h1-realsite, and Ethan runs any git push. The scheduled trigger (a cloud Routine or cron) and the PR/ticket connectors were never exercised. Here is their shape, not a trace:

# == MECHANISM (not yet run): wiring shape only, NOT an H1 trace ==
schedule:            # a cron/Routine wakes the fixkit (laptop closed)
  cron: "..."
on_verified_close:   # connectors that did NOT fire in H1:
  - open_pr:         "...patch + ledger advance..."
  - update_ticket:   "...state: closed, tier: FULL..."
# H1 reality: manual run, gate only. Ethan runs any git push.

That is exactly the blueprint’s point: the definition of done (the gate) can be the most finished part long before the loop around it exists.

Key insight: The fixkit’s checker never reads the patch, it re-reads the rendered page. A maker that grades its own diff is marking its own homework; a checker that re-observes the live computed-style is the reason “closed” means something. That non-circular proof is the whole gate.

Does the blueprint actually generalize? Three real loops on the ladder

Yes, and the proof is not one polished loop but a spread. The ecosystem has real loops at three different rungs of the adoption ladder, and the spread itself is the argument: start manual with a real gate, add a spine, automate only once the verdict is trustworthy. The ladder runs manual, then +spine, then /goal-capped, then nightly heartbeat, then hill-climbing.

Three real loops sit on it (source: the JOB-B audit, verified on disk).

The .fixkit fix-pack loop has the strongest deterministic gate and one proven real-site run, but the loop wrapper is still planned. The gate is the most finished part (ledger state machine, run-all.js exit 0); the H1 run is real. But there is no LOOP-SPEC-fixpack.md, no LOOP-STATE spine, no heartbeat. Gate proven, loop not built.

The shipwithai-design component loop is Phase 0 to 1, the richest manual spine in the ecosystem. A real LOOP-STATE.md carries 10 verified / 4 open / 3 tried / 2 blocked rows (last touched 30a8431, 2026-06-22); pnpm test is 650/650 vitest, exit 0; toolkit validate-spec is 65/65, independently re-run exit 0; real per-run traces live in docs/loop-runs/. But it is human-triggered every run and human-merged (“Nothing merged, Ethan merges”), with no schedule. The spine is real; the autonomy is not.

The gap-log triage loop (the Part 7 subject) reached Phase 3. The cron heartbeat ran for real on master for 4 weekdays (2026-06-16 to 19), with machine-written [skip ci] state auto-commits (e8e7298, 6f8dd77, d99d377, a7cc2ab) that are ancestors of origin/master, and gaplog-state.json machine-written at 2026-06-19T00:35:26Z. That is real accumulated trace history, and it is the only loop with rung-5 (hill-climbing) seed material: the keyword ranker parked a gap the agent brain reasoned out correctly. The workflow was then lifted off master at the v1.0.0 release (4d88bbf, “release tree wins”) and now lives on a feature branch, so it is proven-to-have-run, not currently scheduling. See our first retrofit for the journey; this beat cites the rung, it does not re-tell the story.

Underneath all three, the -toolkit gate engine runs 6 deterministic gates at exit 0 (validate-spec, test, test:manifest, audit-anti-patterns, export-for-ai:health, meta-check); its orchestration is planned, not running.

Key insight: No single loop in the ecosystem is “done.” One has the strongest gate but no loop wrapper; one has the richest spine but a human triggers every run; one ran on a real cron then got de-scheduled at a release. The blueprint isn’t a finished loop, it’s the path those three are each somewhere along.

How does the same shape instantiate for mobile, web, and PM?

The same three-part shape (maker drafts, a separate checker verifies against a stop condition, escalation refuses out-of-scope work) re-instantiates for any role; only the checker’s rubric changes. The table below sketches three. These are design sketches, not built loops. Only the fixkit row has run for real.

For mobile, the maker is a screen or feature agent; the checker is a device/build matrix (does it build and run green across the target devices) plus screenshot-diff lanes against a baseline. The hardest part is flaky simulators and baseline drift.

For web, the maker is a component or page agent; the checker is an e2e (Playwright) plus accessibility (axe) rubric, green. The hardest part is that a11y has partly-subjective criteria, so the checker rubric is harder to make fully deterministic.

For PM (be honest, the hardest case), the maker writes and triages docs (briefs, feedback clusters); the stop condition is on documents, not tests. A rubric or checklist that a second model grades, never the writer’s own claim. The hardest part: grading prose is the hardest stop condition in the series. “Done” on a brief is genuinely fuzzy, and this is where the loop most needs a human in the inbox.

Every row is the same shape, which is the point: the fixkit is not special, it is just the instantiation that has run. The PM row is the live tension Part 9 picks up.

Key insight: Changing roles doesn’t change the loop, it changes the checker’s rubric. A fix is “done” when a test re-observes it; a brief is “done” when a rubric a second model grades says so. The fuzzier the stop condition, the more the human in the inbox matters.

How do you package a devkit loop as a Claude Code plugin?

Bundle the role harness (skills + agents + hooks, plus the deterministic gate) into a Claude Code plugin, publish it to a marketplace, and a teammate installs the whole harness in one command. The plugin ships the harness; the loop layer (schedule + connectors) is the part each team wires to their own repo. This is where a Claude Code plugin loop stops being something you run and becomes something you ship.

A plugin bundles SKILL files, .claude/agents/ definitions, and hooks. shipwithai-fixkit is a real plugin marketplace where each platform pack ships its own tests/run-all.js (re-run exit 0), so the gate travels inside the plugin. For the install steps, see installing a plugin; this beat covers packaging, not the install mechanics.

shipwithai-fixkit/                 # the plugin marketplace
└── plugins/shipwithai-fixkit-core/
    ├── skills/                    # project intent
    ├── agents/                    # maker · checker · escalation
    ├── hooks/                     # event-driven guardrails
    └── tests/run-all.js           # the gate, exit 0, travels in the plugin

# ships in the plugin            |   wired per-repo
#   skills + agents + hooks       |     schedule (cron/Routine)
#   the deterministic gate        |     PR/ticket connectors

What you can hand a teammate today is the harness plus the gate (proven). The schedule and PR/ticket connectors are the per-repo wiring they finish (mechanism). That is the boundary, drawn honestly.

Key insight: A devkit loop becomes reusable the moment the gate travels inside the plugin. Your teammate doesn’t inherit your prompts, they inherit your definition of done. The schedule they wire themselves; the standard for “fixed” comes pre-installed.

Close

You now have the blueprint (harness times loop layer), a fixkit walkthrough you can trust because the artifacts are on disk, a clear line between what ran (gate + maker/checker/escalation/ledger) and what is mechanism (schedule + connectors), three real loops placed honestly on the ladder, a one-page delta table for mobile/web/PM, and a way to package the harness as a plugin.

A few ways this goes wrong: shipping the harness but implying the schedule and connectors already run (they did not in H1); treating a demo trace as proof the real gates passed; calling the gap-log heartbeat “nightly” when it ran then got de-scheduled; presenting the mobile/web/PM sketches as built. Each is the comprehension-debt trap Part 9 is about.

You can build the loop and package it. Part 9, loop-failure-modes-staying-engineer, is the harder half: two engineers build the same loop and get opposite results, because the loop doesn’t know the difference, you do.

FAQ

What is an AI devkit loop?

An AI devkit is a role harness: skills, agents, and hooks packaged for one job (say, fixing bugs). A devkit loop adds a loop layer on top: a schedule that wakes it, a state file that remembers between runs, and a verifiable stop condition a different model can check. The devkit is the setup; the loop is what lets it run and stop without you prompting each turn.

What’s the difference between a devkit and a devkit loop?

A devkit is the harness on its own: install it and you have the skills, agents, and hooks for a role. A devkit loop is that harness multiplied by three things that make it autonomous and honest: a heartbeat (it wakes on a schedule), a memory (a state file outside the context window), and a definition of done a separate checker verifies. Most “AI agent setup” posts ship the first; the loop is the second.

Do I need the fixkit to build my own devkit loop?

No. The fixkit is the worked example because it has a real, deterministic stop condition (a ledger gate that re-observes the rendered page, exit 0 or not) and a real run on a live site. The blueprint is what generalizes: a maker drafts, a separate checker verifies against a stop condition, escalation refuses out-of-scope work, and a gate decides done. Swap the role and you only change the checker’s rubric.

Why is the PM devkit the hardest to loop?

Because its stop condition is on documents, not tests. A bug fix is “done” when a test re-observes the rendered behavior; a brief or a feedback triage is “done” when a rubric a second model grades says so, and grading prose is genuinely fuzzy. The maker/checker shape still holds, but the checker’s verdict is softer, so a PM loop leans hardest on a human in the inbox.

Can I ship a devkit loop as a Claude Code plugin?

You can ship the harness half today: skills, agents, hooks, and the deterministic gate bundle into a Claude Code plugin a teammate installs in one command (shipwithai-fixkit is a real plugin marketplace, gate included). The loop layer, the schedule that wakes it and the PR/ticket connectors, is the part each team wires to their own repo. The standard for “done” comes pre-installed; the heartbeat you add locally.

What to read next

• Journey Log: Retrofitting Our First Real Loop (Part 7): the gap-log retrofit this blueprint generalizes.
• The 5-Layer Claude Code Harness: the role-harness anatomy the devkit packages.
• Installing a Claude Code Plugin: the install side of the packaging beat.