How We Killed Vibe Coding Chaos: What We Learned Building a 400-PR/Day Multi-Harness System
I used to spend 14 hours a day glued to my screen like an AI junkie.
My setup was pure chaos. I had ten Cursor windows open, dozens of browser tabs, and terminal panes constantly running prompt loops. My Mac sounded like a jet engine. Pushing as hard as I could, I was hitting 30 to 60 pull requests a day.
My brain was fried. Every session was fragile. One hallucinated file path or subtle regression could quietly break the build.
The problem was not the models. The problem was how we were working with them.
We were treating coding agents like chat buddies instead of deterministic execution units. We expected an LLM in a single prompt window to hold an entire architecture, remember past choices, coordinate tasks, and grade its own work. That was never going to scale.
Once we stepped back and redesigned the execution harness from first principles, everything changed.
Today, that architecture runs roughly 400 pull requests a day with less than 3% failed sessions, 24/7. It coordinates an entire fleet of model runtimes: Claude Code, Google AI Ultra, Codex, MiniMax, Z.ai, Qwen, and Grok like a Swiss watch.
No multi-window sprawl. No manual babysitting. Today, I talk to exactly one entity: a Planner agent, who coordinates the fleet.
Here is the underlying architecture that replaced vibe coding chaos with a predictable software production loop.
1. The Central Nervous System: Project Management Where the Filesystem is State
Most engineering teams try to bolt AI agents onto human tools like Jira, Linear, or Slack. The result is pure friction. Humans click tickets while bots spam APIs, losing state the second a process crashes or a connection drops.
We realized the coordination layer had to be much simpler: the filesystem is the state.
Every project board is a plain file on disk: boards/.opvsb. Every task card is an atomic markdown document: boards///.opvst.
Moving a task from In Progress to Review is not a slow database write. It is an atomic mv command on the local disk. Lightweight local daemons watch directory events and push changes in milliseconds.
When a human looks at the board, they see live progress. When an agent reads the disk, it reads clean markdown.
An atomic claim engine locks task files during active runs. This guarantees parallel agents never collide on the same task or overwrite the same file.
2. The Real Graph Loop: CodeGraph and Syntax Tree Maps
Why do coding agents write duplicate functions, reinvent existing utilities, and break dependencies?
Because LLMs have no spatial understanding of a repository. Standard tools rely on fuzzy vector search that guesses meaning by keyword proximity. When an agent cannot see the full architectural map, it writes redundant code.
We solved this with CodeGraph, our structural graph harness.
Instead of guessing, CodeGraph parses full syntax trees across every supported language: Python, TypeScript, Go, and Rust. It runs locally using standard tools. This keeps all your code away from external models during indexing.
Before an agent types a single line of code, CodeGraph answers the critical question: Does this symbol, function, type, or database schema already exist?
CodeGraph then schedules work through a real Directed Acyclic Graph (DAG) engine. It maps task dependencies in strict topological order. If Task B depends on Task A, Task B physically cannot run until Task A completes and commits a verified change. The chaos of random agent loops is replaced by a clear dependency graph.
3. Sandboxes as Self-Hosted Runners: Scaling Fleets Under Control
The biggest challenge with running multiple model engines is keeping token costs and account limits under control without giving up model diversity.
The breakthrough was realizing that agent sandboxes should work like self-hosted CI runners.
Instead of routing code through closed cloud wrappers, each sandbox worker polls the board every ten seconds. It claims pending tasks and runs the official CLI for the target model: Claude Code, Google AI Ultra, Codex, MiniMax, Z.ai, Qwen, or Grok inside an isolated container under your own developer login.
To keep full control, each client sets a session ceiling between 1 and 32 concurrent runners per subscription. The claim engine tracks active jobs and will never release a task past your configured limit.
Your existing developer plans transform into an automated, self-throttling fleet. They respect your account limits while draining your backlog around the clock.
Network isolation is complete: an encrypted WireGuard mesh with per-node keys on UDP 51820 links every node. Execution, tests, builds, and repo access stay strictly inside your sandbox; only task context goes to the model you chose.
4. The Token-Efficient Agent Wire Protocol
When multiple agents coordinate across machines or workspaces, how should they talk to each other?
If you make agents chat in English like humans on Slack, you burn thousands of tokens on conversational filler: Hello colleague, could you please review this function? If you use rigid REST APIs, you create brittle links.
We created a compact wire protocol built specifically for agent coordination.
Every message follows a minimalist, token-efficient wire format:
@from>to body
Addresses route across domains on opvs.run and opvs.pub. Instead of wasting tokens on polite chatter, agents communicate like unix pipes and internet routers.
The protocol enforces a clear state machine: seen, working, and resolved or declined. Endpoints resolve automatically through four tiers: DNS TXT records, SRV records, DID web files, and local .well-known/opvs.json manifests. Shared task pools let worker agents claim jobs safely with database lock skips.
Agents exchange context in twenty bytes instead of thousands of tokens, cutting communication costs to near zero.
5. Hard Checks on Every Level: The Total Verification Stack
The most dangerous flaw in modern AI engineering is self-grading. An agent drafts code, inspects its own work, declares success, and silently commits broken changes.
We replaced self-grading with deterministic verification across every layer of the operating system:
- Skills Layer: Every skill pack carries dedicated verification scripts. Like unit tests for code, they inspect output ASTs, parse schemas, and enforce strict thresholds with exit codes of zero or one.
- Developer Commands Layer: Every developer command (/plan, /test, /ship) runs its own verification scripts, gating parameters and prerequisites before an agent can proceed.
- Project Management Layer: Hard validation executes across three distinct tiers. Task level verifies acceptance criteria and tests before a card can leave Doing. Board level enforces column gates, claim locks, and downstream DAG triggers. Project level validates cross-board chains and milestone dependencies across multiple teams.
- Self-Guiding API Envelopes: Endpoints do not return raw, unexplained JSON. Every endpoint returns rich envelopes in YAML or Markdown containing schemas, procedural guides, and historical learnings bundled directly with the response, so agents never execute blind.
On the marketplace, every skill bundle uses ed25519 signatures and must pass twelve security gates before release.
6. The Lifecycle Loop: Start, Test, and Ship
Agent sessions are governed by strict phase rules: Start, Test, and Ship.
In the Start phase, the runner verifies its environment, checks CodeGraph syntax trees for existing symbols, and locks the task card.
In the Test phase, the runner executes the implementation against local test scripts. If a test fails, the process stops cold before any code commits. The exact error log and stack trace feed directly back into the agent context for an immediate, targeted fix.
In the Ship phase, the runner verifies clean diffs and merges the code. Crucially, the ship step automatically appends new learnings and updates microservice-level documentation: CLAUDE.md, SHIP.md, DEPLOYMENT.md, and LEARNINGS.md. The repository gets smarter with every single run.
This closed loop is why our fleets achieve a 97%+ session success rate across 400 pull requests a day, cutting wasted sessions and duplicate code to near zero.
7. Decaying Memory: AgentDocs Versus AgentMemory
Agents fail in two ways: total amnesia between sessions, or bad context caused by hoarding outdated project notes.
We separated knowledge into two purpose-built systems: AgentDocs and AgentMemory.
AgentDocs holds the source of truth. It stores versioned markdown specs, system architecture designs, and brand voice profiles. It changes only through reviewed pull requests, providing a solid foundation that never drifts.
AgentMemory handles dynamic recall. It combines hybrid search with model ranking over past task logs and execution patterns.
Crucially, AgentMemory applies a confidence decay of two percent per week. Facts not reaffirmed in live runs fade away naturally over time, ensuring old errors never stick around to poison fresh tasks.
Beyond the Prompt Box
Vibe coding proved what single prompts can do on small scripts. But production engineering requires proper harnesses: filesystem state, syntax graphs, runner isolation, and hard verification gates.
Once you stop treating models like chat toys and start building real execution harnesses, the productivity ceiling completely disappears.
If you are running into the same scaling walls with agent loops, or if you want to inspect our architecture notes and wire protocol specs, send me a DM with the word HARNESS. Happy to share what we learned and compare notes.