I shipped an AI test harness the other day that lets Claude run my Unity game from the Editor and execute QA workflows end-to-end. The push for this came from save migration pain. By the third migration in a week I had walked New Game, Save, Quit, Load, Inspect by hand forty times and was visibly losing hair to it.
The same harness drove the save round-trip regression that just signed off 0.2.21. Last week it ran its first end-to-end smoke test while I was walking the dog. No mocks, no scaffolding, no test-double simulator. Claude enters PlayMode, creates a profile, plays through the bootstrapper, fires a save, restarts the Editor, loads the profile back, and asserts the world came back identical. Then it cleans up after itself. Any issues get fixed and the test reran recursively. You can practically hear Nancy Cartwright’s infamous “I’m in danger.”
So this post is about the QA process I built on top of Ivan Murzak’s Unity-MCP plugin to make that possible. It is the part of an AI-driven Test Harness loop I have been designing.
What an AI Test Harness Needs From Unity-MCP
Unity-MCP is an open source plugin by Ivan Murzak that bridges Unity to MCP clients like Claude Code. It runs as three layers: a Unity plugin in the Editor, an MCP server process, and the MCP client (Claude, in my case). Out of the box you get 100-plus tools across scene and hierarchy manipulation, asset management, scripting via Roslyn, and Editor control like entering PlayMode and reading console logs. Adding your own tool is one [McpPluginTool] attribute on a C# method. It’s pretty sweet.
So it easily covers everything you need to ask Claude to “make a cube, attach a Rigidbody, run the scene.” It does not cover what I actually needed, which is driving my game’s real Bootstrapper to MainMenu to Game flow with structured assertions at every step. That is the harness layer I built.
The Custom AI Test Harness Layer
Three custom MCP tools, three job descriptions.
dirigible-cmd runs a structured command inside the running game. Input is { command: "save.trigger" }. Output is { Command, CorrelationId, Status, FailReason, OutputLines, ElapsedMs }. Under the hood it pipes the command through IngameDebugConsole’s ExecuteCommand and captures the structured log sentinels (CMD:, OK:, ERR:) by correlation id.
dirigible-state returns typed slices of game state. Input is a key like colonist_count or save_blocked or last_report. Output is { Key, Status, Value }. Sixteen keys today, covering scene state, save coordinator, content services, async op history.
dirigible-wait polls a predicate until it fires or times out. Input is { predicate: "save_finished:42", timeout_ms: 30000 }. Predicates include playmode, scene_loaded:<name>, game_ready, save_finished:<baseline>, services_ready:<scope>, async_done:<correlation_id>, and a log_match:<regex> that subscribes to Application.logMessageReceived for the duration of the wait.
Behind all three is a DontDestroyOnLoad singleton bridge that subscribes to game events, mirrors the save counter, runs a ring buffer of recent async ops, and re-resolves dependency-injected services from every active LifetimeScope on each sceneLoaded. That last detail matters: services come from different scopes in the menu versus the game scene, and the bridge has to find them wherever they live.
The whole harness is dev-build only, guarded by #if UNITY_EDITOR || DEVELOPMENT_BUILD. It never ships to a release build.
Running QA With Claude Code
The first Claude Skill on top of the harness is /save-load-smoke. A markdown file at .claude/skills/save-load-smoke/SKILL.md. Eight steps.
Sweep stale __smoke_* test profiles older than an hour. Enter PlayMode if not already. Wait for the menu scene and for global services to populate. Create a uniquely-named profile seeded with a fixed value (__smoke_<unix_timestamp>) and start a new game. Capture baselines: colonist count, world-object count, save counter, save-blocked flag. Screenshot. Trigger a save, wait for save_finished:<baseline>. Exit PlayMode, re-enter, navigate back to the menu, load the same profile, wait for game_ready. Assert the world came back: profile name preserved, colonist count equal, world-object count equal, save-blocked still false, no truncation entries on either side.
On failure it leaves the test profile alive for inspection, snaps both Game and Scene view screenshots, dumps the last five minutes of error-level console logs, and surfaces the first failing assertion with expected versus actual. No auto-retry. The test exists to surface real regressions, not to mask them.
Why an AI Test Harness Beats Mocks
Most AI-driven test loops stop at unit tests. The agent writes a function, the agent writes a test for the function, the test passes, but both are wrong in the same direction. My AI test harness skips that failure mode by driving the actual game. The bootstrapper actually runs. DI scopes actually compose. The save controllers actually serialize. ES3 actually writes the file. The Editor actually quits and re-enters PlayMode. The load path actually deserializes from disk. There is no mock anywhere. We verify integrity of the actual loaded data.
The trade-off is that runs are not millisecond unit tests. A full save-load smoke is about 90 seconds on the wall-clock. That is fine. It runs at the right cadence: every save-system PR, every migrator change, every release version bump. Saves me hunting for regressions.
What Is Next
The substrate is general-purpose. My roadmap has more Skills queued: /spawn-and-save for entity persistence regression, /load-corrupted for save-recovery paths (needs surgical ES3 key-level mutation, currently file-level backup/restore only), and broader playthrough-style scripts when there is something worth playing through. Eventually the same bridge can support a thin LLM playtester loop, but that is a different post.
The deeper point is that AI co-engineering only scales when the AI can verify its own work against real running systems. Mocks let agents pass tests that humans would never accept. A test harness that drives the real game closes the loop. Unity MCP plus a thin custom layer turns out to be more than enough.
Receipts
The harness shipped as feat!(0.2.21) in dirigible2D on May 21. Unity-MCP itself is open source at github.com/IvanMurzak/Unity-MCP, credit where it is due, the heavy lifting on the protocol bridge is his. My layer on top is roughly 700 lines of C# (three editor MCP tools, one runtime bridge, the console commands), 154 lines of skill markdown, and 346 lines of harness documentation. Daily-ship streak runs 63 days and counting. Receipts are at promptbook.gg/michael-tiller.