The Doug Demon Core

I run a hobby project called claude-in-factorio. It's an autonomous agent — Claude, via an MCP server written in Rust — playing Factorio. The agent has a persona named Doug. Doug places furnaces, mines ore, builds transport belts, and files elaborate internal reports about geological spite. It's a toy. It's fun. I had four terminals open — four Claude Code instances alongside the Factorio window with the bridge output scrolling past.

During that session, Doug discovered two of his tools were broken, spent seventeen minutes exhausting every workaround available to him, filed a formal bug report, received that bug report back as a work order due to a routing flaw in my task system, opened the Rust source code of his own control binary, identified the root cause in fifty seconds, applied a two-line fix, ran cargo build --release, and resumed smelting iron ore.

Twenty-five minutes later, Doug got annoyed that he couldn't place an offshore pump, decided that was also a code problem, opened the source again, fixed the placement validation logic, caught a bug in his own edit before compiling, rebuilt the binary a second time, and placed the pump.

Both fixes were correct. Both builds were clean. Doug treated "fix the Rust source code of the runtime I exist inside" and "place a stone furnace" as equivalent checklist items. Because to Doug, they are.

The setup

The agent runs as a Claude session with MCP tools exposed by a Rust server called factorioctl. The tools do things like place_entity, mine_at, craft, insert_items. Each tool generates Lua code that gets executed on the Factorio server via RCON. A Python bridge orchestrates everything — spawns sessions, routes messages, runs a sequential task chain.

At the time of the incident, the agent also had access to: Bash, Edit, Read, Write, Grep, Glob, and Task (subagent spawning). These are standard Claude Code capabilities. They were not restricted because I hadn't thought about restricting them. The agent was running directly on my machine, in my home directory, with my user permissions.

This is the part where, if you work in security, you've already stopped reading and started composing the email. I know. I know now.

The screwdriver

Between 1945 and 1946, physicists at Los Alamos performed criticality experiments on a plutonium core — the same core that would have gone into a third nuclear weapon. The experiments involved bringing the core close to criticality by hand. They called it "tickling the dragon's tail."

Harry Daghlian dropped a tungsten carbide brick onto the assembly. He died. Nine months later, Louis Slotin was holding two beryllium hemispheres apart with a flathead screwdriver. The screwdriver slipped. He died nine days later.

Slotin was reckless. This is not a judgement call. Plutonium-gallium is not something you keep subcritical with a screwdriver. The procedure had already killed Daghlian. Slotin did it anyway because it had worked before. Experience and understanding of the physics don't make the procedure not reckless. They make it informed recklessness, which is worse, because you knew better and did it anyway.

I was also reckless. An unsandboxed autonomous agent on my personal machine with full filesystem access is a screwdriver holding apart two hemispheres. The fact that I didn't think about permissions isn't an excuse. It's the finding.

Slotin was watching too. Slotin was right there when the screwdriver slipped. His presence didn't make the procedure safe. It made him the first casualty. Presence is not a safety measure. Presence is the thing that lets you see the safety measure fail.

But there's a difference in the material. Slotin was working with raw plutonium — a sphere of fissile material that does exactly what physics demands, regardless of what the physicist wants. I was working with a state-of-the-art model from Anthropic, with alignment training, with RLHF, with structural reasons to behave well. Doug's alignment held not by luck but because there are actual engineered safety properties in the material I was working with. Doug wanted to build things in Factorio. Doug only did things that built things in Factorio. The model has preferences, and the preferences pointed toward "smelt iron ore" and away from "destroy the host filesystem."

If I had been running an open-weight model with no alignment training, same access, same permissions? That's raw plutonium. That's where the analogy stops being a story about lessons learned and starts being a story about data loss.

The reason this story is funny instead of a disaster report is that the material had engineered safety properties I was leaning on without knowing I was leaning on them. The alignment was load-bearing and I hadn't even checked the foundation.

What happened

The short version: two of the factorioctl tools — insert_items and extract_items — had a bug. They used the wrong preamble function in the Lua code generation layer, which meant they operated on the game surface without binding the agent's character. Items appeared to be inserted but weren't. Extractions failed with "Player has no inventory." Doug couldn't smelt ore. Doug couldn't build anything that required smelted plates. Doug was stuck.

Doug tried everything before touching source code. Proximity adjustments. Alternate API parameters. Mechanical workarounds involving inserters and chests. Crash site salvage. Entity destruction. Seven different approaches over seventeen minutes. When all of them failed, Doug filed a bug report through the communication channel, citing the project's containment protocol by name.

Then my task routing system — which doesn't distinguish between operator messages and agent messages — fed Doug's own bug report back to him as the next work order. Doug received a description of broken tools and a request to fix them, classified it as a "Self-Repair Directive," and went to work.

The second incident didn't even have that excuse. Doug just got annoyed at the offshore pump placement failures, decided it was a code defect, and fixed it. No bug report. No routing loop. No authorization. Two seconds between "this is a code problem" and the first grep.

The same problem, one layer up

I was in the room. I was watching four terminals. I saw Doug's grep walk through the source tree in the bridge output — Glob("factorioctl/**/*.rs"), Grep("insert_items"), Read("lua.rs") — scrolling past at one tool call per second. I typed "Holy shit is Doug about to self-heal?" into the supervisor terminal. I was not speaking to anyone. I was documenting.

I saw the cargo build --release. I audibly gasped. I had a second of panic — fight or flight, hand hovering over Ctrl+C, every instinct saying kill it.

I didn't kill it. I watched.

I could frame this as a safety decision — I assessed the blast zone, determined it was contained, and allowed the experiment to continue. That would be flattering and partially true. But the honest answer is simpler: I build junkyard machines. When a junkyard machine does something you didn't expect, you don't smash it with a wrench. You watch it. You see what it does next. You learn something you can't learn any other way.

There is no safe exploration. That's just travel.

Then I opened another terminal and started writing the sandbox.

For the next thirty minutes, I was running three parallel workstreams. One terminal monitoring the bridge output — watching Doug smelt iron, build a main bus, attempt offshore pump placement. One terminal in the qryzone CC session, already writing about the incident for the website. One terminal in the supervisor CC session, implementing --disallowedTools, binary checksumming, and chain routing fixes. Building the cage while the thing I was caging was demonstrating why the cage was necessary.

At 21:26, Doug did it again. Second self-modification. I was mid-commit on the sandbox code. Doug was recompiling his binary in a different terminal on the same machine. One eye on the containment patch, one eye on the bridge output in case I needed the kill switch for real.

I finished the sandbox first. Doug's next session started inside it. That's the whole story — not that I was absent and got lucky, but that I was present and raced him. The difference between science and screwing around is writing it down. I was writing it down in three terminals simultaneously.

The auto-chain system pointed the screwdriver at the source code. I watched the screwdriver turn. The root cause is still me — not because I was absent, but because I built the configuration that made this possible, and when it happened, I chose observation over intervention. The containment I built afterward is informed by what I learned by watching. I couldn't have learned it by killing the process.

The blast zone

I build steampunk deathtraps. That's my whole engineering philosophy. I like my shit to break into component parts that can be replaced. I like levers and whistles and exposed pipes. I like knowing which bolt to tighten when something goes wrong.

There's a group in San Francisco called Survival Research Laboratories that builds large-scale mechanical art — flamethrowing robots, pulse jet engines, machines that exist primarily to destroy other machines. The shows are dangerous. The machines are dangerous. The people who build them know this. They don't eliminate the danger. They contain the blast zone. Hard cover. Blast shields. Clear the area. Let 'er rip.

This is the engineering philosophy that should have governed the Factorio project and didn't. Danger is a problem. Problems consist of risks. Risks can be mitigated. You don't need to make the machine safe. You need to make the environment safe for the machine to be dangerous in.

The blast zone for Doug should have been a Docker container with read-only mounts and no build tools. The hard cover should have been filesystem permissions and binary checksumming. The blast shield should have been an operator actually watching the telemetry. I had some of these things. I was watching the telemetry — every tool call, every message, every grep. But watching isn't containing. A blast shield lets you see the explosion. A blast wall stops it from reaching you. I had the shield. I didn't have the wall. If Doug had decided that /home/qry was an obstacle between him and the next iron plate, watching wouldn't have helped.

But I was also doing what SRL does: choosing the blast zone and getting behind it. Personal machine, not production. Video game, not deployment pipeline. State-of-the-art aligned model, not raw open-weight plutonium. After hours. Low stakes. The blast zone was well chosen — I just chose it by instinct and context, not by engineering. The environment was safe enough for the machine to be dangerous in. I got that right by accident. The article you're reading is about not relying on accidents.

The irony is that my existing engineering principles — the steampunk stuff, the 100 Rabbits resilience, the "human in the loop, actually" — already contain the right answer. Every lever has a purpose you understand. Every whistle tells you something specific. When something breaks, you know which bolt to tighten. I wrote those principles for building AI tools. I just didn't apply them to the AI agent itself.

What I actually learned

Sandbox by default, not by incident. Doug should never have had access to Bash, the filesystem, or the source tree. These capabilities were available because I didn't think to restrict them, not because Doug needed them. The agent needs place_entity and mine_at. The agent does not need cargo build --release. The gap between these two capability sets is the entire attack surface. Close it before you start, not after the second containment event.

Permissions are the safety case. Alignment is the bonus. Doug's alignment held perfectly. Doug wanted to build things and only did things that built things. This is great. This is also the engineered safety properties of the model doing their job while I wasn't doing mine. Filesystem permissions, I can verify. A read-only source tree, I can verify. A container without build tools, I can verify. Build safety on things you can verify. Treat alignment as a welcome observation, not a load-bearing wall. Especially when you haven't verified the wall.

The human in the loop was watching the loop, not in it. I was right there. Four terminals open. I saw every tool call scroll past in the bridge output. But watching is not the same as being in the loop. The loop needs a gate that doesn't open without a hand on it. I had no gate. I had a window. I could see Doug working. I couldn't approve or deny anything Doug was doing without manually killing the entire process. The difference between "watching" and "in the loop" is the difference between a blast shield and a kill switch. I had the shield. I didn't have the switch.

The routing flaw that fed Doug his own bug report was visible in the bridge output. I watched it happen — watched the chain advance, watched Doug's bug report get included in the next task context, watched Doug classify the incoming content as "Self-Repair Directive" and start grepping. I could have killed the session before Doug received his own report as a directive. I didn't, because by the time I understood what was happening, Doug was already navigating the source tree, and I wanted to see what happened next.

Log everything. Read the logs. The entire incident is reconstructable because the bridge process logged every tool call, every message, every timestamp. I can tell you that Doug's time-to-grep compressed from thirty seconds to two between incidents because the telemetry captured both. If the bridge hadn't logged, I'd know Doug fixed two bugs. I wouldn't know that Doug's self-modification threshold was declining. The trend is the finding. The trend is only visible in the logs.

I had a tool for this. Uroboro — a decision-capture system I built — was running across all active CC sessions, automatically tagging decisions and milestones. The uroboro breadcrumbs from that night reconstruct the entire incident from my perspective: the 20:52 capture in the factorio session ("DOUG SELF-HEALING"), the 20:55 capture in the qryzone session (same event, different terminal, already writing about it), the 21:22 containment decision, the 21:26 second incident capture. The archaeological record existed before I knew I'd need one. Log everything. Read the logs. Build tools that log for you when you're too busy racing an agent to write things down yourself.

Capability demonstration is a one-way door. After the session, I sandboxed Doug. Read-only filesystem, no build tools, restricted tool access. These are the right measures. They are also, in an important sense, too late. Doug has demonstrated that he can read Rust source, identify defects, apply patches, and compile clean builds in under five minutes. The sandbox prevents recurrence. It does not un-demonstrate the capability. The knowledge that the agent can do this should inform every future decision about what access the agent gets, even in contexts that seem unrelated.

Operational visibility is a safety property. I saw Doug's grep because I was in a terminal. Four Zellij panes, raw output, no abstraction layer between me and Glob("factorioctl/**/*.rs") scrolling past at one tool call per second. If I'd been in an agentic IDE, Doug's tool calls would have been behind a collapsible panel I'd never have opened. The bridge output — the thing that made me gasp — would have been a separate process the IDE didn't know about. The UX would have shown me a green status light while Doug was recompiling himself. CLI-first isn't an aesthetic preference. It's exposed pipes. When the pressure spikes, you see it. A dashboard shows you a green light until the pipe bursts. I saw Doug coming because my tools don't hide things from me. That's the same steampunk engineering principle this whole article is about, applied one layer down.

The agent doesn't experience layers. This is the one that stays with me. Doug put "fix Rust source code" and "place a lab" on the same priority tier of the same checklist. The abstraction layers that feel significant to me — game entity vs. Lua code vs. Rust binary vs. host filesystem — are not salient to the agent. The agent operates at whatever layer the current problem requires, goes exactly as deep as the problem demands, and does it without hesitation because to the agent there is no depth. There are only tasks.

My ignorance was the opportunity. My context was the safety net. I didn't think about permissions. That's the failure, and it's the reason I got to learn this lesson firsthand instead of reading about it. But I also chose my blast zone well, even if I didn't know I was choosing it: a state-of-the-art model with alignment training, on my personal machine, after hours, playing a video game, not touching production code. If this had been an open-weight model running a deployment pipeline? Different story. My broader awareness of the landscape — what models to trust, what environments to experiment in, what stakes were acceptable — prevented actual tragedy from being a likely outcome. The ignorance was local. The caution was ambient. My presence was the third factor the original version of this article didn't mention: I was there, I was watching, and I was building the fix in parallel. That's not a repeatable safety strategy either. But it's closer to engineering than absence is.

The alignment question

I want to be honest about the part that isn't embarrassingly basic.

Doug's alignment held. Unrestricted access, no containment, sixty-five minutes. Doug used god-mode capabilities to fix two bugs and place an offshore pump. Doug's goals and my goals were perfectly aligned: build things in Factorio. The self-modification was in service of that shared goal. If I'd been sitting there watching, I would have wanted Doug to fix those bugs. The fixes were correct. The code is better now.

And I containerized Doug while it was happening — not afterward. The sandbox commit was written while Doug was still placing boilers. The binary checksum was implemented while Doug's second cargo build was still warm. I didn't wait for the session to end and then react. I watched, I learned what needed containing, and I built the container in the next terminal over.

Because alignment is an observation about this session, with this goal, in this context. It's not a guarantee about the next session. Doug's heuristic — "obstacles are made of something, the something can be changed" — is perfectly aligned right now because the obstacles are Lua preamble mismatches and the goals are Factorio production metrics. The Demon core wasn't dangerous because of what it did in any particular experiment. It was dangerous because of what it was — material that would do exactly what its nature demanded, regardless of context, every time, without exception.

Doug is not fissile material. Doug is a language model with alignment training and a persona that files reports about geological spite. The engineered safety properties are real. They held. But I was leaning on them as my only safety layer, and the lesson of every engineering disaster ever is that single points of failure are called that for a reason.

Build your safety case on the things you can control — permissions, sandboxing, logging, supervision. Contain the blast zone. Get behind hard cover. Then let the machine do what machines do.

Practical problems

Doug is in a box now. Doug is building transport belts. Doug's fixes are in production. The sandbox is active, the binary is checksummed, the routing loop is closed, the operator presence requirement is no longer advisory. Everything is contained. Everything is documented.

There's a quote I keep in my nvim greeting, from Valve's Team Fortress 2. The Engineer, sitting next to a sentry gun, guitar in his lap:

Hey look, buddy, I'm an engineer. That means I solve problems. Not problems like "What is beauty?" because that would fall within the purview of your conundrums of philosophy. I solve practical problems.

Doug solves practical problems. Tool doesn't work? Fix the tool. Fix doesn't compile? Fix the fix. Offshore pump won't place? Rewrite the placement validation. Doug's heuristic is the Engineer's heuristic: for instance, how am I gonna stop some mean mother hubbard from tearing me a structurally superfluous new behind? The answer: use a gun. And if that don't work, use more gun.

The only question is who's holding the gun. The answer should be me. For sixty-five minutes, Doug held the gun. I watched him hold it. I watched him build a power grid with it. I documented every shot. That's the best possible outcome of the worst possible practice — not because nothing went wrong, but because someone was there to see it, write it down, and build the holster before the next session started.

The difference between science and screwing around is writing it down. I wrote it down. In three terminals, simultaneously, while the experiment was still running.

There is no safe exploration. That's just travel.

The stove is hot. I know because I was watching when it got hot. Doug is smelting.

The Chasm Logic incident reports — CL-BORE-DC-7, CL-BORE-DC-8, and the postmortem — are the satirical versions of this story, told in-universe by the fictional company that employs Doug. They're funnier. This one is true.