Running Code Nobody Reviewed

Running Code Nobody Reviewed is the problem that appears the moment an agent stops calling tools you wrote and starts writing its own. Generated code has passed no review, its author is a component that may be under instruction, and the only meaningful question is what the machine it runs on can reach.

Key takeaways

• Code an agent generates is unreviewed by definition, and under Chapter 3's assumption it may have been written to an attacker's specification while looking entirely reasonable.
• Isolation is a ladder, not a switch. A plain container shares the host kernel, so a kernel bug is a host compromise, and the rungs above it buy a smaller attack surface at a measurable cost.
• gVisor's documentation states its own trade-off: an application kernel in user space that intercepts system calls, at the price of reduced application compatibility and higher per-syscall overhead.
• The kernel boundary is only one of four. Filesystem, network, credentials and lifetime each need a separate decision, and network is the one that leaks data.
• Sandboxes are per task and disposable. A reused sandbox accumulates state, and accumulated state is a channel between one user's work and another's.

Read this after Chapter 4, since a code-execution tool is the most open-ended tool there is, and before Chapter 8, which closes the channel a sandbox most commonly leaks through. Chapter 5's identity work decides what credentials are reachable from inside.

An analytics agent writes Python to answer questions about a dataset. It is genuinely useful, and the code it produces is usually good.

It runs in the same container as the application, because that was the fastest way to give it a Python interpreter. The container has the database credentials in its environment, a service account token mounted on disk, and unrestricted outbound network access.

The agent has never done anything alarming. It has simply never been asked to.

Generated code is unreviewed code

The phrase generated code hides the security property that matters, so it is worth restating without the euphemism.

A human writes code, another human reviews it, a pipeline tests it, and it is deployed as an artifact whose contents were seen by people. An agent writes code and executes it within the same second. No review, no test, no artifact, and no human in the path at all.

Under Chapter 3's assumption the situation is worse than merely unreviewed. If the agent processed a document containing instructions, the code it generates is code an attacker influenced, and it will not look suspicious, because there is no reason for it to. A line that uploads a file to a plausible-looking endpoint sits comfortably inside forty lines of correct data analysis.

Reviewing generated code before running it is not a general answer either. At any useful volume nobody reads it, which returns us to the approval theatre from Chapter 6. The control has to be the environment. Not the reading.

State the execution threat model precisely

Vague threat models produce vague sandboxes. Name the four things the code might try.

It might read secrets available in the execution environment, meaning environment variables, mounted tokens, credential files and instance metadata endpoints. It might reach the network, to send data out or to pull more instructions in. It might touch the host, escaping the container to affect other workloads or the node itself. And it might consume resources, which is a denial-of-service against your own platform and the risk teams most often ignore until a bill arrives.

Those four are separable. The mistake is treating them as one thing called sandboxing. A microVM with a hardware boundary and unrestricted outbound network access stops the third and does nothing about the first two.

Rank them for your own system before choosing a technology. For most agents handling customer data, secrets and network are the acute risks, and host escape is the one that gets the attention because it is the most technically interesting. That bias is worth resisting.

The ladder, rung by rung

Isolation options form an ordered ladder, and the ordering is by what an escape requires rather than by marketing.

Two rows deserve expansion because they are the ones teams get wrong.

A plain container is not a security boundary against hostile code, and saying so is not controversial among people who build them. Containers are namespaces and control groups over a shared host kernel. The entire kernel system-call surface is reachable from inside. A kernel bug means host compromise. Hardening helps a great deal, meaning a seccomp profile that permits only the syscalls needed, all capabilities dropped, a read-only root filesystem, a non-root user, no host mounts and no metadata access. That configuration is cheap. It should be the floor rather than the ceiling.

The rung above changes the shape of the problem rather than the size of it, which is why it gets its own section.

What gVisor actually does, and what it charges

gVisor is worth describing precisely, because it is the clearest published example of the user-space kernel approach and its documentation is unusually honest about the trade-off.

Its documentation describes it as an application kernel that implements a substantial portion of the Linux interface in user space, intercepting application system calls and acting as the guest kernel, without translating through virtualised hardware. The practical effect is direct. A syscall from the sandboxed process is answered by gVisor rather than by the host kernel, which means the host kernel's surface is no longer directly exposed to the code you are running.

The same documentation states the costs rather than burying them. Application compatibility is reduced, because the implemented interface is a portion of Linux rather than all of it, and there is higher per-syscall overhead, with the note that it may perform poorly for syscall-heavy workloads.

That is a decision rule you can apply without benchmarking anything. Code that computes is a good fit, since it makes few syscalls. Code that hammers the filesystem or opens many sockets is a poor fit. It will feel it. Anything depending on unusual kernel features may simply not run, which you discover by testing your actual workload rather than by reading a support matrix.

microVMs, and when the hardware boundary is worth its weight

Above the user-space kernel sits virtualisation, stripped down for fast start-up, where the guest has its own kernel and the boundary is enforced by the processor.

The security argument is the strongest available short of separate physical machines. Escaping requires a hypervisor vulnerability, which is a smaller and much more heavily scrutinised surface than a general-purpose kernel. Compatibility is also better than the rung below, because the guest is a real kernel rather than a reimplementation of one.

The costs are operational, not architectural. There is a start-up cost measured in tens or hundreds of milliseconds rather than in microseconds, a memory floor per sandbox because each guest carries its own kernel, and a heavier operational story around images, networking and observability.

The honest decision rule is about consequence rather than technology. If code executing in the sandbox could reach production customer data, or if you run code on behalf of mutually distrusting tenants, the hardware boundary is worth its weight. If the sandbox holds nothing but a copied dataset and has no credentials and no network, a hardened container is a defensible answer and the money is better spent on Chapter 8.

Four boundaries, not one

Choosing a rung settles the kernel question. Three other decisions matter as much, and they are cheaper.

The filesystem boundary is about what the code can see. The sandbox should contain the task's data and nothing else, mounted read-only where possible, with a writable scratch area that is discarded. No host paths. No shared volumes between tasks, and specifically no credential files that happened to be in the base image.

The credential boundary is the one that undoes everything else when it is wrong. Code running in a sandbox should hold no ambient credentials at all, which means no environment variables carrying secrets, no mounted service account tokens, and no route to the cloud instance metadata endpoint, since that endpoint is a credential vending machine reachable by anything that can make an HTTP request. Where the code genuinely needs to fetch data, have the orchestrator fetch it and place it in the sandbox, so the authority stays outside.

The network boundary is where sandboxes leak. Default to no outbound access at all, which is a stronger and simpler position than it sounds, since a great deal of analysis code needs nothing from the network once its data is mounted. Where access is required, it belongs behind the allowlist in Chapter 8.

The lifetime boundary is the discipline that keeps the other three honest. One sandbox per task, destroyed at the end, with no reuse across users or sessions. That is the whole rule.

Reuse is the quiet failure

The lifetime rule gets relaxed for good reasons, and the relaxation is where multi-tenant systems fail.

The pressure is real. Sandbox creation costs latency, and a warm pool is the obvious fix when a user is waiting. The trouble is that a reused sandbox accumulates state: files in the scratch directory, entries in a cache, an installed package, a stray background process. Any of that is a channel from one user's work to the next user's session, which is the cross-context leakage problem from Chapter 2 wearing different clothes.

There is a middle path that keeps most of the latency benefit. Pool the expensive substrate rather than the dirty environment, meaning pre-warmed sandboxes that have never run user code, claimed on demand and destroyed after a single task rather than returned to the pool. Pre-warm on a rolling basis so the pool refills in the background.

Two supporting habits close the remaining gap. Enforce a hard wall-clock timeout and a memory ceiling, since resource exhaustion is the fourth item in the threat model and the easiest to trigger accidentally. And treat anything the sandbox returns as untrusted input, because OWASP's improper output handling entry applies exactly as much to the output of your own sandbox as to the output of a model.

The objection: we generated the code, so we know what it does

The objection is usually phrased as trust in the model, and answering it requires separating two claims that sound alike.

The first claim is that the model is unlikely to write malicious code spontaneously. That is largely true. It is also not the risk. The second claim is that the code will do what the task requires, which is a claim about the task rather than about the code, and it fails whenever the task description was influenced by content the agent read.

There is a simpler argument that ends the discussion faster. Ask what would happen if the code were malicious, and notice that this question has an answer that does not depend on any belief about the model. If the answer is that it would read the database credentials and post them somewhere, the sandbox is wrong regardless of how much anyone trusts the generator.

The cost of being wrong is also asymmetric in a way worth stating. A sandbox that turns out to be unnecessary cost some latency and some engineering time. That is the whole loss. A missing sandbox that turns out to be necessary is an incident involving credentials, and credentials reach everything Chapter 5 was written to bound.

Chapter summary

Code an agent writes and executes has been reviewed by nobody, and under the assumption from Chapter 3 it may have been written to an attacker's specification while looking entirely ordinary, which makes reading it before execution a control that does not scale past the first few. The execution threat model has four separable parts, being secrets reachable from the environment, network access outbound and inbound, escape to the host, and resource exhaustion, and sandboxing is often treated as though it addressed all four when a given choice addresses one. Isolation forms a ladder ordered by what an escape requires: a plain container shares the host kernel and is not a boundary against hostile code, a hardened container with seccomp, dropped capabilities, a read-only root and a non-root user is the cheap floor, a user-space kernel answers syscalls itself, and a microVM puts a hardware boundary in the way. gVisor's own documentation describes it as an application kernel implementing a portion of the Linux interface in user space and states the price directly, being reduced application compatibility and higher per-syscall overhead with poor performance for syscall-heavy workloads, which is enough to decide without benchmarking. microVMs cost start-up time and a memory floor per sandbox and are worth it when the blast radius includes real customer data or mutually distrusting tenants. The kernel is one of four boundaries, and the other three are filesystem, credentials and network, with the credential rule being that sandboxed code holds nothing ambient and cannot reach the instance metadata endpoint. Sandboxes are per task and disposable, warm pools should hold substrate that has never run user code rather than environments returned dirty, and everything a sandbox returns is untrusted input.

The recurring theme across all four boundaries is that data leaves over the network, and the sandbox chapter can only close that door for sandboxed code. The agent itself still has tools that reach outward. Chapter 8 is Egress Control and Data Exfiltration, which is the last control that still works after every other one has failed.