Agree or Disagree

Pre-execution AI monitoring can substantially reduce catastrophic risk when dangerous action sequences are detectable and approval cannot be bypassed.

An AI agent’s ability to devise a dangerous plan does not necessarily give it the ability to carry that plan out. If its consequential external actions require approval from a sufficiently reliable monitor, and it cannot bypass that requirement, supervision can interrupt the plan before catastrophic harm occurs.

This claim is conditional. It concerns deployments where the monitor can obtain enough information to identify dangerous behavior while there is still an opportunity to prevent its consequences. It does not assume that every possible catastrophe has an easily recognizable precursor.

The first requirement is that review precedes execution. The acting agent proposes an action; a separate mechanism holds it pending approval. Rejection prevents execution, and a missing or delayed decision does not count as approval. Under this arrangement, the agent cannot defeat supervision merely by acting faster than a human could react.

The second requirement is dependable judgment. A monitor must distinguish acceptable actions from dangerous ones even when the acting agent tries to deceive it. There are reasons to expect that a narrowly tasked, equally capable reviewer can be made sufficiently reliable for some deployments: it can have a restricted role, limited authority, and no responsibility for achieving the acting agent’s objective. These advantages support a hypothesis to test, rather than a guarantee that another model instance will be trustworthy.

The third requirement is enough context to evaluate developing plans. Individually innocuous actions may collectively create a dangerous situation. A monitor therefore needs access to relevant history, commitments, and changes in the environment. It must sometimes reject an action because of what that action contributes to a larger sequence, even if the same action would be acceptable elsewhere.

Consider an agent managing data-centre construction. It might propose bypassing a safety inspection to meet a deadline. A monitor could reject that proposal before an instruction reaches the contractor. A subtler plan might distribute the same violation across scheduling changes, altered records, and separate messages. Preventing that plan would require the supervisor to connect those actions and intervene before unsafe work begins. The example illustrates both the opportunity for prevention and the inadequacy of reviewing isolated messages alone.

The fourth requirement is enforcement outside the acting agent’s control. The agent must not be able to use an unmonitored account, delegate execution to an unrestricted system, change an approved action after review, or modify the approval mechanism. A monitor’s correct judgment has protective value only if its decision constrains what actually happens.

When these requirements hold, a dangerous plan must survive both the agent’s own execution difficulties and an additional opportunity for detection and rejection. The size of the resulting risk reduction depends on how often supervision interrupts plans that would otherwise succeed. It cannot be inferred simply from the number of actions reviewed or the percentage approved.

Nor should repeated review automatically be counted as multiple independent chances to catch a plan. If every decision relies on the same mistaken assumption, the monitor may approve an entire dangerous sequence. Evidence of substantial protection must therefore come from evaluating complete adversarial attempts, including attempts to deceive the monitor over time.

The strongest objection is that a sufficiently capable agent could make its plan appear harmless until intervention is too late. If that is true for a particular deployment, this argument does not establish that the deployment is safe. The practical question is whether its permissions and scope can be restricted so that dangerous plans encounter a detectable, enforceable checkpoint before irreversible harm. Where they cannot, pre-execution monitoring is insufficient.

My conclusion is that catastrophic-risk assessments should account for the constraints imposed by the deployed supervision system. Showing that an unsupervised agent could cause catastrophe does not establish that the same agent could do so under effective pre-execution control. Establishing that such control will be reliable—and that frontier labs will implement and preserve it—requires further evidence.

Conditional argument about a complete approval gate and reliable review. Sources:

OpenAI, Running Codex safely at OpenAI: https://openai.com/index/running-codex-safely/

Kutasov et al., Evaluating Control Protocols for Untrusted AI Agents: https://arxiv.org/abs/2511.02997

Kutasov et al., SHADE-Arena: https://arxiv.org/abs/2506.15740

0

Agree

0

Disagree

0 people have voted.

Log in to vote.

Given in support of