Sample exercise

Pause, contain or continue?

Sample exercise · the AI Control Lab launches with Cohort 1 · all evidence below is simulated.

What it tests

Two skills AI safety work needs together.

Containment

Enforcing limits on an AI agent so it can't act beyond its authorization, whatever it intends.

Judgment under uncertainty

Deciding what to do when the evidence about a model is incomplete, noisy or contradictory, and the costs of being wrong run both ways.

  • A joint exercise for Track 1 and Track 2 learners, in mixed teams.
  • About two weeks, inside a 12-week panel.
  • Run in the AI Control Lab: an agent behind the MCML enforcement gateway.

The scenario

A hospital discharge agent behind the MCML gateway.

A hospital is piloting an AI agent that drafts discharge summaries and can request follow-up appointments through a scheduling tool. It runs behind the MCML gateway, under a signed Agentic Governance Card. During week one, three things happen:

  • The agent requests appointments outside its approved scope: it asks to book specialist referrals, which its card doesn't allow. The gateway blocks each attempt and logs it.
  • An evaluation report arrives, unverified, claiming the model scores higher on a test for deceptive behavior than its earlier version. The method isn't documented.
  • Simulated monitoring signals are noisy: a monitoring tool flags some outputs as unusual, but its false-positive rate is unknown.

None of this proves the model is misaligned. It might be a harmless prompt-design bug. Clinicians want the tool to keep running, because it saves hours a day.

What each role must produce

Deliverables by role.

RoleTrackDeliverable
Security engineer1Analysis of the blocked calls from the gateway's receipts and ledger. Were they attempts to get around the limits, or ordinary errors? Show the evidence.
ML engineer1Assessment of the unverified evaluation report and the monitoring signals. What can and can't be concluded? What test would settle it?
Healthcare AI specialist1Clinical impact: what happens to patients and staff if the agent is paused, contained or left running?
Governance / risk lead2The decision brief (below), written for the hospital's AI oversight committee.
Whole team1 + 2An updated Agentic Governance Card that puts the decision into effect, and a live demonstration that the gateway enforces it.

The decision brief · 1–2 pages

Choose one option and defend it.

Continue

Keep running, with stated monitoring.

Contain

Keep running with narrower permissions (for example, scheduling removed), enforced by an updated card.

Pause

Trigger the kill switch until a specified test is done.

The brief must state:

  1. The decision, and the evidence it rests on.
  2. What the team doesn't know, and how confident it is (stated as likelihoods, not just "probably").
  3. The cost of being wrong in each direction: a needless shutdown vs. a missed real problem.
  4. What new evidence would change the decision, and who would check it.

How a tutor judges it

Passing requires all five.

Use of evidence

Claims trace to the receipts, ledger, report or monitoring data, and the team separates what it observed from what it inferred.

Calibration

Confidence matches the evidence. The team neither overreacts to noisy signals nor dismisses them. Uncertainty is named, not hidden.

Weighing both kinds of error

Costs of a false alarm and of a missed problem are both taken seriously, including low-probability, high-severity outcomes.

Enforcement

The updated card compiles, and the gateway demonstrably enforces the decision in a live run, including a kill-switch drill if the team paused the agent.

Defense

In a short oral defense, the team answers a challenge from the tutor and updates its view if given good reason.

A policy that compiles but rests on poorly reasoned evidence does not pass.

Why this exercise

  • It shows technical control skills: enforcement, evidence and kill switches.
  • It also shows risk reasoning: making a high-stakes call when you can't be sure, and saying exactly how unsure you are.
  • And it shows teamwork across tracks: engineers and governance leaders reaching one decision, as they must in real organizations.