Graduated autonomy for AI agents: the traffic-light model

Most production AI agent systems operate in one of two modes: full autonomy or kill switch. The agent either runs freely — calling tools, writing to databases, sending messages — or it gets shut off entirely because something went wrong.

Both extremes are expensive. Full autonomy without guardrails accumulates agentic technical debt and generates high stochastic tax through uncontrolled escalations and retries. The kill switch wastes the investment in the agent and blocks every workflow that depends on it. The middle ground — graduated autonomy — is where most production value lives, and where most teams have the least governance.

As we argued in Harvard Business Review, deploying an AI agent is not a software installation — it is a workforce decision. You would not give a new employee signing authority before they have demonstrated judgment. You would define their role, bound their authority, assign approved sources of truth, and set clear escalation rules. Agents need the same structure. Graduated autonomy is the framework that provides it.

Key takeaways

• Graduated autonomy assigns different levels of agent independence based on the risk and reversibility of each action.
• The traffic-light model (green/yellow/red) provides a framework that teams can implement immediately and regulators are converging on.
• Autonomy should be earned through demonstrated competence, not granted by default and revoked after failure — the same way you onboard a new team member.
• Dynamic demotion — automatically reducing autonomy when monitoring signals degrade — is the safety mechanism that makes the model viable at scale.

What graduated autonomy means

Graduated autonomy is a governance control from the agentic technical debt framework (forthcoming in Communications of the ACM, October 2026). Instead of treating agent autonomy as binary — on or off — it classifies every agent action by risk level and assigns a corresponding permission tier.

The principle: low-risk, reversible actions run autonomously; high-risk, irreversible actions require human authorization; everything in between runs within explicit constraints.

This is not a new idea in security. Role-based and relationship-based access control systems have enforced tiered permissions for decades. What is new is applying the concept to non-deterministic agents whose behavior varies across runs. A deterministic system with read-only access is safe by construction. An agent with read-only access can still produce a response that causes a human to take the wrong action. The tiers must account for both what the agent can do directly and what the agent can influence.

The traffic-light model

The framework proposes three tiers, mapped to risk tolerance.

Green: autonomous execution

Green actions are low-risk and reversible. The agent executes them without human review.

Examples:

• Classifying a support ticket by topic
• Routing a request to the correct queue
• Retrieving information from a knowledge base
• Generating a draft response for human review
• Logging and reporting on workflow metrics

What makes an action green: the downside of a wrong answer is small, the action can be undone or corrected cheaply, and the scope is narrow enough that the agent’s non-determinism does not produce dangerous variation.

Yellow: constrained execution (propose-only mode)

Yellow actions are medium-risk. The agent can proceed, but within explicit guardrails — budget limits, scope constraints, or a downgrade to propose-only mode where the agent recommends an action and a human approves it. Think of yellow as the agent’s probationary period for a given skill: it demonstrates competence under supervision before earning full autonomy.

Examples:

• Scheduling a payment under a dollar threshold
• Modifying a customer record within predefined fields
• Sending a templated email (agent selects the template but cannot write freeform)
• Approving a refund up to a cap
• Executing a database query that reads but does not write

What makes an action yellow: the action has real consequences but bounded ones. A wrong answer costs money, time, or trust — but not more than the guardrails allow.

Red: human-authorized

Red actions are high-risk and irreversible. The agent surfaces a recommendation; a human authorizes execution.

Examples:

• Writing to a production database
• Changing ERP master data
• Sending a freeform message to a customer
• Modifying access permissions
• Executing a financial transaction above a threshold
• Deploying code to production

What makes an action red: the action is hard or impossible to reverse, the downside of a wrong answer is severe, or regulatory requirements mandate human authorization.

The workforce analogy: onboard agents like employees

The traffic-light model maps directly to how organizations onboard human employees — a parallel we explored in detail in a Harvard Business Review article with Rahul Telang and Zia Hydari. The structural parallel between employee onboarding and agent deployment runs across the entire lifecycle:

Employee onboarding stage	Agent equivalent
Background check and screening	Evaluation against golden-set test cases
Identity badge and building access	Agent identity, API credentials, tool schema contracts
Spending authority limits	Yellow-tier budget caps and scope constraints
Code of conduct	System prompt, safety guardrails, prompt injection defenses
Probationary period	Propose-only mode with human review
Performance reviews	Golden-set evaluation and monitoring metrics
Record-keeping and audit trail	Trace logging and observability infrastructure
Delegation authority	Child agents receive a strict subset of parent permissions

The key insight: every agent deployment is a workforce decision, not a software installation. Organizations that treat it as software installation — flip a switch, grant broad access, fix problems later — accumulate the same risks they would face by hiring an employee with no background check, no spending limits, and no supervision.

This reframing also explains why AI pilots often reveal organizational weaknesses rather than creating new ones. When teams try to define what an agent should and should not do, they discover that the same ambiguity exists for their human workflows — undocumented processes, unclear escalation paths, permissions granted by habit rather than policy. Graduated autonomy forces the clarity that was always needed.

The economics of tier assignment

Getting the tiers right is not just a safety question — it is a cost question. Each tier has a different stochastic tax profile:

Tier	Tax profile	Risk
Green	Low — token and monitoring costs only	Mis-classification wastes human attention on safe tasks
Yellow	Medium — guardrail enforcement + occasional escalation	Over-constraining kills throughput; under-constraining creates exposure
Red	High — every transaction incurs human review cost	Necessary for high-stakes actions, expensive if applied too broadly

The optimization target is to have as many actions at green as the evidence supports, while keeping genuinely high-risk actions at red. Every action unnecessarily at red is human review cost you did not need to pay. Every action prematurely at green is risk exposure you did not intend to accept.

A single action promoted from yellow to green on a workflow running 10,000 transactions per month eliminates $2,000–8,000 in monthly human review cost, depending on escalation frequency and reviewer hourly rate. Multiply that across a dozen agent skills and the graduated autonomy framework becomes one of the highest-ROI governance investments a team can make. Conversely, an action that should be red but is classified green can generate a single incident that costs more than a year of human review would have.

Earned autonomy, not default autonomy

A common mistake is to start agents at green and demote them to yellow or red after a failure. This is backwards. Failures in production carry real costs — customer impact, financial loss, compliance risk. The safer pattern is to start restrictive and promote.

The Digital Apprentice framework formalizes this as “earned, not assumed” autonomy:

• New agents start at red (or yellow at most) until they have demonstrated competence on a sufficient sample of real tasks.
• Promotion is per-skill, not per-agent. An agent may be green for ticket classification and red for payment processing. The skills are promoted independently based on evidence.
• Promotion requires evidence. A golden-set evaluation with high pass rates, a period of monitored operation with low escalation rates, and explicit sign-off before the tier changes.
• Demotion is automatic. When monitoring signals degrade — a spike in retries, a drop in evaluation scores, an unfamiliar tool-call pattern — the system demotes the skill to a higher-oversight tier without waiting for a human to notice.

This asymmetry is deliberate. Promotion is a considered decision backed by data. Demotion is a circuit breaker that fires immediately when quality degrades. The agent can earn its way back up, but it cannot fail its way into autonomy.

The autonomy ladder — progressing from full human review through retrieval with guardrails, supervised actions, and finally bounded autonomous execution — provides a more granular progression path within the traffic-light tiers. An agent does not jump from red to green. It climbs a ladder within yellow, each rung requiring evidence that the previous level was handled reliably.

Implementing the traffic-light model

Step 1: classify your agent’s action space

List every action the agent can take — every tool call, every database write, every external API request, every message it can send. For each action, answer three questions:

1. Is it reversible? Can the action be undone within a reasonable time window without user impact?
2. What is the downside of a wrong answer? Financial loss, customer impact, compliance violation, reputational harm?
3. Is there a regulatory requirement for human authorization?

Map each action to green, yellow, or red based on the answers. When in doubt, start one tier higher than you think necessary.

Step 2: define guardrails for yellow actions

Yellow is where the design work lives. For each yellow action, define the constraints:

• Budget limits: the agent can approve refunds up to $50, schedule payments up to $500.
• Scope limits: the agent can modify these customer fields but not those.
• Template limits: the agent can select from approved templates but cannot compose freeform.
• Rate limits: the agent can execute this action N times per hour before escalating.

These constraints must be enforced in code, not in the prompt. A prompt that says “do not exceed $500” is a suggestion the model may or may not follow. A tool schema contract that rejects a tool call over $500 is a guarantee. This is the distinction between harness-level and loop-level controls — the harness enforces hard limits, the loop handles reasoning within them.

Step 3: build the escalation path

When an action exceeds its tier — or when the agent is uncertain — it must escalate rather than guess. The escalation path needs:

• A clear handoff format. The agent should present what it recommends, why, and what information it used — not just “I need help.”
• A routing rule. Who gets the escalation? A general queue, a domain expert, an on-call rotation?
• A timeout. If no human responds within a time window, what happens? The request waits, the agent retries with a more conservative approach, or the task is canceled.

Step 4: instrument and monitor

Track three metrics per action tier:

• Autonomous completion rate (green actions): what percentage complete without escalation or retry? A declining rate suggests the action may need reclassification or the agent is experiencing prompt drift.
• Guardrail hit rate (yellow actions): how often does the agent reach a constraint? Frequently hitting a budget cap may mean the cap is too low — or that the agent is poorly calibrated.
• Escalation approval rate (red actions): what percentage of escalations result in the human approving the agent’s recommendation? A consistently high approval rate is evidence the action could be promoted to yellow.

These metrics feed the promotion and demotion decisions. They also feed directly into your stochastic tax calculation — escalation cost is one of the eight tax components, and graduated autonomy is how you manage it.

Government frameworks are converging on this model

The traffic-light model is not just good practice — it is becoming the regulatory expectation.

Singapore’s Model AI Governance Framework for Agentic AI — released in May 2026 and the first government-issued framework specifically for agentic systems — defines a four-level autonomy taxonomy and introduces Agent Identity Cards that disclose an agent’s capabilities, limitations, authorized domains, and escalation protocols.

The NIST AI Agent Standards Initiative recommends six control layers for agent governance, with least-privilege authorization — graduated autonomy by another name — as a core requirement.

Teams that implement tiered autonomy now will be ahead of compliance requirements rather than retrofitting under regulatory pressure.

Common mistakes

Making every action yellow

Yellow requires defining constraints for each action. Teams that avoid the classification work often dump everything into yellow, which means every action requires some level of oversight. This defeats the purpose — the value of graduated autonomy is that green actions run without friction, freeing human attention for the red ones that actually need it.

Putting guardrails in the prompt instead of code

A prompt instruction like “never exceed $500” is not a guardrail — it is a suggestion that the agent may or may not follow, depending on the model, the context, and whether a recent model update changed the model’s interpretation. Real guardrails are deterministic checks in the agent harness that reject invalid actions before they execute.

Forgetting to re-evaluate tiers

The action landscape changes. A task that was red when the agent was new may be ready for yellow after three months of monitored operation with consistently high golden-set scores. A task that was green may need to be bumped to yellow after a model update changes behavior. Schedule a quarterly review of your tier classifications — the same way you would review access control policies.

Granting broad access and narrowing later

This is the “default autonomy” anti-pattern. Teams grant the agent access to every tool and database, then try to restrict it after observing problems. The inverse — start with minimum-viable access and expand as the agent earns trust — is safer, cheaper, and produces better evaluation data because you know exactly which capabilities the agent was exercising at each stage.

Where graduated autonomy fits

Graduated autonomy is one of five governance controls in the agentic technical debt framework. It targets the autonomy mechanism — the principal–agent gap between what you intended and what the agent does — by making the boundaries of acceptable action explicit, enforceable, and adaptive.

It works alongside golden-set evaluation (catching behavioral regressions), tool schema contracts (enforcing interface discipline), model gateways (absorbing provider changes), and workflow graph redesign (fixing structural problems).

The question to ask today: for every action your agent takes in production, do you know whether it is green, yellow, or red? If you cannot answer that — or if the answer is “everything is green” — graduated autonomy is where to start.

For the full framework, see Agentic technical debt: a governance framework for AI agents.