AI Agents
An AI agent is a named, permission-scoped worker you configure once. It wakes on a trigger, gathers the context available for that trigger, and writes up what it found. Every run is recorded, and — unless the agent is in Act mode and the action is on its pre-approved list — anything it wants to do is proposed for a person to approve.
Agents are off by default and must be enabled by your operator before they appear. See Enabling agents.
What an agent does
Section titled “What an agent does”Manage agents under Settings → AI Agents.
An agent has a kind: triage, patch, or helpdesk. The kind decides what the agent is for and what triggers it. Triage agents accept alerts and opted-in anomaly incidents. Helpdesk agents accept newly created tickets.
Each agent carries its own limits — devices per run, concurrent runs, runs per hour, turns per run, spend per run and per day, wall-clock time, and a ceiling on the percentage of your fleet it may touch in a day. These are enforced per run, not advisory.
When a trigger passes admission, the agent opens a run. The run is the audit surface: open it to see what triggered the agent, which devices it looked at, and what it concluded.
Runs are triggered by an alert, an anomaly, a schedule, a ticket, or manually from the agent itself. A run moves through these states:
| Status | Meaning |
|---|---|
queued |
Accepted and waiting for a runner. |
running |
Actively working. |
awaiting_approval |
The agent proposed an action and is waiting on a person. |
completed |
Finished. |
failed |
Stopped on an error. |
cancelled |
Stopped by a person. |
expired |
Exceeded its wall-clock budget. |
skipped |
A run record was created but stopped before executing, for example when the agent’s policy was revoked between admission and start. |
Admission checks happen before Breeze creates a run. If an agent policy filters out a trigger, no skipped run appears in the runs list. See Why nothing happened for the skip signals that are recorded instead.
Approving what an agent proposes
Section titled “Approving what an agent proposes”When an agent wants to take an action, it does not perform it by default. It records the proposal and the run moves to awaiting_approval. You are notified in the app, and the proposal appears in the Approvals inbox, where you approve or reject it.
An agent in Act mode is the one exception: for a small set of pre-approved operations, it executes directly instead of recording a proposal. See Act mode. Everything else, in every mode, always proposes and waits. Rejecting a proposal ends it; the agent does not retry the same action behind you.
Act mode
Section titled “Act mode”Act mode lets an agent execute a fixed, closed set of low-risk, narrowly-scoped operations — restarting a service, running an already-authorized script, executing a disk-cleanup plan it already previewed, running a built-in playbook — without a person approving each one in the moment. Anything not on that list still becomes a proposal in the Approvals inbox, exactly as in Shadow mode. The set is not configurable per agent beyond allowlisting which of those operations it may reach; you cannot add arbitrary tools to Act mode’s unattended list.
Act mode is bounded by more than the operation list:
- Guardrails re-check every call. Each tool call an agent makes is classified before it runs. A call outside the fixed set — or one that fails a live re-check just before execution (the target device doesn’t match the run, the script’s content has changed since it was authorized, a required preview no longer exists) — is denied outright or downgraded into an ordinary proposal, never silently executed.
- One device per run. A run only ever acts on the single device it was triggered against.
- A per-run action cap. A run stops taking unattended actions once it hits its configured limit, in addition to the run-level limits (concurrency, spend, wall-clock time) every agent already has.
- An organization-wide exposure ledger. Every unattended action Act mode takes is recorded against the organization it ran in, alongside the equivalent record for the separate, not-yet-generally-available policy-decide capability. This ledger is the foundation for a shared, org-wide cap on unattended activity across all of an organization’s agents.
- Verification, not certainty. Act mode checks the result of what it did (a service is running again, disk usage improved) and reports if that check fails — but it does not roll anything back. If verification fails, a person resolves it, the same as they would resolve any other failed run.
Turning Act mode on
Section titled “Turning Act mode on”In Settings → AI Agents, set the agent’s mode to Act. The API refuses the change unless the agent already has both a recipient that resolves to a real person (Act mode still needs someone to notify) and at least one act-eligible operation in its Tool allowlist (for scripts, at least one authorized script). The form shows an explicit warning — unattended execution, no rollback, single-device scope, the action cap — and, the first time you switch an agent into Act mode, requires you to check an acknowledgement before it lets you save.
Turning it off
Section titled “Turning it off”Set the agent’s mode back to Off or Shadow, or disable the agent — either stops unattended execution for that agent immediately. Disabling BREEZE_AI_AGENTS_ENABLED (see Enabling agents) is the platform-wide switch: it stops every agent, in every mode, for the whole deployment.
Graduation (feedback loop)
Section titled “Graduation (feedback loop)”Graduation is how an agent earns narrower supervision by evidence rather than by an operator’s optimism. Breeze records the outcome of every operation an agent completes, and once a single operation has a long enough clean record you can ask a second person to pre-authorize it — so future runs take that one action without waiting in the Approvals inbox.
Open Settings → AI Agents, edit an existing agent, and scroll to Graduation. The panel only appears on an agent that already exists; there is nothing to show while you are creating one.
What gets recorded
Section titled “What gets recorded”Every operation an agent finishes produces exactly one piece of evidence, per outcome:
| Outcome | What it means |
|---|---|
| Executed | The agent actually attempted the operation and the execution reported success. |
| Verified | It executed and the follow-up check confirmed the result held — or no follow-up check applied to it. This is the count graduation is measured on. |
| Failed | An attempted failure only. A proposal that was refused, blocked by a live guardrail re-check, or stopped before anything ran is not a failure — nothing was tried. |
| Recurred | The follow-up check saw the same condition come back after the agent had fixed it. |
Thumbs-up and thumbs-down on an alert verdict are recorded the same way. They do not feed graduation — a thumbs-up is a supervision signal, not proof the verdict was right — but they are reported on the AI impact page.
The panel reads a rolling 30-day window. If an operation was previously demoted, the window starts at the demotion instead, so a bad record cannot be outlived by evidence that predates it.
When an operation becomes eligible
Section titled “When an operation becomes eligible”Each row shows the operation, its state, and the verified, failed, recurred and first-verified figures behind that state. States are Tracking, Eligible, Promoted and Demoted.
An operation becomes Eligible only when all of these hold:
- The partner-wide agent allows that operation.
- 20 verified executions in the window. This is the promotion threshold, tunable per partner; an organization can raise it but never lower it below the partner’s figure.
- The first verified execution is at least 14 days old — volume alone is not enough, the record has to have lasted.
- Zero failures and zero recurrences in the same window.
- The operation is one that can be pre-authorized at all. Not every action is.
When a row is not eligible, the Why not yet column says which of those it fell on — “The partner-wide agent does not allow this operation yet”, “Not enough verified executions yet”, “The first verified execution is still too recent”, “A failure or a recurrence was recorded in this window”, or “This operation can never be pre-authorized”.
Below the graduation table, an act-mode reliability table shows how Act mode’s own operations have been performing — executed, verified, failed and recurred. It is read-only by design: those are not policy operations, so they can never be pre-authorized and never carry a Promote button.
Promoting an operation
Section titled “Promoting an operation”Promotion is never automatic, and never something an agent can do for itself.
-
In the Graduation panel, click Promote on an Eligible row. The button only appears on an Eligible row, and only when pre-authorized execution is switched on for the deployment.
-
Confirm. This raises a four-eyes approval request — see Four-eyes actions. A different person has to approve it; the requester cannot.
-
Once approved, the operation is added to that organization’s agent and its graduation row moves to Promoted.
The same request can be raised straight from the Approvals inbox. On an eligible card, Approve and always allow approves the request in front of you and raises the pre-authorization request in one step. It only appears on supervised cards whose operation is already eligible, and never on a critical-risk one.
Promotion depends on the deployment-wide pre-authorized-execution setting (BREEZE_AI_AGENTS_POLICY_DECIDE_ENABLED, off by default and not yet generally available). With it off the panel still loads and still shows the evidence — it says “Pre-authorized execution is turned off for this deployment; these figures are informational” — but no operation can be promoted.
Demotion is automatic
Section titled “Demotion is automatic”Demotion is always on and does not depend on any setting.
The first attempted failure of a pre-authorized operation, or a recurrence after the agent fixed something, removes that operation from the organization’s agent immediately, in the same step that records the failure. Breeze notifies the agent’s recipients at high priority, naming the agent and the operation and saying whether the last attempt failed or the alert it fixed came back.
Demotion touches nothing else. The agent stays enabled, its mode is unchanged, its other operations are unchanged, and the partner-wide baseline is never edited. The graduation row moves to Demoted and keeps a record of the promotion that preceded it.
Re-promotion is not automatic. The row returns to Tracking as soon as a fresh verified execution lands after the demotion, and it has to earn eligibility again from there — the window now starts at the demotion, so the pre-demotion record cannot count twice.
Agent activity in the audit trail
Section titled “Agent activity in the audit trail”Actions that originate from an agent are attributed to the agent, not to the person who configured it or happened to be signed in. Agent-created entries in the audit log carry the agent as the actor, so an agent’s work is never mistaken for a technician’s during a review.
Permissions
Section titled “Permissions”Agents are gated on their own capabilities rather than on general organization permissions:
| Permission | Grants |
|---|---|
ai_agents:read |
View agents and their runs. |
ai_agents:write |
Create, edit, disable, and trigger agents. |
These are deliberately separate from organizations:write. Authoring an agent policy is what authorizes autonomous action on customer machines once an agent is switched to Act mode, so it is not bundled into a permission every organization admin already holds. Partner Admin retains access through its full-access role.
Creating, editing, disabling, or manually triggering an agent additionally requires you to complete MFA.
Choosing what an agent can do
Section titled “Choosing what an agent can do”The agent form’s own Permissions section groups tools into capabilities — fixed groups like alerts and monitoring, files and disk, or scripts and commands — that the catalog assigns to every tool an agent can reach. Only capabilities and operations an agent can actually reach are offered; a tool the agent runtime doesn’t expose is never shown, no matter how you search for it. Read-only tools (device lookups, reports, and the like) are always available to every agent and need no decision — they’re listed separately, collapsed behind a count of how many.
Use recommended adds the operations the agent’s kind typically needs on top of whatever is already selected; it never clears anything.
Every mutating operation carries one outcome badge, for the agent’s current mode:
| Badge | Meaning |
|---|---|
| Approval request | Tier 3. Recorded as a proposal; a technician decides in Approvals. |
| Logged proposal | Tier 2. Recorded in the run’s outcome and never executed. |
| Executes unattended | Act mode only, and only for the fixed, act-eligible operations described in Act mode. Run a script is the one exception: it stays an approval request until at least one script is authorized for the agent (the Scripts allowed to run unattended list on the Safety step / edit drawer — a partner row’s list is the ceiling, an organization’s agent picks a subset of it), and the picker and review card say so rather than promising an unattended run. |
A green dot marks an operation that can be pre-authorized — eligible, on a clean enough record, to graduate out of the approval queue (see Graduation). The dot means eligible, not authorized: nothing is pre-authorized until a second person promotes it. On an organization’s agent, an operation its partner-wide baseline hasn’t granted is marked Not in partner baseline and its checkbox is disabled — unless it’s already selected, in which case it stays enabled just long enough to be unchecked, so a grant made before the baseline narrowed can still be removed. The partner row is a ceiling on what the organization may be granted, not a starting grant of its own. Pre-authorized operations can no longer be added to an organization’s agent directly in this form: its own supervised-keys section is a read-only list of what it already holds, not a checkbox — a key becomes eligible only by promoting an Eligible row in Graduation or the four-eyes request it raises, never by an operator’s checkbox.
Turn on Show tool names to see the literal key behind each operation. Selections are stored on the agent as tool:action entries — one per operation, even when every current operation of a tool is selected, so an operation added to that tool later is never silently granted; a bare tool entry appears only for a tool with just one operation to begin with. An entry the picker can’t represent — an unknown or unreachable tool, or a hand-typed bare entry on a multi-operation tool — is listed under Unrecognised or unreachable entries with a reason and a remove action; it is never dropped or rewritten silently.
The same catalog and ceiling behind the picker are available directly:
GET /api/v1/ai/agents/tool-catalogGET /api/v1/ai/agents/ceiling?kind={kind}Alert triage wiring
Section titled “Alert triage wiring”Creating a triage agent automatically creates a matching system-managed automation that wakes the agent when an alert fires. You will see it on the Automations page, marked as system-managed.
Disabling the agent disables its wiring too, so an agent that is switched off never gets live alerts routed in front of it.
Alert verdicts
Section titled “Alert verdicts”A triage agent can classify an alert without opening a full run. The verdict appears on the alert itself, under AI Verdict, as one of:
| Verdict | Meaning |
|---|---|
| Actionable | Something genuinely needs doing. |
| Self-healed | The condition cleared on its own. |
| Recurring pattern | Part of a repeating daily, weekly, or post-event pattern rather than a new problem. |
| Duplicate | The same underlying condition as other alerts already grouped together. |
| Needs human review | The agent could not reach a confident conclusion. |
Each verdict shows a confidence percentage and the agent’s reasoning. Use Hide AI-flagged noise on the alert list to filter out the ones the agent judged non-actionable — the alerts are suppressed from your view, never deleted.
Verdicts are advisory. If the agent thinks an action is warranted it does not take it: the alert shows AI suggested an action — review in Approvals and the proposal waits for a person.
Thumbs-up and thumbs-down buttons on a verdict record whether it was useful. One person’s feedback per verdict is kept.
Scheduled sweeps
Section titled “Scheduled sweeps”A sweep is a scheduled pass over your fleet looking for conditions nobody has alerted on yet. Configure them under Settings → AI Agents → Scheduled sweeps.
Choose which checks a sweep runs:
- Disk pressure
- Stale agents
- Pending reboots
- Failed backups
- Service down
- Unpatched critical vulnerabilities
Schedules are set with a five-field cron expression and a timezone — for example 0 3 * * * for every day at 03:00.
Findings appear on the run under Sweep findings, listed by check, severity, and device. Where the agent wants to act on a finding it raises an approval request rather than acting. If the fleet was too large to examine exhaustively, the run says so rather than implying it saw everything.
Weekly report
Section titled “Weekly report”A Weekly report schedule has an agent write a plain-language summary of what happened in an organization over the past week. It appears in Reports as Weekly AI Operations Narrative, marked Managed by AI schedule.
Set it up the same way as a sweep, choosing Weekly report as the schedule type. It must fire exactly once a week — a specific minute and hour on a single day, for example 0 7 * * 1 for every Monday at 07:00.
Ticket helpdesk lane
Section titled “Ticket helpdesk lane”A newly created ticket can wake a helpdesk agent when all of these conditions are met:
BREEZE_AI_AGENTS_ENABLED=true.- An enabled partner-level baseline agent of kind
helpdeskexists. An organization-level agent cannot enable itself without this baseline. - The effective policy is enabled for both the partner and the organization, and its mode is not Off.
- The ticket matches the optional
triggers.ticketCategoriesandtriggers.ticketPrioritiesfilters. An absent filter is unrestricted.
Every ticket creation is written to a transactional outbox. A background publisher drains the outbox onto the event bus approximately every five seconds, so you do not need a separate ticket trigger flag.
Ticket-triggered runs have no device context, so device-changing tools are denied outright.
By default a ticket run only ever proposes: it writes an AI-drafted reply or an AI-drafted resolution note onto the ticket for a technician to deal with. The draft appears in the ticket with two buttons — Send as me, which posts it as a comment under your name, and Discard, which throws it away. Nothing reaches the customer unless a technician sends it.
You can let a helpdesk agent post ticket updates without waiting for a technician, but it takes two deliberate opt-ins together: the agent must be in Act mode, and the organization must turn on Autonomous ticket writes (an org-level override on the agent’s policy). If either is off — or if the kill switch is engaged — the update goes to a person for approval instead. Both are re-checked at the moment of the write, not just when the run started, so switching either off stops autonomous writes on runs already in flight.
Only ticket creation admits a helpdesk run in this release. Ticket comments and status changes are published on the event bus, but this lane does not read them yet. A ticket that already has agent-originated comment activity is also rejected to prevent a loop.
Anomaly lane
Section titled “Anomaly lane”A newly opened anomaly incident can wake a triage agent when all of these conditions are met:
BREEZE_AI_AGENTS_ENABLED=true.- Anomaly Detection is enabled for the organization through
ml.anomalies.enabled. It is off by default, and the global ML kill switches can still suppress it. See Availability and enabling. - An enabled partner-level baseline agent of kind
triageexists. Triage is the only agent kind that accepts anomaly triggers. - The organization-level triage agent sets
triggers.anomalyEnabledtotrue. Its default isfalse. - The optional narrowing filters match the incident.
The detector collapses raw anomalies into a canonical incident. A background publisher sends each newly opened incident to the event bus, so you do not need another anomaly event flag.
Anomaly-triggered runs are always forced into Shadow mode. They are bound to a device, so device pinning, site scope, and maintenance-window rules still apply normally.
You can narrow admission with triggers.anomalyTypes, triggers.metricNames, triggers.minAnomalyScore, and the usual site and device-tag filters. An absent filter is unrestricted. These filters cannot opt the lane in; triggers.anomalyEnabled is still required.
If the underlying anomaly has already been promoted to an alert, the anomaly run reuses the alert’s deduplication key. One incident therefore does not produce both an anomaly run and a separate alert-triage run.
Opt in through the API
Section titled “Opt in through the API”The Settings → AI Agents form exposes alert severities and the maintenance-window toggle, but it does not show an anomaly control. First list agents and identify the organization-scoped triage agent for the customer:
GET /api/v1/ai/agentsThen patch only the anomaly opt-in:
PATCH /api/v1/ai/agents/{agentId}Content-Type: application/json
{"triggers": {"anomalyEnabled": true}}{agentId} must identify the organization-scoped agent, not the partner baseline. The request requires the ai_agents:write permission and a completed MFA step, as do other agent mutations.
The server merges the triggers object one level onto the stored policy. Fields you omit keep their values, and later edits in the UI do not clear anomalyEnabled because the form submits only its own trigger fields.
Verify the effective policy for the organization:
GET /api/v1/ai/agents/effective?orgId={orgId}&kind=triageConfirm that triggers.anomalyEnabled is true in the response.
Why nothing happened
Section titled “Why nothing happened”When admission is refused, Breeze does not create a run row, so nothing new appears in the runs list. The API always writes a run skipped log line carrying the reason, organization, agent, trigger kind, and device.
| Skip reason | Meaning | Also published as an event |
|---|---|---|
trigger_filter_mismatch |
The anomaly opt-in gate or a narrowing trigger filter did not match. | Yes |
no_effective_agent |
No enabled partner baseline agent of that kind exists. | No, log only |
kill_switch_off |
BREEZE_AI_AGENTS_ENABLED is off. |
No, log only |
Only a skip that happens after the agent has been resolved publishes an ai.agent.run.skipped event. kill_switch_off and no_effective_agent are deliberately log-only: they fire on every trigger in every organization for as long as the condition holds, so publishing them would put a write on the hot alert path for a non-event. If you want to catch those two, alert on the log line rather than the event.
A run that was already admitted can also stop before it starts. If the agent is disabled or its policy is revoked between admission and delivery, the run transitions to skipped with the reason policy_revoked_before_start and publishes ai.agent.run.skipped. Unlike the admission-time skips above, that case does leave a run record in the runs list.
Durable delivery for agent triggers
Section titled “Durable delivery for agent triggers”The event dispatch settings do not enable or disable the ticket-helpdesk or anomaly lanes. Both subscribers are registered when the API starts, and both lanes still run in-process when EVENT_DISPATCH_MODE is unset or off. Naming a subscriber changes the durability of its delivery hop, not whether the feature runs.
In-process delivery is best-effort. If a handler throws, Breeze writes the failure only to the API log, swallows it, and loses the trigger. Queue delivery leaves a delivery receipt and retries a failed hop. The ai-agent-ticket-helpdesk and ai-agent-anomaly subscribers each retry up to five times with a 10-second backoff. ai-agent-alert-verdict retries up to three times with a 30-second backoff.
EVENT_DISPATCH_MODE |
Delivery behavior |
|---|---|
off |
In-process delivery only. This is the default when the variable is unset. |
shadow |
Also mirrors the routing plan into delivery receipts for observability, but executes nothing through the queue. |
enforce |
Routes named subscribers only through the BullMQ queue and skips their in-process delivery. Subscribers that are not named continue in-process. |
EVENT_DISPATCH_QUEUE_SUBSCRIBERS is ignored unless the mode is enforce. To make these trigger hops durable, set the mode to enforce and name ai-agent-ticket-helpdesk and/or ai-agent-anomaly. An empty subscriber list is accepted, but everything remains in-process and the API logs a warning at boot.
See Environment Variables for validation rules and the complete subscriber ID list.
Enabling agents
Section titled “Enabling agents”Agents are off unless your operator sets BREEZE_AI_AGENTS_ENABLED=true. Upgrading does not turn them on, and no existing role gains access to them by default — the ai_agents permissions must be granted to a role explicitly.
Self-hosted operators: see Environment Variables for the flag, and remember it must also be mapped in the api service’s environment: block of your compose file — a value in .env alone has no effect.