Blueprints BP-01
Exception-handling agent with human review
Exceptions in a high-volume process are cleared by hand.
01 / Fit
Use it when
- Exceptions arrive in volume and most follow a small number of known shapes.
- The facts needed to decide sit in contracts, rate cards and past decisions.
- Policy can state which actions are safe to take without a person.
- There is a team that already reviews exceptions and can take the hard ones.
Avoid it when
- The exception volume is low enough that a person clears the queue without strain.
- Nobody can write down the approval policy. The guardrail has nothing to enforce.
- The system of record cannot accept an idempotent write or produce an audit record.
02 / The diagram
Run it. Then run the failure path.
Select a node to see what it does, how it fails and what we measure.
Read the diagram as text
- Trigger (ERP exception). Picks up the exception event from the ERP. Fails when: Events are duplicated or arrive out of order. Safeguard: One idempotency key per exception.
- Retrieve (Contracts, rates). Pulls contract terms, rate cards and prior decisions for the case. Fails when: The relevant document is missing or stale. Safeguard: Freshness check. An empty result goes to human review.
- Agent (Plan and act). Plans the resolution, calls tools and proposes an action with its reasoning. Fails when: A tool errors or the model is unsure. Safeguard: Step and time limits, then the deterministic fallback.
- Guardrail (Policy check). Checks the agent’s proposed action against policy before anything is written. Fails when: The policy is stale, or an amount was misread upstream. Safeguard: Anything over the approval limit goes to a person. Every block is logged with its reason.
- Write (Post to ERP). Posts the approved action to the ERP with an audit record. Fails when: The write partly fails. Safeguard: Transactional write, retried with the same key.
- Fallback (Fixed rules). Takes over when the agent cannot finish: holds the case, attaches what was found and routes it to a person. Fails when: A rule no longer matches how the process runs. Safeguard: Rules are versioned and replayed against the golden set.
- Human review (Approve or edit). A person approves, edits or rejects, with the agent’s summary attached. Fails when: The queue backs up. Safeguard: Alert on queue age. Decisions feed the golden set.
- Evals and tracing (Logs every step, replays the golden set before each release). Logs every step and replays the golden set before each release. Fails when: The golden set drifts from real cases. Safeguard: Refresh it from reviewed cases on a schedule.
03 / Components
Every component, and how it fails.
| Component | Job | Typical choices | Fails when | Safeguard |
|---|---|---|---|---|
| Trigger | Picks up the exception event from the ERP. | Event stream or change feed from the ERP, with idempotent consumers | Events are duplicated or arrive out of order. | One idempotency key per exception. |
| Retrieve | Pulls contract terms, rate cards and prior decisions for the case. | Document index over contracts and rate cards, plus a store of prior decisions | The relevant document is missing or stale. | Freshness check. An empty result goes to human review. |
| Agent | Plans the resolution, calls tools and proposes an action with its reasoning. | Tool-calling language model with step and time limits | A tool errors or the model is unsure. | Step and time limits, then the deterministic fallback. |
| Guardrail | Checks the agent’s proposed action against policy before anything is written. | Rule engine with versioned policy and approval limits | The policy is stale, or an amount was misread upstream. | Anything over the approval limit goes to a person. Every block is logged with its reason. |
| Write | Posts the approved action to the ERP with an audit record. | Transactional ERP API call with an idempotency key and audit log | The write partly fails. | Transactional write, retried with the same key. |
| Fallback | Takes over when the agent cannot finish: holds the case, attaches what was found and routes it to a person. | Deterministic rules service, versioned and replayable | A rule no longer matches how the process runs. | Rules are versioned and replayed against the golden set. |
| Human review | A person approves, edits or rejects, with the agent’s summary attached. | Review queue with approve, edit and reject actions | The queue backs up. | Alert on queue age. Decisions feed the golden set. |
| Evals and tracing | Logs every step and replays the golden set before each release. | Step-level tracing store and a golden-set replay harness | The golden set drifts from real cases. | Refresh it from reviewed cases on a schedule. |
04 / Budgets and evals
What the design is tuned for.
Auto-resolution rate, time to human decision. These are design targets, not results.
| Stage | Design target |
|---|---|
| Retrieve | Fresh context or none. An empty result is cheaper than a stale one. |
| Agent | Bounded steps and time, so a stuck case fails fast into the fallback. |
| Guardrail | Rule-based, so it adds no model latency and gives the same answer twice. |
| Human review | Short time to decision. The summary and evidence arrive with the case. |
Eval plan 01
Golden set
Reviewed exceptions with the action a person approved, covering each exception type, each fallback cause and cases just above and below the approval limit.
Eval plan 02
Release gates
Correct-action rate on the golden set, policy violations that reach the write step, which must be none, and fallback rate by cause.
Eval plan 03
Catching regressions
Every prompt, tool or policy change replays the golden set before release. Human overrides in production are sampled into the set on a schedule.
05 / Design notes
Where the whiteboard and production differ.
- Duplicate and out-of-order events are more common than the event-bus documentation suggests.
- The approval policy is rarely written down in one place. Encoding it for the guardrail is often the first time it is stated in full.
- Auto-resolution rate rises quickly on the common exception types and slowly on the rest. The long tail stays with people for longer than planned.
- A review queue with no owner turns the fallback into a backlog. Queue age needs an alert and a named team.
- Partial writes to the ERP are the failure that damages trust most, so the write step deserves more testing than the agent.
06 / Deploy
Deploy this with us.
A pod can take BP-01 from this page to your stack. Tell us the workflow and we will tell you what changes for your systems.
You will talk to an engineer