Where Human Checks Belong in an AI Workflow

Putting a human “in the loop” sounds responsible, but it does not tell a product team what the person should inspect, what authority that person has, or what happens while the workflow waits.

Some AI-assisted steps are easy to reverse and cheap to repeat. Others send a message, spend money, modify a customer record, disclose information, or make a decision that someone must answer for. Treating both kinds of work the same produces either constant interruption or unsafe automation.

A better design starts with the effect of each step. This guide offers a practical method for placing review, approval, escalation, and hard stops where the risk changes.

Begin with the full workflow, not the model call

Write the workflow as an ordered set of events from trigger to completed outcome. Include data retrieval, transformations, model calls, tool actions, handoffs, waiting periods, and notifications. Many consequential actions occur after the model produces text.

For each step, record:

• the input and its source;
• the action or decision;
• the tool or system involved;
• the possible external effect;
• the role that owns the outcome;
• the evidence the next step receives; and
• the known failure states.

This inventory prevents a common design mistake: reviewing the generated answer while ignoring what a downstream tool will do with it.

The NIST AI Risk Management Framework Core calls for documented roles and communication lines, defined human-AI oversight responsibilities, and post-deployment processes for monitoring, override, incident response, recovery, and change management. That is a useful frame for the workflow as a whole. It is not a requirement to place a person after every step.

Score the point where the effect changes

For every action, ask four questions.

1. How reversible is it?

A private draft can usually be regenerated or discarded. A public post, approved refund, deleted record, or signed submission may be hard to undo. Low reversibility argues for review before the action rather than after it.

2. Who or what is affected?

An internal suggestion has a different exposure from a customer message, hiring decision, credit action, medical instruction, or change to production data. External or rights-affecting outcomes deserve stronger controls and subject-matter review.

3. How visible is uncertainty?

Some steps can be checked against clear rules or source records. Others depend on incomplete evidence, ambiguous language, or a model’s interpretation. A confident tone should not be treated as proof that uncertainty is low.

4. Who is accountable?

The workflow needs a role with authority to accept, reject, or escalate the proposed action. “A human reviews it” is not enough. The reviewer must have the context, permission, and time to make the decision.

These questions can be captured in a compact checkpoint table:

The table does not calculate a universal risk score. It forces the team to explain why a control belongs at a particular boundary.

Choose the lightest control that matches the risk

Human involvement is not one mechanism. Use a control that fits the action.

Observation lets the workflow continue while recording enough information for later inspection. It suits low-impact, reversible steps when a team is still learning how the system behaves.

Sampled review sends a defined share of completed items to a reviewer. It can reveal drift without delaying every run. The sampling rule and response to a defect must be explicit.

Pre-action review asks a person to inspect the proposed output before a later tool acts. The reviewer may edit the draft, request more evidence, or reject it.

Explicit approval pauses a specific tool action until an authorized person accepts or rejects it. The decision and the exact proposed action should be recorded together.

Escalation routes an item to a role with additional authority or expertise. It needs a queue, service expectation, and fallback when no one responds.

Hard stop blocks the workflow because a required condition is absent or the action is outside policy. A hard stop should explain what must change before the run can continue.

The official OpenAI Agents SDK human-in-the-loop documentation demonstrates one implementation pattern: a run pauses when a tool needs approval, exposes the pending call as an interruption, records an approve or reject decision, and resumes from saved state. That pattern is useful because the approval belongs to a specific action, not to a vague review of the whole conversation.

Build a decision packet, not an approval button

A reviewer cannot make a sound decision from “Approve this action?” The approval screen should show the smallest complete packet needed for that decision.

For a customer message, that might include the proposed recipient, subject, body, source records used, policy or template version, and any unresolved warning. For a payment action, it might include the amount, payee, invoice, authorization path, duplicate check, and the exact tool call waiting to execute.

The packet should distinguish source data from model-generated interpretation. It should also show what will happen after approval. If accepting a request sends a message immediately, say so. If it moves the item to another automated step, show that next state.

Keep sensitive fields narrow. A reviewer should not receive an entire customer record when only three fields support the decision. Access to the approval interface is itself an authorization question.

Authenticate the reviewer and bind the decision

An approval link is not proof that the person opening it has authority. The application should authenticate the reviewer, verify that the role may decide this item, and bind the decision to the exact pending action.

If the proposed action changes while it waits, the old approval should not silently apply. A useful implementation records a fingerprint or version of the action, along with the reviewer, time, decision, and optional reason.

Batch approval needs similar care. Ten similar items may be efficient to review together, but the interface should still make differences visible and avoid turning one click into consent for an item the reviewer never saw.

The OpenAI guide also warns that serialized run state can contain tool arguments, approval decisions, and application context. It recommends server-controlled storage, authentication, authorization, validation against pending requests, and protection against replay. Those details matter because a well-placed human checkpoint can still fail if its decision path is weak.

Design for waiting, rejection, and missing reviewers

An approval gate changes the workflow from a straight line into a stateful process. The product must decide what happens while it waits.

Set an owner and a timeout. A low-priority draft may wait until the next workday. A time-sensitive operational issue may need escalation after 15 minutes. If the deadline passes, the workflow should move to a known state rather than approve itself by default.

Rejection should have a route. The reviewer might return the item with a reason, request missing information, select a safer alternative, or end the run. The system should not repeatedly regenerate the same proposal without changing the evidence or instruction that caused the rejection.

Absence also needs a policy. If the only authorized reviewer is away, the workflow might route to a delegated role, remain paused, or stop. The right answer depends on the action. It should be chosen before the first urgent case.

Prevent unsafe retries

External actions can finish even when the workflow does not receive confirmation. A network timeout after “send,” “pay,” or “delete” creates an uncertain outcome. Automatically repeating the call may duplicate the effect.

Mark this state as outcome unknown. Before retrying, reconcile against the destination system using an idempotency key, transaction identifier, message record, or other authoritative evidence. If the system cannot verify the result safely, escalate to a person with the right access.

This is also a reason to place checkpoints before an action, not after it. A reviewer can approve a proposed payment, but a second approval should not be required merely because the confirmation page failed. The workflow needs to determine whether the approved action already happened.

A worked example: renewal outreach

Consider an account team using AI to prepare renewal outreach.

The workflow starts when an account enters a renewal window. It retrieves contract dates, recent support activity, product usage summaries, open commitments, and the account owner’s notes. An AI step drafts an internal account summary and a proposed email.

The summary is reversible and remains internal, so the first runs might use observation plus sampled review. The proposed email is also reversible until it is sent, but it can contain incorrect claims or disclose information. It receives pre-action review by the account owner.

The decision packet shows the recipient, draft, source records, unresolved support issues, and any sentence generated without a direct source. The reviewer can edit the draft, reject it, or approve the exact message for sending.

The send action is external and difficult to reverse. It requires authenticated approval. If the account is marked for legal review, has an unresolved billing dispute, or lacks a verified recipient, a hard stop prevents sending and routes the item to the appropriate queue.

After the tool attempts delivery, the workflow records the provider’s message identifier. If the request times out without a result, it checks for that identifier before any retry. A bounced message returns to the account owner rather than triggering an automatic rewrite and resend.

This design does not put a person after every retrieval or formatting step. It concentrates human attention where the workflow turns private analysis into external communication.

Monitor the system after launch

Checkpoint design is not finished when the workflow ships. Teams should inspect overrides, rejections, escalations, timeout states, repeated edits, incidents, and cases where reviewers approve without meaningful attention.

The NIST Generative AI Profile includes suggested actions for periodic incident monitoring, post-incident review, communication of incidents and performance, risk-proportionate independent evaluation, incident ownership, rehearsals, and alerts that prompt human intervention. These suggestions support an ongoing operating practice, not a one-time checklist.

Review the checkpoint when:

• the model, prompt, tool, or source data changes;
• an action gains a new external effect;
• reviewers consistently make the same edit;
• a queue becomes slow enough that people bypass it;
• an incident reveals missing evidence or authority; or
• a previously rare exception becomes normal.

A checkpoint can move in either direction. Better validation may allow a sampled review to replace a full review for a low-impact step. A new customer-facing action may require stronger approval.

A practical checkpoint worksheet

For one workflow, complete this sequence:

1. List every step from trigger to confirmed outcome.
2. Mark each external, rights-affecting, financial, destructive, or public action.
3. Mark actions whose outcome may be hard to reverse or verify.
4. Record uncertainty and the evidence available to a reviewer.
5. Assign an accountable role with actual authority.
6. Choose observation, sampling, review, approval, escalation, or hard stop.
7. Define the decision packet and allowed responses.
8. Define waiting, timeout, rejection, delegation, and recovery states.
9. Prevent replay and unsafe retries.
10. Record the decision and monitor how the checkpoint performs.

Run the worksheet with product, operations, security, and the people who will make the decisions. A checkpoint that looks sensible in a diagram may be impossible in a real queue.

Keep accountability visible

Good AI workflow design does not ask humans to bless every output. It makes the consequential boundaries visible, gives the right person enough evidence to decide, and preserves a safe state when the answer is no or not yet.

That is a more useful standard than “human in the loop.” The question is where judgment changes the risk, how the system supports that judgment, and what the workflow does next.