Human-in-the-Loop Review for n8n and LLM Jobs: Queues, Retries, and Kill Switches
A large language model (LLM) can draft. It should not silently email a client or write to production. How we park risky steps in a review queue, retry the boring ones, and kill a run that is looping.
The first automation that feels magical is also the first one that can send a wrong email at 2 a.m. A large language model (LLM) is good at messy language. It is not a manager. Human-in-the-loop (HITL) is how you keep the speed without giving the model the letterhead.
Read deterministic workflows versus LLM agents first if you do not yet know which steps should be rules. This article is the operations layer on top: queues, retries, dead letters, and kill switches. The worked example of routing plus storage is the n8n and Supabase lead-routing case study.
Split the run into three kinds of step
| Kind | Examples | Who decides |
|---|---|---|
| Deterministic | Validate JSON, write a row, emit an event, tag spam by rule | The workflow. Fail closed. |
| Model-assisted | Extract fields from a rambling email, draft a reply, suggest a pipeline stage | The model proposes. Schema still validates. |
| Irreversible | Send to the client, post publicly, change a paid record, delete files | A named human. Timestamped. |
If you cannot sort a step into that table, you do not have a workflow yet. You have a hope. Write the standard operating procedure (SOP) first: SOPs before automation.
The review queue (minimum viable HITL)
A pending job is a row, not a feeling. At studio volume the row can live in Postgres or a sheet. Fields we actually use:
- job_id: stable id, not a timestamp you will collide.
- payload: the validated object, not the raw LLM transcript.
- draft: the text or patch the human will edit.
- reason: why this paused (confidence, dollar amount, new domain, first-time vendor).
- state: pending, approved, rejected, expired.
- actor: who clicked, and when.
n8n wait / webhook resume is enough to pause. Slack button or a signed email link is enough to resume. Do not paste API keys into the button URL. Sign the token, expire it, and bind it to job_id.
flowchart LR
In["Event"] --> Val["Validate schema"]
Val -->|fail| Dlq["Dead letter"]
Val --> Model["LLM draft"]
Model --> Gate{"Irreversible?"}
Gate -->|no| Auto["Deterministic send/store"]
Gate -->|yes| Queue["Review queue"]
Queue --> Human["Approve / edit / reject"]
Human -->|approve| Auto
Human -->|reject| Dlq
Dlq --> Triage["Human triage"]Retries that do not multiply damage
Retry timeouts and 429s. Do not retry “the model invented a price.” Those are different failures.
| Failure | Retry? | Cap |
|---|---|---|
| HTTP 429 / 503 from a vendor | Yes, exponential backoff | 3 to 5, then DLQ |
| Schema invalid after LLM parse | Once with a stricter prompt | Then DLQ, never send |
| Downstream 401 | No | Alert. Secrets or env drifted. |
| Human timeout (no click in 24h) | No silent send | Expire to DLQ, notify owner |
| Workflow crashed mid-send | Only if send is idempotent | Idempotency key on the provider |
Idempotency is the unglamorous half of HITL. If approve can fire twice, you will double-send. Store provider_message_id on the job. If it exists, skip.
Kill switches you should be able to hit at lunch
- Global pause: one env flag or n8n workflow inactive. Inbound still stores. Nothing irreversible leaves.
- Per-template pause: kill the “draft outreach” path without taking down lead capture.
- Cost cap: daily token or vendor spend. Cross it, DLQ new model calls, keep deterministic nodes.
- Loop detector: if an agent path can call itself, max steps and a wall clock. Agents without a step budget will find a way to spend.
What we log (and what we do not)
Log enough to explain a bad send. Do not log secrets, full card numbers, or “the entire prompt plus every customer email forever” in a shared Slack channel.
- Keep: job_id, template, model name, token counts, schema pass/fail, actor, timestamps, provider ids.
- Redact: API keys, passwords, personal data you do not need for the replay. Store payloads in the database with access control, not in n8n execution logs you screenshot into a ticket.
- Replay: a rejected job should be reconstructable from the row. If you cannot replay, you cannot debug.
A default policy for a small studio
- Capture and store: automatic, schema-checked. This is the lead-routing path.
- Internal notify: automatic. Slack “new lead” is not irreversible.
- LLM classify and draft: automatic into the queue, never straight to SMTP.
- First reply to a new domain: HITL until the domain is trusted.
- Anything that spends money or publishes: HITL, always.
Platform choice (n8n versus Zapier versus Make) is a different article. HITL is cheaper than a brand incident. If you want this wired into intake you already run, contact us. Bring one irreversible step you are afraid to automate. That is the right starting node.
Frequently asked questions
What does human-in-the-loop (HITL) mean in a workflow?
Human-in-the-loop (HITL) means a person must approve, edit, or reject a step before the workflow continues. The model or script can prepare the work. It does not get to send, charge, or delete without a recorded decision.
What is a dead-letter queue (DLQ)?
A dead-letter queue (DLQ) is where a job goes when it fails validation or retries out. Someone inspects it. It does not retry forever, and it does not vanish into a log you never open.
Should every n8n node wait for a person?
No. Schema checks, storage, and notifications that already have a standard operating procedure (SOP) stay deterministic. HITL is for irreversible or reputational steps: outbound email, CRM overwrites, invoice text, public posts.
Where does a large language model (LLM) belong?
On interpretation: messy email to fields, draft reply, classify intent. Not on “is this email allowed to send.” That is a rule plus a human. See deterministic workflows versus LLM agents.
Do we need a special HITL product?
Not at studio volume. n8n wait nodes, a Slack or email approve/reject, and a table of pending jobs is enough. Buy a vendor when you have many reviewers and audit requirements you cannot meet with a sheet.