AI Automation Observability: Logs, Alerts, and Replay When a Job Goes Wrong
A workflow without traces is a rumor. Keep logs you can search, alerts that page a human, and a replay path that does not double-charge. OpenTelemetry names the three signals. n8n executions are the practical store.
Eval says the contract is sound. Production still lies. A node times out, a token expires, the model returns valid JSON that is the wrong customer. If you cannot find the run, you will “fix it” by clicking around and creating a second charge. Observability is the habit that makes failure boring.
Do not start here. Threat-model first (AI automation security), then evaluate before production. Approvals still live in human-in-the-loop n8n jobs.
Three signals, even if the vendor is n8n
OpenTelemetry’s docs name traces, metrics, and logs as the core signals. You can store them in n8n executions plus a Slack message. The names still help you not forget one. Source: OpenTelemetry observability primer (checked 16 September 2026).
| Signal | Question | Small-studio store |
|---|---|---|
| Trace | Which nodes ran, in what order, for this job? | n8n execution graph |
| Metric | How often did schema_fail or timeout happen this week? | A weekly count in a sheet, or n8n insights if you have it |
| Log | What was the input ID and the error message? | Execution data + Error Trigger payload |
sequenceDiagram
participant J as Job
participant E as Execution log
participant A as Error workflow
participant H as Human
J->>E: write node results
J-->>A: fail
A->>H: alert with execution ID
H->>E: inspect
H->>J: replay if idempotentn8n: executions, retry, Error Trigger
n8n keeps executions so you can inspect and retry. Settings exist to save success, save failure, or save nothing (do not save nothing on a write workflow). Error workflows must start with an Error Trigger node. They receive the failed execution’s data, and they do not run on a manual test click, only when an automatic execution fails. Sourced from n8n docs: View executions, Error Trigger, Handle errors gracefully (checked 16 September 2026). Confirm the Error Trigger field names in your n8n version; they have moved between releases.
n8n expression
{{
'Workflow: ' + $json.workflow.name +
'\nExecution: ' + $json.execution.id +
'\nLast node: ' + $json.execution.lastNodeExecuted
}}Slack text
FAIL ai-intake
execution: 18422
node: HTTP Request
wrote: unknown. Inspect before retryHypothetical Error Trigger handler: page a human with IDs, never with the full customer payload.
If the last node was a write, treat retry as dangerous until you check the far side. Idempotency keys (a unique job ID the CRM rejects as duplicate) are how replay becomes safe. Without them, replay is a coin flip.
What to log (and what the threat model forbids)
- Always: job ID, workflow version, model name, schema pass/fail, HTTP status, duration.
- Sometimes: redacted input hash, last four of an account number, URL path without query secrets.
- Almost never in Slack: full prompts, identity documents, card data, health notes. Log those only in a store with retention you chose, or do not log them.
A weekly pass that is not “open n8n when it hurts”
- Count failed executions per workflow. Anything above “rare” gets an owner.
- Open three failures. Confirm the alert contained an ID you could click.
- Replay one only if the write is idempotent. If you are unsure, do not replay. Re-run from a staging credential.
- If you cannot explain a failure in five minutes, the log is too thin. Add the field before you add a new workflow.
Filled hypothetical: double email
Hypothetical. A timeout after the email node sent, the Error Trigger retried the whole graph, the customer got two “we got your request” notes. The fix is not a nicer prompt. It is: email node last, or a sent flag in the sheet that skips the node on replay. Observability made the duplicate visible. Idempotency would have prevented it.
What this does not prove
A green execution list is not a correct business outcome. Schema-valid JSON can still be the wrong company. Human-in-the-loop (HITL) still sits on writes that spend money. This page only makes failure inspectable.
If you want the error path designed before the happy path ships, send the workflow name and what a write does.
Frequently asked questions
What is observability here?
The ability to ask why a run failed without guessing. OpenTelemetry (OTel) describes three signals: traces (a request’s path), metrics (counts and timings), and logs (events with context). You do not need a full OTel stack on day one. You do need those three ideas attached to each job.
Where does n8n keep this?
Executions. n8n stores each run, success or fail, with node results (unless you turned saving off). You can retry from history. Error Trigger starts a dedicated error workflow when another workflow fails. That is the alert path, not a hope that someone opens the UI.
What should an alert contain?
Workflow name, execution ID, failed node, timestamp, and whether a write may have happened. “Something broke” is not an alert. It is a vibe.
Is retry the same as replay?
No. Retry is n8n running the failed execution again. Replay is you deciding it is safe to run again because the write is idempotent (same input cannot double-charge or double-email). If the HTTP POST is not safe, retry is how you duplicate a customer invoice.
Should I log prompts and completions?
Log IDs, model name, token counts if you have them, schema pass/fail, and a redacted snippet. Full prompts often contain customer data. The threat model spoke is the privacy pass. Observability does not override it.
Do I need Datadog on a five-workflow studio?
No. n8n execution history, an Error Trigger to email or Slack, and a sheet of execution IDs you actually opened this week is enough until volume says otherwise. Buy a vendor when you cannot search last Tuesday.