Add Structured Logging

Before diving in, complete Spec-as-Test so each log line connects to a known expected output and failures are immediately traceable to a specific spec criterion.

Your pipeline fails at step 14 with a 'null reference' error. You spend 20 minutes debugging step 14. The actual cause: at step 7, an MCP call returned an empty object because the file path had a trailing slash. Every step from 7 to 14 silently passed malformed data forward until step 14 finally broke. Without a trace linking all 14 steps, that root cause is nearly impossible to find.

Structured logging is the fix. Each agent action writes one JSON line: timestamp, trace ID, step number, tool called, one-line input summary, one-line output summary, error if any. A trace ID is a single UUID generated at run start and attached to every log line from that run. When a run fails, you filter by trace ID and read the lines in order. The real failure point is where the output summary first shows an unexpected value, not where the exception fired.

Here is the before and after: a five-step deploy pipeline writes one JSON line per step, all sharing trace_id a3f9c2d1. Step 5 throws on a missing DATABASE_URL. Reading the trace from step 1 shows step 3 (fetch_env_vars) returned an empty object. The real failure is the missing .env.staging file, two steps upstream of where the exception fired.

The instrument-first habit: before you scale any task, add logging. Three lines of structured output per step. You will find the step-7 trailing-slash bug in 90 seconds. Without it, you spend 30 minutes on step 14.

Paste this into Claude:

I want to add structured logging to an agent workflow I have. Here is the workflow or skill: [paste your skill definition or describe a multi-step task you run regularly]. Help me add structured logging to it. For each step, log: (1) a trace ID (generate one UUID per run at the start), (2) the step number, (3) the tool called, (4) a one-line input summary, (5) a one-line output summary or error. Format each log line as JSON. Then show me what a trace would look like for a run that fails at step 3 because the tool returned an empty result.

What good looks like:

• Every step in your workflow now writes a structured JSON log line
• A single trace ID links all log lines from one run
• You can filter all log lines for one run using just the trace ID
• You simulated a failure mid-run and can identify the failure step from the trace alone, without reading the full output
• You can read the trace in sequence and name the step where malformed data first appeared

What a good response looks like:

Structured trace for a 5-step deploy workflow (run trace_id: a3f9c2d1):

```json
{"trace_id":"a3f9c2d1","step":1,"tool":"fetch_config","input":"path=/deploy/config.yaml","output":"config loaded: 12 keys","error":null,"ts":"2026-04-26T14:00:01Z"}
{"trace_id":"a3f9c2d1","step":2,"tool":"validate_schema","input":"config: 12 keys","output":"validation passed","error":null,"ts":"2026-04-26T14:00:02Z"}
{"trace_id":"a3f9c2d1","step":3,"tool":"fetch_env_vars","input":"env=staging","output":"{}","error":null,"ts":"2026-04-26T14:00:03Z"}
{"trace_id":"a3f9c2d1","step":4,"tool":"merge_config","input":"config+env","output":"merged: 12 keys (env vars: 0)","error":null,"ts":"2026-04-26T14:00:04Z"}
{"trace_id":"a3f9c2d1","step":5,"tool":"deploy","input":"merged config","error":"TypeError: Cannot read properties of undefined (reading 'DATABASE_URL')","ts":"2026-04-26T14:00:09Z"}
```

Reading the trace: step 3 returned {} (empty env vars). Step 4 silently merged with zero env keys. Step 5 tried to read DATABASE_URL and failed. Root cause: step 3, not step 5. Fix: check why fetch_env_vars returned empty for env=staging (missing .env.staging file).