S1 · Customer Support Resolution Agent
Setting. A company deploys an agent that resolves support tickets end-to-end: understand the issue, look up the customer and their orders, take actions (refunds, replacements, escalations) under company policy, and know when to hand off to a human.
Reference architecture
ticket ──► single agent (system prompt: role, scope, tone, explicit criteria)
tools: lookup_customer · search_orders · get_order_status
issue_refund · create_replacement · escalate_to_human
hooks: PreToolUse on issue_refund → block/queue > $500 (policy gate)
PostToolUse on all actions → audit log
loop: stop_reason-driven, iteration cap, explicit escalation triggers
A single agent — support resolution is sequential and context-dependent (the refund decision needs the same context as the diagnosis). Multi-agent options are over-engineering here.
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| Enforce refund policy |
PreToolUse hook gating the tool call — never a system-prompt rule alone |
D1 |
| Route/act on ticket type |
Explicit criteria in the prompt; forced-choice tool with enum if it must be one of N |
D4 |
| Tool for order lookups |
Prescriptive descriptions ("call when user references an existing order...") distinguishing confusable tools |
D2 |
| Lookup fails (DB timeout) |
Tool returns is_error: true + actionable message; agent tells the truth ("can't check right now") instead of "no orders found" |
D2 |
| When to hand off |
Pre-defined triggers: policy gates, confidence floor, N failed attempts, out-of-scope — with structured handoff (state so far) |
D1/D5 |
| Audit requirement |
PostToolUse logging hook — deterministic, not "mention what you did" |
D1 |
Trap table
| Tempting option |
Why it's wrong |
| "Strengthen the system prompt: NEVER refund over $500" |
Prompts guide; systems enforce. Policy needs a hook. |
| "Spawn a subagent per ticket phase (diagnose → decide → act)" |
Phases are dependent and share context; splitting adds handoff loss for zero parallelism. |
| "Return an empty result when lookup fails so the agent stays positive" |
Silent failure — the named anti-pattern. The agent will fabricate a confident wrong answer. |
| "Let the agent decide case-by-case whether a refund needs approval" |
The scenario stipulates a guarantee; model judgment is probabilistic by definition. |
S2 · Code Generation with Claude Code
Setting. An engineering team adopts Claude Code across a shared repository. They need consistent conventions for everyone, connections to internal systems (issue tracker, CI), safe defaults, and workflows that survive long sessions.
Reference architecture
repo/
├── CLAUDE.md # build cmds, architecture map, conventions (lean!)
├── .claude/
│ ├── rules/ # testing.md · style.md · security.md
│ ├── settings.json # permissions allow/deny · hooks (format-on-edit)
│ ├── commands/ # /review-pr · /write-tests (team slash commands)
│ └── skills/ # release-procedure/ … (on-demand expertise)
└── .mcp.json # team MCP servers (tracker, CI) — secrets via ${ENV}
sessions: plan mode for big changes · /compact with state files · --resume
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| Team-wide conventions |
Project CLAUDE.md + .claude/rules/, checked in — project scope overrides personal user files |
D3 |
| Occasional procedures (releases) |
Skill — on-demand loading, not permanent CLAUDE.md tokens |
D3 |
| "All code formatted, always" |
PostToolUse hook on Edit|Write running the formatter |
D3 |
| Everyone gets the tracker server |
Project-scope .mcp.json, committed, env-var credentials |
D2/D3 |
| Big refactor on unfamiliar code |
Plan mode: explore → plan → approve → execute |
D3 |
| Long session hitting context limits |
Persist decisions/progress to files, then /compact; --resume for continuity |
D3/D5 |
Trap table
| Tempting option |
Why it's wrong |
| "Put the 12-step release guide in CLAUDE.md" |
Always-loaded token tax for a sometimes-needed procedure → skill. |
| "Each developer configures the MCP server personally" |
Fails scoped distribution — project .mcp.json exists exactly for this. |
"Commit the tracker API key in .mcp.json (repo is private)" |
Secrets never get committed; env-var expansion is the pattern. |
| "Ask Claude nicely in CLAUDE.md to always run the formatter" |
Guarantee-word ⇒ hook, not prompt-space. |
"/clear when context fills mid-task" |
Destroys working state; /compact (after persisting key state) continues the task. |
S3 · Multi-Agent Research System
Setting. A research product answers complex questions by fanning out subagents that search, read, and synthesize — with every claim in the final report traceable to a source, and failures handled gracefully.
Reference architecture
coordinator (plans, delegates, merges — its context stays lean)
│ Task tool ── spawns N parallel subagents (independent slices)
▼
subagent_i: FRESH CONTEXT — delegation prompt must carry:
goal · needed facts · constraints · tool budget (allowedTools)
└ returns STRUCTURED: {findings: [{claim, source, confidence}],
status: ok|partial|failed, error?}
coordinator: rejects unsourced findings · merges · flags gaps loudly
bounded re-dispatch for failures → final report with provenance
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| Why subagents at all |
Independent parallel slices + context isolation (workers absorb reading; coordinator stays small) |
D1 |
| Delegation prompt |
Self-contained: subagents inherit nothing |
D1 |
| Result format |
Structured schema with sources and confidence required |
D1/D2/D5 |
| Provenance |
Captured at collection, carried through merges — never post-hoc citation |
D5 |
| Worker fails/times out |
Failure returns as structured data; coordinator degrades loudly (report states coverage gaps) |
D5 |
| Tool scoping |
Workers get the minimum (web_search, maybe Read) — not Bash/Write |
D1/D2 |
Trap table
| Tempting option |
Why it's wrong |
| "Pass the coordinator's conversation to each subagent for context" |
Defeats context isolation — the point of the pattern; and Task-tool subagents get fresh context by design. |
| "Have the final pass add citations to the finished report" |
Post-hoc citation invents sources; provenance is captured at collection. |
| "If any worker fails, restart the whole research task" |
Contain, don't cascade: keep the successes, re-dispatch the failures (bounded), flag remaining gaps. |
| "Workers return prose essays for the coordinator to interpret" |
Unmergeable, unverifiable, and failure becomes invisible — structured returns or nothing. |
| "One agent per report section even though sections build on each other" |
Dependent steps don't parallelize; fan-out is for independent slices. |
S4 · Developer Productivity with Claude
Setting. An organization uses Claude (Code) for day-to-day engineering: exploring unfamiliar codebases, answering "how does X work?", making scoped changes, and long multi-hour investigations that must not lose their thread.
Reference architecture
explore: Glob (find files) → Grep (find usages) → Read (targeted files)
— read-only tools, in that funnel order; Bash only when no tool fits
change: Edit (targeted diffs) over Write (wholesale rewrites)
long runs: notes.md / progress.json scratchpads, updated as facts are learned
→ survive /compact and session loss (pairs with --resume)
guardrails: permissions allow read-only trio broadly; gate Bash & Write
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| "Where is this function used?" |
Grep (dedicated, permissionable, parallel-safe) — not Bash(grep ...) |
D3 |
| Understanding before editing |
Read the relevant files first; plan mode for big/risky work |
D3 |
| Multi-hour investigation state |
Scratchpad files on disk — not "keep it all in the conversation" |
D5 |
| Tool results flooding context |
Targeted reads (specific files/ranges) rather than dumping directories |
D5 |
| Small fix in a known file |
Just do it — plan mode everywhere is overhead; judgment, not ritual |
D3 |
Trap table
| Tempting option |
Why it's wrong |
| "Use Bash for everything — it can cat/grep/find" |
Dedicated tools are individually permissionable & auditable; Bash is the escape hatch, not the default. |
| "Paste the whole repo into context so nothing is missed" |
Context is a budget; funnel (Glob→Grep→Read) curates it. |
| "The conversation is the memory for a 6-hour task" |
Compaction/crashes eat it; durable state lives in files. |
| "Plan mode for every one-line change" |
Reliability ≠ maximal ceremony; the exam rewards matching process weight to risk. |
S5 · Claude Code for Continuous Integration
Setting. A team embeds Claude Code into CI: automated PR review, test generation, failure triage — running headless, with no human present to approve anything, on a budget, with machine-readable outputs.
Reference architecture
CI step:
claude -p "<task prompt>" \
--output-format json # parse the envelope, never scrape prose
--allowedTools "Read,Grep,Glob" # pre-authorized MINIMUM (no human to approve)
--max-turns 20 # bounded run: stuck ⇒ fail fast, not hang
env: ANTHROPIC_API_KEY from CI secrets
exit code gates the pipeline step; downstream parses JSON result
iterate: e.g. generate tests → run tests → feed failures back (bounded rounds)
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| No human to approve tools |
--allowedTools pre-authorizes the exact minimum; deny-by-default posture |
D3 |
| Pipeline consumes results |
--output-format json; never regex the free text |
D3 |
| Runaway protection |
--max-turns + CI timeout — fail fast and visibly |
D3/D5 |
| Secrets |
CI secret store → env var; never in the repo or prompt |
D3 |
| Iterative refinement (tests that must pass) |
Loop: generate → run → feed specific failures back → bounded retries → red pipeline on exhaustion |
D4/D5 |
| Review comments must cite files/lines |
Structured output contract (JSON schema of findings), validated before posting |
D4 |
Trap table
| Tempting option |
Why it's wrong |
| "Reuse the interactive flow; CI will answer permission prompts" |
Nothing answers prompts in CI — headless must be fully pre-configured. |
| "Give CI full tool access so it never gets blocked" |
The opposite of least privilege, in the least supervised environment. |
| "Parse the answer out of the plain-text output" |
Free text drifts; the JSON envelope is the contract. |
| "Retry the whole job until the tests pass" |
Unbounded retry burns budget and hides real failures — bounded feedback loops, then fail red. |
S6 · Structured Data Extraction
Setting. Thousands of documents (invoices, forms, contracts) must become clean database rows: schema-conformant, business-rule-valid, cheap at volume, with bad documents surfaced — never silently dropped or silently wrong.
Reference architecture
per document:
extraction tool (input_schema = target row; enums for closed sets)
+ tool_choice = {"type": "tool", "name": "record_extraction"} # forced
→ Pydantic validation (formats, cross-field rules: items sum to total)
→ on failure: retry WITH the validation error in the prompt (2–3 max)
→ still failing: human review queue with errors attached
at volume:
Message Batches API (50% cost, ≤24h) · custom_id per document
→ per-request result handling: succeeded / errored / expired
→ resubmit only failures; dead-letter after N rounds
The decisions the exam probes
| Decision |
Anthropic-aligned answer |
Domain |
| Guarantee output shape |
Schema-enforced via forced tool — not "respond only with JSON" |
D4 |
| Business rules (totals add up) |
Code validation (Pydantic) after generation — schemas can't do cross-field math |
D4 |
| Failed validation |
Feedback retry (specific error in prompt), bounded, then human queue |
D4/D5 |
| 50k documents nightly |
Batch API — latency-tolerant bulk is its exact profile |
D5 |
| Batch results handling |
Correlate by custom_id (any order); handle per-request outcomes; resubmit failures only |
D5 |
| A document that can't be parsed |
Explicit failure artifact (review queue + errors) — never a silently skipped or half-filled row |
D5 |
Trap table
| Tempting option |
Why it's wrong |
| "Tighten the JSON instructions in the prompt" |
Prompt-begging: format guarantees come from schemas + forced tools. |
| "Put 'total must equal sum of items' in the JSON schema" |
JSON Schema can't express cross-field arithmetic — that's code's job. |
| "Blind-retry failed extractions" |
Without the error fed back, the retry re-rolls the same dice. |
| "Live API with 200 parallel workers to finish faster" |
Pays 2× for latency nobody needs — the Batch trade-off question in disguise. |
| "Skip unparseable documents to keep the pipeline green" |
Silent data loss — the worst reliability outcome. Fail loudly into review. |
Cross-scenario patterns — the meta-lessons
Look at what repeated across all six trap tables:
- Guarantee-words → structural mechanisms. never / always / must / audit ⇒ hooks, permissions, schemas, forced tools. Prompt-space options are the perennial distractors.
- Failures are data, and they are loud.
is_error: true, structured failure returns, partial-coverage flags, review queues. Every "keep it smooth / stay positive / skip it" option is wrong.
- Bounded everything. Loop caps, retry budgets,
--max-turns, then explicit escalation. "Retry until it works" never wins.
- Least privilege everywhere.
allowedTools, permission denylists, minimal tool surface, read-only workers, CI minimums.
- Context is a budget with durable backup. Curate what enters; persist decision-critical state to files; compaction is lossy by design.
- Right-sized architecture. Single agent until independence + context pressure justify fan-out; plan mode for risk, not ritual; Batch for latency-tolerant bulk, live API for humans waiting.
If you internalize these six, you can usually eliminate two options on sight and adjudicate the remaining two with the specific domain fact.
Next: the quiz's Timed-mock mode — 60 questions, 4 scenario blocks, timed.