1. The context window is a budget
Everything competes for the same finite window: system prompt, tool definitions, conversation history, tool results, and the answer being generated. Architect-level thinking treats it as a budget to be allocated, not a bucket to be filled:
- Spend deliberately: a 3,000-token CLAUDE.md or 40 verbose tool definitions is a tax on every single call.
- Big consumers: raw tool results (whole files, full API responses) dominate long agentic sessions. Return the relevant slice, not the dump — filter in the tool, not in the model.
- Symptoms of a blown budget: earlier instructions "forgotten," degraded mid-conversation quality, truncation, forced compaction at the worst moment.
Exam angle: options that "just include everything, the window is huge" lose to options that curate. Bigger windows raise the ceiling; they don't repeal the budget.
📊 Diagram — “Context window budget stack with durable side-channel state” (a version appears earlier on this page)
2. Lost in the middle & placement
Models attend most reliably to the beginning and end of the context; material buried in the middle of a long prompt is recalled worst — the "lost in the middle" effect.
Placement rules that show up as answer options:
- Critical instructions: at the start (system prompt) — and for very long contexts, restate the key question/instruction after the reference material, near the end.
- Long documents: document first, question after it (question lands in the high-attention tail).
- Don't sandwich the one fact that matters between 60k tokens of logs and expect consistent recall — surface it explicitly (or extract it to a summary the model sees late).
3. Compaction vs decision-critical context preservation ★
Passers report this as one of the most-tested Domain 5 ideas, in exactly this framing:
"Don't just compress context. Preserve decision-critical context."
Compaction/summarization is lossy. A generic summary keeps the gist and drops precisely the things a long-running agent later needs. The tested checklist of what must survive any compression:
| Must survive |
Why |
| Goals — what we're ultimately doing |
Prevents drift after compaction |
| Constraints — budget, deadline, "never touch prod" |
Violating one is catastrophic |
| Key facts & figures — IDs, amounts, versions |
Approximations corrupt downstream steps |
| Assumptions — what we decided to assume and why |
Revisit if invalidated |
| Dates — deadlines, event ordering |
Temporal reasoning breaks silently |
| Sources — where each fact came from |
Provenance (§5) dies in generic summaries |
| Prior decisions + rationale — what was chosen, what was rejected |
Prevents re-litigating and flip-flopping |
Reliable designs make this explicit: a structured compaction template ("Goals / Constraints / Decisions / Open items / Facts-with-sources") rather than "summarize the conversation so far." And the strongest answers pair compaction with §4: durable state doesn't rely on the summary at all.
4. Scratchpad files: durable state outside the window
State that must not be lost does not belong (only) in the conversation. Long-running agents write working files — notes.md, progress.json, decisions.md — and re-read them when needed.
Why this is the exam answer whenever sessions are long or interruptible:
- Survives compaction — files aren't summarized away.
- Survives session loss — pairs with
--resume (Domain 3) for true recovery.
- Selective reload — the agent re-reads exactly what it needs, instead of hauling everything in context forever.
- Auditable — humans can inspect the scratchpad mid-run (escalation handoffs get this for free).
Scenario S4 (developer productivity) leans on this: "the agent works across many files over hours — how does it keep track?" → progress/notes files on disk, updated as it goes.
5. Provenance & claim-source mapping
For research and reporting systems (Scenario S3), reliability means every claim in the output is traceable to a source. The tested design points:
- Capture at collection time: each finding is recorded with its source (URL, document ID, page) the moment it's found — you cannot reliably reconstruct sources afterwards.
- Carry it through the pipeline: subagents return
{finding, source, confidence} triples; merges keep the mapping; compaction preserves it (§3).
- Enforce structurally: the return schema requires
sources — a coordinator rejects unsourced findings (Domain 1/2 overlap). A final "please add citations" pass is the trap option: post-hoc citation invents sources.
6. Prompt caching — at exam depth
The official guide scopes this narrowly: know it exists, its headline mechanics, and when it helps. (Implementation internals are explicitly out of scope.)
Exam-depth facts:
- Caches a stable prefix of the prompt (system prompt, tool definitions, long documents) so repeat calls don't reprocess it.
- Marked with
cache_control breakpoints; minimum cacheable prefix ≈ 1,024 tokens; default TTL ≈ 5 minutes (refreshed on use).
- Cache reads cost ~10% of normal input price; writes cost slightly more than normal (~25% premium).
- It's a prefix match: any change earlier in the prompt invalidates everything after it → put stable content first, volatile content (the user's question) last.
- When it's the answer: high-volume agents re-sending the same big system prompt + tools; multi-turn conversations; batch-style repeated analysis with a shared preamble.
Current-API nuance (not exam material): the minimum cacheable prefix now varies by model (some models require ~2k–4k tokens). For the exam, "1,024 tokens / 5 minutes / ~0.1× reads" are the numbers passers report being tested.
7. Message Batches API — the trade-off table
The facts the exam tests:
| Property |
Value |
| Cost |
50% of standard price — the headline |
| Latency |
Asynchronous; most complete in under an hour, guaranteed window is up to 24 hours |
| Identification |
Each request carries a custom_id; results arrive in any order — correlate by custom_id, never by position |
| Failure granularity |
Per-request: one batch can contain succeeded, errored, canceled, expired results — handle partially, resubmit only the failures |
| Scale |
Thousands of requests per batch |
When to use it — the recurring judgment call:
| Workload |
Batch? |
| Nightly classification of the day's 50k tickets |
✅ perfect fit |
| Backfill/re-analysis over historical data |
✅ |
| Interactive chat, live support agent |
❌ user is waiting |
| "Results needed within the hour, guaranteed" |
❌ only up to 24h is guaranteed |
Named passer pattern — "Batch API trade-offs": the wrong options put latency-sensitive work on Batch (to save money) or latency-tolerant bulk work on the live API (paying 2×). Also tested: resilient consumers key results by custom_id and resubmit only failed/expired items, not the whole batch.
📊 Diagram — “Batch vs live decision plus custom_id correlation sketch” (a version appears earlier on this page)
8. Error propagation in multi-agent systems
When agents call agents, failures must travel as data, not disappear:
- Structured failure results: a subagent that fails returns
{"status": "failed", "error": ..., "partial": ...} through the same schema as success — the coordinator can then decide (retry, degrade, escalate). A subagent that returns prose apologies or nothing forces the coordinator to guess.
- No silent degradation: if 2 of 5 research workers failed, the final report must say its coverage is partial — the worst outcome is a confident report silently missing 40% of its inputs.
- Contain, don't cascade: one worker's failure shouldn't abort the fan-out; gather what succeeded, then decide centrally.
- Attempt budgets at every level (worker retries, coordinator re-dispatches) — bounded, then escalate. (Same rule as Domains 1 and 4: bounded attempts, explicit failure.)
9. Confidence calibration & escalation
The reliability tie-breaker the exam applies to agent outputs:
- Collect confidence structurally: make
confidence (and sources) required fields in output schemas. You can't route on what you didn't capture.
- Route on it: high → auto-proceed; medium → cheap verification pass; low → human review. Thresholds are design decisions set in code, not vibes in a prompt.
- Calibration checks: spot-audit — if "0.9 confidence" answers are right 60% of the time, recalibrate the rubric you give the model for scoring itself.
- Guarantee-levels thinking (a passer-reported cross-cutting theme): match the mechanism to the required guarantee — nice-to-have → prompt; should-happen → validation + retry; must-happen → structural enforcement (hooks, schemas, permissions); must-happen-and-prove-it → structural + audit log.
10½. Official exam-guide addenda (v1.0)
Specifics from the official task statements (Exam Guide v1.0, July 2026):
The "case facts" block (TS 5.1). The tested pattern for long support-style sessions: extract transactional facts (amounts, dates, order numbers, statuses) into a persistent case-facts block included in every prompt, outside the summarized history — progressive summarization is precisely where numbers and customer-stated expectations degrade. Corollary: trim verbose tool results (40 fields when 5 matter) before they enter context, and when downstream agents have tight budgets, upstream agents return structured facts + citations, not reasoning chains.
Escalation rules (TS 5.2). Three sharp edges: when a customer explicitly asks for a human, honor it immediately — don't investigate first (offering to resolve is only right when they haven't demanded a person and the issue is clearly in-capability); sentiment analysis and self-reported confidence scores are unreliable proxies for case complexity — escalate on policy gaps and lack of progress, not vibes (calibrated field-level confidence from §9 is different: it's validated against labeled data); and when identity lookup returns multiple matches, ask for additional identifiers — never pick heuristically.
Batch API fine print (TS 4.5). Two facts the pages' table didn't carry: a batch request cannot do multi-turn tool calling (no execute-tools-mid-request-and-continue — agentic loops don't fit inside Batch), and there's no latency SLA — "usually under an hour" is not a guarantee. SLA math is tested: with a 24 h processing window and a 30 h commitment, submit every ~4 h. And before batching 50k documents, refine the prompt on a small sample — first-pass success is the cheapest optimization.
Crash recovery (TS 5.4). Beyond scratchpads: each agent exports structured state to a known location (a manifest); on restart the coordinator loads the manifest and injects it into agent prompts. Also: summarize each exploration phase before spawning the next phase's subagents.
Review calibration (TS 5.5). Aggregate accuracy (97%!) can hide a document type or field that fails constantly — validate accuracy per segment before reducing human review. Ongoing safety net: stratified random sampling of high-confidence extractions to measure real error rates and catch novel failure patterns. Field-level confidence is only meaningful once calibrated against a labeled validation set.
Provenance edge cases (TS 5.6). When two credible sources give conflicting statistics, annotate the conflict with both attributions — never arbitrarily pick one (the coordinator decides how to reconcile). Require publication/collection dates in structured outputs so temporal differences aren't misread as contradictions. And synthesis should render by content type — financial data as tables, news as prose, technical findings as lists — not flattened into one format.
