Exam Overview — Claude Certified Architect – Foundations (CCAR-F)

Everything on this page comes from the official Exam Guide v1.0, effective July 2026 ("the exam guide" below). Anything that comes from elsewhere, or is interpretation, is marked (not in the exam guide).

Key facts

Scoring is criterion-referenced: you are measured against a fixed standard set by a formal standard-setting study, not against other candidates. 720 is a scaled score, not 72% correct, so you can't work out how many questions you can afford to miss.

Version note

The v0.2 draft (June 2026) and v1.0 (July 2026) have the same domains, the same 30 task statements and the same 12 sample questions. Only the surrounding material changed. Two differences matter:

• v0.2 said "All questions are multiple choice… one correct response and three incorrect." v1.0 says multiple-choice and multiple-response. Expect some "select two" items, and read the instruction line on every question.
• v1.0 adds the policy sections (retakes, ID, NDA, appeals, recertification) that v0.2 lacked. It also introduces the exam code CCAR-F and adds percent-correct by domain to the score report; v0.2 reported pass/fail only.

Blueprint

Domain 1 is the largest, with seven task statements. Domains 1, 3 and 4 together are two-thirds of the exam.

The six scenarios (four appear on your form)

You can't choose which four you get, so study all six. Each scenario is a paragraph describing a production system, followed by a set of questions about it.

1. Customer Support Resolution Agent. A support agent built with the Claude Agent SDK handling high-ambiguity requests (returns, billing disputes, account issues) through custom MCP tools: get_customer, lookup_order, process_refund, escalate_to_human. Target is 80%+ first-contact resolution while knowing when to escalate. Primary domains: 1, 2, 5.

2. Code Generation with Claude Code. A team using Claude Code for generation, refactoring, debugging and documentation, integrating it with custom slash commands and CLAUDE.md, and deciding plan mode vs direct execution. Primary domains: 3, 5.

3. Multi-Agent Research System. A coordinator delegating to four specialized subagents — web search, document analysis, synthesis, report generation — producing comprehensive cited reports. Primary domains: 1, 2, 5.

4. Developer Productivity with Claude. An Agent SDK agent helping engineers explore unfamiliar codebases, understand legacy systems, generate boilerplate and automate repetitive tasks, using built-in tools (Read, Write, Bash, Grep, Glob) plus MCP servers. Primary domains: 2, 3, 1.

5. Claude Code for Continuous Integration. Claude Code in a CI/CD pipeline running automated code reviews, generating test cases and giving PR feedback, with a stated emphasis on actionable feedback and minimizing false positives. Primary domains: 3, 4.

6. Structured Data Extraction. Extracting information from unstructured documents, validating output with JSON schemas, maintaining high accuracy, handling edge cases gracefully, integrating downstream. Primary domains: 4, 5.

The scenarios are different settings for the same five domains. One idea, such as "enforce the rule in code rather than in the prompt", appears as a refund gate in scenario 1 and as a CI gate in scenario 5.

Out of scope — do not study these

The exam guide lists what is excluded, and the list is long. Nothing below is tested, so skip it, even though Anthropic's docs and most community study guides cover it:

Fine-tuning or training custom models · Claude API authentication, billing, or account management · language- and framework-specific implementation detail beyond what tool and schema configuration needs · deploying or hosting MCP servers (infrastructure, networking, containers) · Claude's internal architecture, training, or weights · Constitutional AI, RLHF, safety training · embedding models and vector database internals · computer use (browser/desktop automation) · vision and image analysis · streaming API implementation and server-sent events · rate limiting, quotas, pricing calculations · prompt caching implementation details beyond knowing it exists · token counting algorithms and tokenization · OAuth, API key rotation, authentication protocols · specific cloud provider configuration (AWS, GCP, Azure) · performance benchmarking and model comparison metrics.

Knowing this list lets you eliminate distractors quickly. Much community prep material drills prompt-cache TTLs, 429 handling, pause_turn and MCP transport specs; none of that is on this exam.

In-scope topics (the exam guide's own summary)

Agentic loop implementation (control flow on stop_reason, tool result handling, termination) · multi-agent orchestration (coordinator-subagent, decomposition, parallel execution, iterative refinement) · subagent context management (explicit passing, structured state persistence, crash recovery via manifests) · tool interface design (descriptions, splitting vs consolidating, naming) · MCP tool and resource design · MCP server configuration (project vs user scope, env var expansion, multi-server access) · error handling and propagation (structured responses, transient vs business vs permission, local recovery before escalation) · escalation decision-making · CLAUDE.md configuration (hierarchy, @import, .claude/rules/ glob patterns) · custom commands and skills (scope, context: fork, allowed-tools, argument-hint) · plan mode vs direct execution · iterative refinement (I/O examples, test-driven iteration, interview pattern) · structured output via tool_use (schema design, tool_choice, nullable fields) · few-shot prompting · batch processing · context window optimization · human review workflows (confidence calibration, stratified sampling, accuracy segmentation) · information provenance.

Appendix: technologies named in the exam guide

The exam guide lists the specific technologies questions may refer to. Recognize each by name:

• Claude Agent SDK — agent definitions, agentic loops, stop_reason handling, hooks (PostToolUse, tool call interception), subagent spawning via the Task tool, allowedTools configuration
• MCP — servers, tools, resources, the isError flag, tool descriptions, tool distribution, .mcp.json, environment variable expansion
• Claude Code — CLAUDE.md hierarchy (user/project/directory), .claude/rules/ with YAML frontmatter path scoping, .claude/commands/, .claude/skills/ with SKILL.md frontmatter (context: fork, allowed-tools, argument-hint), plan mode, direct execution, /memory, /compact, --resume, fork_session, the Explore subagent
• Claude Code CLI — -p / --print, --output-format json, --json-schema
• Claude API — tool_use with JSON schemas, tool_choice (auto, any, forced), stop_reason values (tool_use, end_turn), max_tokens, system prompts
• Message Batches API — 50% cost savings, up to 24-hour window, custom_id, polling, no multi-turn tool calling
• JSON Schema — required vs optional, enums, nullable, "other" + detail string, strict mode
• Pydantic — schema validation, semantic validation errors, validation-retry loops
• Built-in tools — Read, Write, Edit, Bash, Grep, Glob
• Few-shot prompting, prompt chaining, context window management, session management, confidence scoring

Current product (not tested): this list reflects the exam guide, not the full current API and product. The domain pages mark where the product has moved on, e.g. the Task tool's current SDK name, extra stop_reason and tool_choice values, and a third MCP scope.

Intended audience

The exam guide describes the ideal candidate as a solution architect with 6+ months of hands-on experience with the Claude API, Agent SDK, Claude Code and MCP. This is a description, not a requirement; the exam guide lists no prerequisites (inference, not stated).

Logistics and policies

Registration runs through the Anthropic Partner Academy; you then create a Pearson VUE account and schedule. The fee at checkout reflects any discount that applies to your partner tier. The exam guide states no eligibility restriction.

Cancel or reschedule up to 24 hours before your appointment. Inside 24 hours, or a no-show, forfeits the fee.

ID. Valid, unexpired, government-issued photo ID. The name on it must match your registration exactly — name corrections go to [email protected] before scheduling.

Retakes. 14 days after a first fail, 30 days after a second, 90 days after a third. Maximum four attempts per rolling twelve months, per exam. Full fee each time.

Conduct. Stay in view of the webcam if testing online; workspace clear of notes, books, phones, secondary monitors; no communication with anyone; no capturing exam content. Prohibited: phones, smart watches, headphones, study materials, recording devices. Scratch paper only if the proctor provides it.

NDA. You accept a confidentiality agreement before the exam starts. Decline and the session ends with no refund.

Recertification. On-time renewal is a free, non-proctored assessment on the Partner Academy. Let it lapse and you retake the full exam at full price.

Appeals within 14 days, via Pearson VUE support. The standard-setting outcome and individual item content are not appealable.

Accommodations must be requested and approved by Pearson VUE before you schedule.

Three-Day Study Plan

For someone who uses Claude Code daily and needs to turn that into exam knowledge, including the API and SDK layer underneath it, which the exam tests and daily Claude Code use doesn't show you. Plan on 5–6 hours a day; if you have less, do only the starred (★) items.

The exam guide describes the exam as a test of practical judgment about architecture, configuration, and tradeoffs, not recall. So the plan starts with the two things that matter most: the task statements, which define what can be tested, and the sample-question answer explanations, which show how the exam chooses between options.

Day 1 — Map the territory, then own Domain 1

Morning (~2.5h)

★ Read Exam Overview in full. Spend most of the time on the six scenarios and the out-of-scope list. The out-of-scope list rules out perhaps a third of what community prep material covers, and knowing a topic is out of scope lets you eliminate distractors immediately.

★ Read §6 of the official PDF, Detailed Objectives by Domain, once straight through without stopping to study. The aim is an overview of all 30 task statements before you study any one in depth.

Afternoon (~2.5h)

★ Study Domain 1 carefully. It is 27% of the exam and seven task statements. Don't skim it because agentic loops feel familiar: the exam tests the anti-patterns as much as the patterns.

Then do practice questions 1–11 (Practice Questions). If you got one right for the wrong reason, count it as a miss.

Evening (~45min)

Read the four Preparation Exercises in §8 of the PDF. You won't have time to build them, but they show what the question writers had in mind. Exercise 1 (multi-tool agent with escalation) and Exercise 4 (multi-agent research pipeline) correspond closely to scenarios 1 and 3.

End of day, you should be able to say without looking: the five domains and their weights, the six scenarios, what terminates an agentic loop, why subagents start with no context, and when a hook beats a prompt.

Day 2 — Domains 2, 3 and 4

Morning (~2h)

★ Domain 2 — Tool Design & MCP, then questions 12–18.

The two ideas to remember: descriptions are the primary selection mechanism (and usually the correct first fix), and the four error categories with only transient being retryable.

★ Domain 3 — Claude Code, then questions 19–27.

Domain 3 is mostly memorization: which file, which directory, which flag. Write out the file map from the Quick Reference on paper from memory. If you can reproduce it without looking, you should get most of Domain 3's ~12 items.

Afternoon (~2.5h)

★ Domain 4 — Prompt Engineering & Structured Output, then questions 28–36.

Spend extra time on syntax versus semantics. A schema fixes malformed JSON. It does not fix JSON that is well-formed but wrong. Questions about values that don't add up, swapped fields or invented data are testing whether you choose validation logic over a stricter schema.

Also memorize the Batch API facts together: 50% · ≤24h · no SLA · no multi-turn tool calling · custom_id. Whether to use batch is decided by how long the result can wait, not by cost.

Evening (~1h)

★ Do the twelve official sample questions in §9 of the PDF. Read every answer explanation twice, including for questions you got right. They are the best available evidence of how the exam weighs options. Note the phrases "over-engineered," "most effective first step," "addresses tool availability rather than tool ordering," and "probabilistic LLM compliance."

Day 3 — Domain 5, synthesis, and consolidation

Morning (~2h)

★ Domain 5 — Context Management & Reliability, then questions 37–40 and the five additional items 41–45.

Domain 5 is only 15%, but it is a primary domain in four of the six scenarios, so its ideas appear in other domains' questions too. Focus on the three valid escalation triggers, the four-part structured error report, and the provenance checklist.

Midday (~1.5h) — the scenario pass

For each of the six scenarios, write yourself half a page from memory: what is it, which domains the exam guide lists as primary, and what are the three most likely things to go wrong in it. Then check against Exam Overview.

This is the most useful exercise on day 3. Every question on the exam belongs to one of these six scenarios, and if you already know a scenario's typical failures, most questions become recognition.

Afternoon (~1.5h) — weak-spot repair

Go back to whichever domain page you scored worst on. Re-read only its decision table and ten things to have cold. Then re-attempt the questions you missed, out of order.

If you have time left, re-read the titles of all 30 task statements and say what each one covers. If you can't, re-read that section.

Evening (~45min)

★ Read the Quick Reference once, then stop. Don't learn new material the night before; it pushes out what you already know.

Confirm the logistics: ID name matches your registration exactly, Pearson VUE system test done if you're testing online, room clear, 30 minutes of buffer before your slot.

Exam-day tactics

Time. 60 items in 120 minutes is two minutes each. Most take under a minute once you recognize the pattern; save the time for the four or five that are close calls.

Read the instruction line. v1.0 says multiple-choice and multiple-response. Each item states how many to select. Missing a "select two" loses the item for no reason.

Find the deciding phrase. Almost every question contains one phrase that decides the answer: must never, before any, every time, available to every developer, developers wait for, regardless of directory, the same session. Find it before reading the options.

Eliminate options by rule, not by how plausible they sound. Three rules usually work: - Remove any option that meets a must always requirement with a prompt instruction. - Remove any option that adds infrastructure (a classifier, a trained model, a routing layer, a bigger model, a bigger context window) when a description, a criterion or a schema field would do. - Remove any option that pushes work onto a human or the customer when the system should handle it.

When two options both look right, choose the one that fixes the root cause described in the question. The exam guide's explanations often reject fixes that would help but address a different problem.

Answer every item. There is no penalty for guessing, and v0.2 noted the platform requires an answer before you can move on anyway.

If you only have one day

Overview (especially the out-of-scope list) → Domain 1 → Domain 3 → the twelve official sample questions with explanations → Quick Reference. That covers 47% of the blueprint in depth, plus the exam's own worked examples.

Domain 1 — Agentic Architecture & Orchestration (27%, ~16 items)

Domain 1 is the largest domain: 27% of the exam and seven task statements. Most questions describe a production system that misbehaves and ask you to pick the fix. Two principles decide most of them:

• When a mistake has real consequences (money, compliance, security), enforce the rule in code, not in the prompt. A prompt instruction is followed most of the time; code is followed every time.
• Don't add machinery the failure doesn't need. If a small fix addresses the root cause, the bigger one is wrong.

In the book: Chapter 2 — The Agentic Loop (1.1) · Chapter 5 — Orchestration (1.2, 1.3) · Chapter 6 — Hooks & Gates (1.4, 1.5) · Chapter 1 — Agents & Workflows (1.6) · Chapter 7 — Sessions (1.7)

1.1 Design and implement agentic loops

An agentic loop is the code that sends a request to Claude, runs any tools Claude asks for, sends the results back, and repeats until Claude is done. Claude Code runs this loop every time you press enter; on the exam you are the one writing it.

The loop

1. POST /v1/messages  →  system, messages[], tools[]
2. Read response.stop_reason
3. If "tool_use":
     append the assistant message (with its tool_use blocks) to messages
     execute each requested tool
     append a USER message containing tool_result block(s)
     go to 1
4. If "end_turn": done — present the final response

The exam guide states three facts about it:

• stop_reason controls the loop. "tool_use" means Claude wants a tool run, so continue. "end_turn" means Claude has finished, so stop.
• Tool results are appended to conversation history. That history is the only place the model can see what its tools returned, so it is how the agent builds up knowledge across steps.
• Model-driven decision-making: Claude reasons from context about which tool to call next. That is agentic design. A pre-configured decision tree or a fixed tool sequence is a different design.

The three anti-patterns

The exam guide names three wrong ways to decide the loop is finished. Expect them as distractors:

1. Parsing natural language signals to determine loop termination: searching the response text for "done" or "I've finished."
2. Setting arbitrary iteration caps as the primary stopping mechanism. A cap protects against a runaway loop. It does not tell you the task is complete.
3. Checking for assistant text content as a completion indicator. A response with stop_reason: "tool_use" often contains text too, because Claude frequently says what it is about to do before calling a tool.

Answer rule. Any option that decides completion by something other than stop_reason is wrong.

Current product (not tested): current models also return "model_context_window_exceeded", meaning the response hit the context window limit mid-generation. The exam guide uses only tool_use and end_turn, and that is what is tested.

1.2 Orchestrate multi-agent systems with coordinator-subagent patterns

Hub and spoke

In a hub-and-spoke design, one coordinator agent sits in the middle and every subagent talks only to it. Subagents never talk to each other. The coordinator handles all communication, error handling and routing of information between them.

The exam guide gives three reasons for routing everything through the coordinator: observability (one place to see what happened), consistent error handling, and controlled information flow (each subagent gets exactly what the coordinator chooses to give it).

The coordinator's job:

• break the task into pieces (decomposition)
• hand each piece to a subagent (delegation)
• combine the results (aggregation)
• deciding which subagents to invoke based on query complexity, rather than running the whole pipeline every time

Subagents have isolated context. A subagent does not see the coordinator's conversation history. It knows only what the coordinator puts in its prompt. Several questions in Domains 1 and 5 depend on this fact.

Failure mode: decomposition that is too narrow

The exam guide warns that a coordinator can split a broad topic too narrowly, so the subagents between them cover only part of it. Sample question 7 shows this: every subagent succeeds, yet the report on "creative industries" misses most of the topic, because the coordinator split it into three visual-arts subtasks.

When every downstream agent worked correctly but the overall coverage is wrong, the cause is the coordinator's decomposition.

Skills tested

• The coordinator should analyze query requirements and dynamically select subagents, instead of always running the full pipeline.
• Partition the scope so subagents don't duplicate each other, for example by giving each one distinct subtopics or distinct source types.
• Iterative refinement loops: the coordinator reviews the synthesis for gaps, sends targeted follow-up queries to the search and analysis subagents, runs synthesis again, and repeats until coverage is sufficient.
• All subagent communication goes through the coordinator.

1.3 Configure subagent invocation, context passing, and spawning

Skills tested

• Put the complete findings from earlier agents directly into the subagent's prompt. If the synthesis subagent needs the web search results and the document analysis, the coordinator pastes both in. A prompt that says "analyze the findings" without including them is the classic wrong answer.
• Pass context as structured data that keeps content separate from metadata (source URLs, document names, page numbers), so each claim can still be traced to its source after the handoff.
• To run subagents in parallel, emit several Task tool calls in one coordinator response. One Task call per turn runs them one after another.
• Give subagents goals and quality criteria, not step-by-step procedures, so they can adapt to what they find.

Quick diagnostics:

Current product (not tested): the current Claude Agent SDK calls this tool Agent, and keeps Task as an alias. The exam guide says Task throughout, so answer with Task.

The SDK can also resume a specific subagent's own transcript (capture its session_id/agentId and pass resume: sessionId), instead of pasting prior findings into a fresh prompt. The exam guide's mechanism is still "provide the context explicitly in the prompt."

1.4 Multi-step workflows with enforcement and handoff

Programmatic enforcement vs prompt guidance

Programmatic enforcement means a rule is checked by code; prompt guidance means the model is asked to follow it. This is the most tested idea on the exam. In the exam guide's words: "When deterministic compliance is required (e.g., identity verification before financial operations), prompt instructions alone have a non-zero failure rate." A system prompt rule will eventually be skipped. If skipping it costs money or breaks compliance, it needs a check in code.

Prerequisite gates are that check: your code refuses to execute a tool call until an earlier step has completed. The exam guide's example is to block process_refund until get_customer has returned a verified customer ID. Sample question 1 tests the same idea; its correct option gates both lookup_order and process_refund on identity verification.

Sample question 1's explanation also says why the other options fail:

• A stronger system prompt (B) and few-shot examples (C) both "rely on probabilistic LLM compliance, which is insufficient when errors have financial consequences."
• A routing classifier (D) "addresses tool availability rather than tool ordering, which is not the actual problem."

Structured handoff to a human

When the agent escalates mid-task to a human who cannot see the conversation transcript, it should send a structured summary: customer ID, root cause, refund amount, recommended action. A raw transcript dump makes the human redo the agent's work.

Requests with several concerns

When a customer raises several issues in one message, split them into distinct items, investigate each in parallel using shared context, then combine the results into a unified resolution. Handling the first issue and asking the customer to open a second ticket is wrong.

1.5 Agent SDK hooks for interception and normalization

A hook is your code, registered with the Agent SDK, that runs automatically at a fixed point in the loop. The model cannot skip it. Claude Code's permission prompt is a similar idea: a check that runs before a tool executes, whatever the model intended.

The exam guide names two uses:

The exam guide ties this back to 1.4: it contrasts "using hooks for deterministic guarantees versus relying on prompt instructions for probabilistic compliance." Use a hook when a business rule must always hold.

The direction matters, and it is a common distractor. PostToolUse sees results coming back from a tool. Interception sees calls going out. A PostToolUse hook cannot stop a refund, because by the time it runs the refund has already been processed.

The exam also tests the opposite mistake: don't use a hook for a soft preference. Match the strength of the enforcement to the cost of getting it wrong.

1.6 Task decomposition strategies

There are two ways to break a task into steps:

Skills tested

• Use prompt chaining for predictable, multi-aspect reviews. Use dynamic decomposition for open-ended investigation.
• Split a large code review into one pass per file plus a separate cross-file integration pass. The exam guide's reason is attention dilution: when one pass covers many files, the model gives each one shallower attention and can contradict itself across files. Sample question 12 (a 14-file review that comes back shallow and inconsistent) has this as its answer. Its explanation rejects two alternatives: a bigger context window "doesn't solve attention quality issues," and requiring 2 of 3 independent passes to agree "would actually suppress detection of real bugs."
• For an open-ended task, first map the structure, then find the high-impact areas, then write a prioritized plan that changes as you discover dependencies.

1.7 Session state, resumption, and forking

The exam guide states two judgments outright:

• "Starting a new session with a structured summary is more reliable than resuming with stale tool results." Resuming is not automatically right.
• When you do resume after code has changed, tell the agent which files changed so it re-analyzes those, rather than re-exploring everything.

Current product (not tested): continue / continue_conversation resumes the most recent session in the working directory, with no session ID. --resume takes a specific session. If an option confuses "most recent, no ID" with "specific, by ID", that is the trap.

Domain 1 decision table

Ten things to have cold

1. stop_reason is the only correct signal for loop control: "tool_use" → continue, "end_turn" → stop.
2. An iteration cap is a safety net, never the way the loop ends normally.
3. Text in a response does not mean the turn is finished.
4. Subagents inherit nothing: no conversation history, no memory between invocations.
5. allowedTools must include "Task" for the coordinator to spawn subagents.
6. Parallel subagents = several Task calls in one response.
7. Prompt instructions fail some of the time. Where the cost is financial, compliance or security, use a gate or hook.
8. PostToolUse changes results after a tool runs; interception blocks calls before it runs. They are not interchangeable.
9. Every downstream agent succeeded but coverage is wrong → the coordinator's decomposition is the problem.
10. A fresh session with a structured summary beats resuming with stale tool results.

Domain 2 — Tool Design & MCP Integration (18%, ~11 items)

Five task statements. This domain is about how you design tool interfaces and configure MCP servers, not about the MCP protocol itself. The exam guide's out-of-scope list removes "deploying or hosting MCP servers (infrastructure, networking, container orchestration)" and "OAuth, API key rotation, or authentication protocol details", so there are no questions on transports, OAuth, protocol internals or hosting.

In the book: Chapter 3 — Designing Tools (2.1, 2.3, 2.5) · Chapter 4 — Errors & Recovery (2.2) · Chapter 8 — MCP (2.4)

2.1 Design effective tool interfaces

Descriptions drive tool selection

The model chooses which tool to call mainly by reading the tool descriptions. The exam guide: tool descriptions are the primary mechanism LLMs use for tool selection, and minimal descriptions make selection between similar tools unreliable. This is the answer to many Domain 2 questions.

A good description covers:

• what the tool does and what it returns
• the input formats it accepts
• example queries
• edge cases
• boundary explanations: when to use this tool rather than a similar one

Failure mode: misrouting

Misrouting is the model calling the wrong one of two similar tools. It happens when descriptions are vague or overlap. The exam guide's example is analyze_content and analyze_document with near-identical descriptions.

Sample question 2 has two tools, get_customer ("Retrieves customer information") and lookup_order ("Retrieves order details"). Both descriptions are minimal and both tools accept similar identifiers, so the agent confuses them. The correct answer is expand the descriptions. The explanation rejects the other options:

• Few-shot examples "add token overhead without fixing the underlying issue."
• A routing layer is "over-engineered and bypasses the LLM's natural language understanding."
• Merging the two tools is "a valid architectural choice but requires more effort than a first step warrants."

Keywords in the system prompt

Instructions in the system prompt that mention particular keywords can make the model associate those words with a tool, overriding good tool descriptions. If the descriptions are good and selection is still wrong, check the system prompt for such keywords.

Three ways to fix tool selection

Current product (not tested): tool definitions now accept "strict": true, which guarantees the tool's input matches its schema. It does not fix misrouting: it controls whether the arguments are well-formed, not which tool is chosen. The exam's answer to misrouting is better descriptions.

The tool search tool (tool_search_tool_bm25_20251119 / tool_search_tool_regex_20251119) keeps most tool definitions out of context and loads them when needed. It helps with very large tool sets (see 2.3), but it does not replace the exam's answer there, which is to give each agent only the tools its role needs.

2.2 Structured error responses for MCP tools

How a tool reports failure

An MCP tool reports failure by setting the isError flag on its result. The error comes back to the agent as readable content, not as an exception, so the agent can decide what to do next. This is why Claude Code can recover when a Bash command fails: the failure output comes back as a tool result it can read.

Current product (not tested): in the Messages API, the equivalent field on a tool_result block is spelled is_error. The exam guide only uses isError.

The four error categories

What a good error response contains

• errorCategory: transient, validation or permission (the categories the exam guide lists in the skills bullet)
• isRetryable: a boolean
• a human-readable description of what went wrong
• for a business-rule violation, retriable: false and a customer-friendly explanation, so the agent explains the policy to the customer instead of retrying

Why a generic error is wrong

A generic "Operation failed" prevents the agent from making appropriate recovery decisions. Structured error metadata prevents wasted retry attempts on errors that will never succeed.

Two more skills

• Subagents recover locally from transient failures. They pass to the coordinator only what they cannot fix themselves, and when they do, they include partial results and what was attempted.
• An access failure is not the same as an empty result. A timeout is a failure that needs a retry decision. A query that ran and matched nothing is a success that returned an empty set. Options that confuse the two are a recurring distractor.

2.3 Distribute tools across agents and configure tool choice

Too many tools makes selection worse

The exam guide gives a number: an agent with 18 tools instead of 4–5 picks tools less reliably, because it has more options to choose between. Agents that hold tools outside their role also tend to misuse them; the exam guide's example is a synthesis agent that starts running web searches.

The fix is scoped tool access: each agent gets only the tools its role needs, plus limited cross-role tools for specific high-frequency needs.

Sample question 9 is the model case. The synthesis agent needs to verify facts. 85% of verifications are simple fact-checks; 15% need real investigation. The correct answer is a narrow verify_fact tool for the simple cases, with complex verifications still sent through the coordinator. The rejected options are giving synthesis full web search (too much access, and it mixes up the agents' responsibilities) and batching verifications (it makes synthesis wait on them).

The exam guide also names replacing a generic tool with a constrained one, for example replacing fetch_url with a load_document tool that only accepts valid document URLs.

tool_choice

tool_choice is a request parameter that controls whether and which tool the model must call.

Current product (not tested): a fourth value, {"type": "none"}, forces a text-only reply with no tool call. The exam guide lists only the three above.

2.4 Integrate MCP servers into Claude Code and agent workflows

Where MCP servers are configured

Environment variable expansion lets .mcp.json reference a secret such as ${GITHUB_TOKEN} without containing it, so the file can be committed safely. An option that hardcodes a token in .mcp.json is always wrong.

Tools from all configured MCP servers are discovered at connection time and available simultaneously. Project and user servers load side by side. Scope decides who gets the server, not which server's tools take priority.

Current product (not tested): Claude Code now has a third MCP scope, local. It is also stored in ~/.claude.json, but applies to one project only and is not committed. It is the current answer for "a personal server with credentials I don't want in git", which the exam guide assigns to user scope. It is unrelated to .claude/settings.local.json, despite the similar name.

The "no priority" statement is true for which tool the model picks. Separately, if the same server name is defined in several scopes, Claude Code resolves the name clash in the order local > project > user > plugin-provided > claude.ai connectors. That is a different question and is unlikely to be tested, since the exam guide names only two scopes.

MCP resources

An MCP server can expose resources as well as tools. Resources are read-only content catalogs (issue summaries, documentation hierarchies, database schemas) that let the agent see what exists, to reduce exploratory tool calls. The exam guide's summary: "resources for content catalogs, tools for actions."

Current product (not tested): MCP also defines prompts, reusable prompt templates a server can provide, as a third primitive alongside tools and resources.

Two judgment calls

• Write detailed descriptions of your MCP tools' capabilities and outputs. Otherwise the agent keeps using built-in tools like Grep instead of your more capable MCP tool. This is a description fix again, not a scope or configuration fix.
• Use an existing community MCP server for a standard integration (the exam guide's example is Jira). Build a custom server only for workflows specific to your team.

2.5 Select and apply built-in tools

If Edit fails because the text it should match appears more than once, fall back to Read + Write. The exam guide names this fallback.

Exploring a codebase

The exam guide describes building understanding incrementally: use Grep to find entry points, then Read to follow imports and trace flows, rather than reading every file up front. To find where a function is used through wrapper modules, first list every name the wrapper exports, then search the codebase for each name.

Domain 2 decision table

Ten things to have cold

1. Tool descriptions are the main way the model selects tools, and improving them is usually the correct first fix.
2. A good description gives input formats, example queries, edge cases, and when to use it instead of similar tools.
3. MCP tools report failure with isError; the error payload carries errorCategory and isRetryable.
4. Only transient errors are retryable. Validation, business and permission errors are not.
5. An empty result is not an error. A timeout is an error; zero matches is a success.
6. Subagents recover from transient failures themselves and pass up only what they can't fix, with partial results.
7. About 4–5 tools per agent. 18 makes selection worse. Tools outside an agent's role get misused.
8. auto may reply with text · any must call some tool · a named tool forces that tool.
9. .mcp.json = project, shared; ~/.claude.json = user, personal. ${ENV_VAR} for secrets. Both load at the same time.
10. Grep searches contents. Glob matches file names. Edit needs text that appears once; otherwise use Read + Write. Bash is available, but prefer a dedicated tool where one exists.

Domain 3 — Claude Code Configuration & Workflows (20%, ~12 items)

Six task statements. Most questions here ask where a file goes, which frontmatter key to use, or which flag to pass. You know these features from daily use; this page gives you the exam's vocabulary for them and the exact details it tests.

The exam covers less of Claude Code than the product docs do. settings.json precedence, permission modes, the full list of hook events, plugins, marketplaces, output styles and authentication are not in the objectives or the technologies list in the exam guide's §17 appendix. Hooks are tested, but under Domain 1 (task 1.5), and only PostToolUse and tool-call interception.

In the book: Chapter 9 — Configuring Claude Code (3.1, 3.3) · Chapter 10 — Extending Claude Code (3.2, 3.4) · Chapter 12 — Prompt Engineering (3.5) · Chapter 11 — Claude Code in CI (3.6)

3.1 CLAUDE.md hierarchy, scoping, and modular organization

The three levels (exam guide) — plus two more in the current product

The exam guide names three levels of CLAUDE.md:

The exam guide calls out one diagnosis: "a new team member not receiving instructions because they're in user-level rather than project-level configuration." If a question says the instructions work for one developer but not the rest of the team, they are in ~/.claude/CLAUDE.md and need to move into the project.

Current product (not tested): Claude Code has two more levels. Above user and project there is a managed (enterprise) policy CLAUDE.md at a fixed OS path, controlled by the organization; it loads first and an engineer can't remove or exclude it. Below project there is CLAUDE.local.md, a gitignored file at the project root: personal like the user file, but for this project only, loaded right after the project's CLAUDE.md. The exam guide's three levels are what's tested. If a question mentions an org-wide policy, or a personal preference for one project that shouldn't go in the shared file, these are the current answers.

Splitting CLAUDE.md into modules

• @import pulls another file into CLAUDE.md, so each package's CLAUDE.md can import just the standards files that apply to it, as chosen by the people who maintain that package.
• .claude/rules/ is a directory of topic-specific rule files as an alternative to a monolithic CLAUDE.md: testing.md, api-conventions.md, deployment.md.

/memory and /context

Exam answer: use /memory to check which memory files are loaded, and to diagnose behavior that differs between sessions. This is the wording of task 3.1; /context does not appear in the exam guide.

Product reality, not the exam answer: in current Claude Code, /memory lists and opens the CLAUDE.md and rules files, and the docs point to /context to see what was actually loaded in this session. Use /context in real work; answer /memory on the exam.

Current product (not tested): Auto Memory is memory Claude writes for itself and keeps across sessions, separate from the CLAUDE.md files you write. It holds things Claude has noticed about how you work, not versioned project standards. It is not a level in the hierarchy and is not tested.

3.2 Custom slash commands and skills

Where they live

Sample question 4 asks where to put a /review command that every developer should get when they clone or pull the repo. The answer is .claude/commands/ in the project repository. The explanation rejects the others: ~/.claude/commands/ isn't shared, CLAUDE.md holds context rather than command definitions, and a .claude/config.json with a commands array does not exist.

To customize a team skill for yourself without affecting teammates, create a personal copy in ~/.claude/skills/ under a different name.

Current product (not tested): skill frontmatter now has direct controls over who can trigger a skill: disable-model-invocation (only a person can run it, e.g. a /deploy with side effects) and user-invocable: false (only Claude can load it, for background knowledge). These are the more precise current answer to "control which skills trigger automatically".

The scope table is also incomplete for the current product, which adds plugin-provided and managed skills and commands, and discovers skills in nested directories (e.g. apps/web/.claude/skills/ → /apps/web:deploy). The exam tests only project vs user.

SKILL.md frontmatter: the three tested options

Skills vs CLAUDE.md

The exam guide states the difference: skills are loaded on demand for a specific task; CLAUDE.md is always loaded and holds standards that apply everywhere.

3.3 Path-specific rules for conditional convention loading

A file in .claude/rules/ can start with YAML frontmatter containing a paths field of glob patterns:

---
paths: ["terraform/**/*"]
---

Path-scoped rules load only when editing matching files. The rest of the time it stays out of context, which saves tokens and keeps unrelated instructions away.

Path rules vs subdirectory CLAUDE.md

This distinction is what task 3.3 is about, and sample question 6 depends on it:

• A subdirectory CLAUDE.md is directory-bound: it applies to one directory.
• A glob rule applies to a type of file wherever it is.

Sample question 6: test files sit next to the code they test all over the codebase (Button.test.tsx beside Button.tsx), and they must all follow the same conventions, loaded automatically. The answer is a .claude/rules/ file with paths: ["**/*.test.tsx"]. The explanation rejects the others:

• a CLAUDE.md in every directory can't cover files spread across many directories
• one root CLAUDE.md with a section per file type relies on Claude inferring which section applies, instead of matching explicitly
• a skill needs someone to invoke it, or Claude to decide to load it, so it isn't automatic

Rule of thumb: conventions that belong to a directory → subdirectory CLAUDE.md. Conventions that belong to a file type found in many directories → .claude/rules/ with a glob.

3.4 Plan mode vs direct execution

Sample question 5 asks about restructuring a monolith into microservices. The answer is plan mode. The explanation rejects "start with direct execution and switch to plan mode if it gets complicated", because the complexity is already stated in the requirements: it isn't something that might appear later.

You can combine the two, and the exam tests it: use plan mode to investigate, then direct execution to implement. For example, plan a library migration, then carry out the plan directly.

The Explore subagent

Use the Explore subagent for verbose discovery phases. It does the searching in its own context and returns summaries, so the main conversation's context isn't used up during a multi-phase task.

3.5 Iterative refinement techniques

Four techniques, each with the situation it fits:

Concrete input/output examples. When a prose description keeps being interpreted differently, show 2–3 concrete examples of input and expected output. This also fixes edge cases, e.g. test cases with example input and expected output showing how a migration script should handle null values.

Test-driven iteration. Write the tests first (expected behavior, edge cases, performance requirements), then iterate by sharing test failures with Claude each round.

The interview pattern. Before implementing, have Claude ask questions to surface things you might not have thought of. Use it in unfamiliar domains, where it brings up issues like cache invalidation strategy and failure modes.

One message or several. If the problems affect each other, describe them all in one detailed message. If they are independent, fix them one at a time.

3.6 Integrate Claude Code into CI/CD

The flags

Sample question 10 describes a CI job that hangs waiting for interactive input. The answer is -p. The explanation says CLAUDE_HEADLESS=true and --batch don't exist, and that redirecting stdin from /dev/null is a Unix workaround that doesn't fix how the command is invoked.

Use --output-format json together with --json-schema to get structured findings that a script can post as inline PR comments.

Current product (not tested): -p fixes a hang on stdin. A CI job can also hang on a permission prompt; that needs --permission-mode (auto/dontAsk/acceptEdits) or --allowedTools to pre-approve specific tools. If a question describes a hang that isn't about input, this is the distinction.

The official claude-code-action GitHub Action wraps this pattern: it runs claude -p with your flags and exposes structured_output as an Actions output, so you don't write the parsing glue yourself.

CLAUDE.md gives CI runs their project context

A CI job running Claude Code gets its project context from CLAUDE.md: testing standards, fixture conventions, review criteria. Writing down what makes a test valuable and which fixtures exist improves generated tests and cuts low-value ones.

Current product (not tested): --bare mode skips automatic discovery of hooks, skills, commands, plugins, MCP servers, CLAUDE.md and auto memory, so CI runs behave the same on every machine. This conflicts with the paragraph above: a --bare run does not load CLAUDE.md, so a job that wants both has to pass the standards explicitly, e.g. with --append-system-prompt-file.

Use a separate session to review

The exam guide: the session that generated the code is worse at reviewing it than an independent review instance, because it still holds the reasoning that produced the code. Task 4.6 covers this in more depth.

Avoiding duplicate work

• Re-running reviews after new commits: include the previous review's findings and tell Claude to report only new or still-unaddressed issues, so it doesn't post the same comments again.
• Test generation: include the existing test files so Claude doesn't propose tests that already exist.

Domain 3 decision table

Ten things to have cold

1. User-level CLAUDE.md is not shared. "Works for me but not the team" = wrong level.
2. .claude/rules/ with paths: globs beats subdirectory CLAUDE.md when the files are spread across directories.
3. .claude/commands/ = team; ~/.claude/commands/ = personal. .claude/config.json does not exist.
4. The three tested SKILL.md frontmatter options: context: fork, allowed-tools, argument-hint.
5. context: fork runs the skill in a separate subagent context, keeping long output out of the main conversation.
6. Skills load on demand. CLAUDE.md is always loaded.
7. Use plan mode when the requirements already show the task is complex; don't wait for complexity to appear.
8. Explore subagent for discovery that produces a lot of output.
9. -p for CI. CLAUDE_HEADLESS and --batch don't exist.
10. --output-format json + --json-schema for structured CI findings; CLAUDE.md supplies the review criteria.

Domain 4 — Prompt Engineering & Structured Output (20%, ~12 items)

Six task statements. Most questions come from two scenarios: Structured Data Extraction, and Claude Code for CI, whose scenario text says the goal is to minimize false positives.

This domain does not test extended thinking settings, prompt caching mechanics, temperature, prefill or model selection; they are either missing from the objectives or on the out-of-scope list. Extended thinking appears once, in 4.6, as a wrong answer.

In the book: Chapter 12 — Prompt Engineering (4.1, 4.2) · Chapter 13 — Structured Output (4.3, 4.4, 4.5) · Chapter 15 — Escalation & Provenance (4.6)

4.1 Explicit criteria to improve precision and reduce false positives

Specific criteria beat general instructions

A false positive is a finding the model reports that isn't a real problem. The way to reduce them is to state exactly which cases count. The exam guide's example:

The exam guide also says general instructions like "be conservative" or "only report high-confidence findings" do not improve precision compared with specific criteria. An option that tells the model to be more careful or conservative is wrong.

Why false positives cost so much

When one category of finding produces many false positives, developers stop trusting the other categories too, even the accurate ones.

Three skills

• Write criteria that say which issues to report (bugs, security) and which to skip (minor style, local conventions), instead of filtering by the model's confidence.
• Turn off a category with many false positives for a while, to win back developer trust while you improve its prompt. The exam treats this as a correct answer.
• Define explicit severity criteria with concrete code examples for each severity level, so findings are classified consistently.

4.2 Few-shot prompting

Few-shot prompting means putting a few worked examples in the prompt, so the model copies the pattern instead of interpreting a description.

When few-shot is the answer

The exam guide calls few-shot "the most effective technique for achieving consistently formatted, actionable output when detailed instructions alone produce inconsistent results." It names four uses:

1. Demonstrating ambiguous-case handling, such as which tool to pick for an ambiguous request, or how to report branch-level gaps in test coverage.
2. Helping the model generalize to new patterns, not just the cases you listed.
3. Reducing hallucination in extraction (the model inventing values), e.g. informal measurements or documents with varied layouts.
4. Format consistency: showing the exact shape you want.

How many examples, and what they contain

Use 2–4 targeted examples for the ambiguous cases. Each example should explain why one action was chosen over a plausible alternative. An input → reasoning → output example teaches more than an input → output pair.

Other patterns the exam guide names:

• Examples that show a specific output format; the exam guide's is location, issue, severity, suggested fix.
• Examples distinguishing acceptable code patterns from genuine issues, which cuts false positives and still generalizes.
• Examples that cover varied document structures: inline citations vs bibliographies, methodology sections vs details embedded in the text, narrative descriptions vs tables.
• Examples that fix empty/null extraction of required fields by showing correct extraction from several formats.

Criteria (4.1) or examples (4.2)?

Both reduce false positives. Which one is right depends on what is missing:

• The rule itself is undefined or vague → write explicit criteria (4.1).
• The rule is defined, but the model applies it inconsistently, especially on ambiguous cases → add few-shot examples (4.2).

Few-shot is not always the answer. In sample question 2, the agent confuses two tools because their descriptions are thin; there, few-shot examples "add token overhead without fixing the underlying issue", and the fix is better descriptions (Domain 2).

4.3 Structured output via tool use and JSON schemas

How to get structured output

tool_use with a JSON schema is the most reliable way to get output that matches a schema. You define an extraction tool whose input_schema is the structure you want, the model "calls" it, and you read the data from the tool_use block in the response, not from its text.

tool_choice for extraction

A schema guarantees syntax, not correctness

Strict JSON schemas via tool use eliminate syntax errors but do not prevent semantic errors, such as line items that don't add up to the total or values placed in the wrong field.

• The JSON is malformed → the answer is a schema.
• The JSON is well-formed but wrong → the answer is validation logic (4.4), not a stricter schema.

Schema design

Current product (not tested): two newer API features strengthen the schema guarantee:

• strict: true on a tool definition guarantees the tool's input matches the schema. Without it, forced tool_choice plus your own validation is reliable in practice but not guaranteed.
• output_config.format: {"type": "json_schema", ...} guarantees schema-valid JSON with no tool involved; the JSON comes back in a normal text block. This suits extraction jobs like extract_invoice, where the "tool" exists only to get structured output.

Neither checks semantics. A strictly valid extraction can still have stated_total and calculated_total disagree; catching that is still the job of 4.4. There is also a fourth tool_choice value, {"type": "none"}, which forces a text-only reply; the exam guide lists only the three above.

4.4 Validation, retry, and feedback loops

Retry with the error

When an extraction fails validation, retry with the original document, the failed extraction, and the specific validation error. The error tells the model what to correct; a plain retry usually produces the same output.

When retrying won't help

The exam guide: retries are ineffective when the required information is simply absent from the source document. A retry can fix a format or structure error. It cannot produce information that was never there, for example a value that only exists in another document that wasn't provided.

So work out which kind of failure it is before retrying, and send the ones a retry can't fix to human review.

Validation the model does itself

The exam guide names two designs:

• Extract a calculated_total next to the stated_total and flag any difference. This catches a semantic error that a schema can't.
• Add a conflict_detected boolean for when the source data contradicts itself.

detected_pattern

Add a detected_pattern field to each finding, recording which code construct triggered it. You can then look at which patterns developers keep dismissing, which shows where your false positives come from.

4.5 Batch processing

Message Batches API facts

• 50% cost savings compared with normal requests
• Up to 24-hour processing window, with no guaranteed latency SLA
• Does not support multi-turn tool calling within a single request: you can't run a tool mid-request and send its result back
• custom_id on each request lets you match each result to its request

When to use it

Appropriate: non-blocking, latency-tolerant work nobody is waiting on: overnight reports, weekly audits, nightly test generation. Inappropriate: blocking workflows, i.e. work that blocks someone, such as a pre-merge check a developer is waiting for.

Sample question 11 is exactly this split: batch the overnight technical-debt report, keep the pre-merge check on the normal synchronous API. The explanation rejects the argument that "batches are often faster in practice" for a blocking check, and notes that results arriving out of order doesn't matter because custom_id matches them up.

Operational skills

• Choose the API per workflow, by how long it can wait, not once for the whole organization.
• Work out how often to submit from the deadline. The exam guide's example: batches take up to 24 hours, so to guarantee results within 30 hours, submit every 4 hours.
• Handle failures by custom_id: resubmit only the documents that failed, with a change where needed, such as chunking documents that exceeded context limits.
• Tune the prompt on a small sample before running a large batch, so most documents succeed first time and you avoid expensive resubmissions.

4.6 Multi-instance and multi-pass review

Why a model reviews its own work badly

A model retains reasoning context from generation, making it less likely to question its own decisions in the same session. It still has the reasoning that produced the output, so it tends to agree with it. Therefore:

Independent review instances — without the generator's prior reasoning context — are more effective at catching subtle issues than self-review instructions or extended thinking.

Extended thinking is named here as the weaker option. Don't choose it.

Several passes for large reviews

Split a large review into one pass per file plus a separate cross-file integration pass. This avoids attention dilution and contradictory findings: one pass over many files spreads attention thin and can disagree with itself across files. It is the same pattern as task 1.6, and sample question 12 (a 14-file review that comes back shallow and inconsistent) tests it.

Confidence scores: when they are valid

Task 4.6 includes this skill: "Running verification passes where the model self-reports confidence alongside each finding to enable calibrated review routing." That looks like it contradicts sample question 3, where the model's self-reported confidence is the wrong answer for deciding when to escalate. It doesn't:

• Self-reported confidence used on its own to decide to escalate in a live conversation → wrong. It is poorly calibrated, and the agent is often confidently wrong on exactly the hard cases.
• Self-reported confidence checked against a labeled validation set, then used to decide which items human reviewers look at first → right (see task 5.5).

Confidence is fine as an input to a calibrated routing system. It is not fine as the decision itself.

Domain 4 decision table

Ten things to have cold

1. Explicit criteria beat "be conservative" or "only report high-confidence findings."
2. Few-shot examples should show the reasoning, not just input → output. Use 2–4.
3. Schemas fix syntax. They do not fix semantics.
4. Nullable / optional fields stop the model inventing values to fill required fields.
5. "unclear" and "other" + detail are the named enum patterns.
6. Retrying fixes format and structure errors; it can't produce information that isn't in the source.
7. A retry must include the original document, the failed extraction and the specific error.
8. Batch: 50% cheaper, ≤24h, no latency guarantee, no multi-turn tool calling, custom_id to match results. Never for work someone is waiting on.
9. An independent review instance beats self-review, and beats extended thinking, which the exam guide names as the weaker option.
10. Self-reported confidence: invalid as the sole trigger for escalation, valid as a calibrated input for routing human review.

Domain 5 — Context Management & Reliability (15%, ~9 items)

Six task statements. This is the smallest domain by weight, but it is a primary domain in four of the six scenarios (1, 2, 3, 6), so its ideas also appear inside questions from other domains.

It does not test the context-editing API, the memory tool, the compaction API, the 1M-token beta, rate limits or prompt-cache mechanics. "Context management" here means design decisions: what you put in the context window, and what you keep outside it.

In the book: Chapter 14 — Context Engineering (5.1, 5.4) · Chapter 15 — Escalation & Provenance (5.2, 5.5, 5.6) · Chapter 4 — Errors & Recovery (5.3)

5.1 Preserve critical information across long interactions

Four risks the exam guide names

Progressive summarization risk. When a long conversation is summarized again and again (as /compact does in Claude Code), exact numbers, percentages, dates and what the customer said they expect get blurred into vague statements. Those specifics are what you most need to keep.

Lost in the middle. Models reliably process information at the beginning and end of long inputs, but may omit findings from middle sections.

Tool results pile up. Tool results consume tokens disproportionately to their relevance. The exam guide's example: an order lookup returns 40+ fields when only 5 matter.

Full history is required. The API keeps no conversation state, so every request must include the complete conversation history, or the model loses track of the conversation.

Six fixes

The common idea: keep the facts that must survive in a structured block that is never summarized, and let the rest of the conversation be compressed.

Current product (not tested): the memory tool (memory_20250818, a persistent /memories directory) and the context editing API (clear_tool_uses_20250919) are official mechanisms for the case-facts and trim-at-source problems above. They are outside this domain's tested scope.

5.2 Escalation and ambiguity resolution

Exactly three valid escalation triggers

1. The customer asks for a human.
2. The policy has an exception or a gap for this request. The exam guide adds: "not just complex cases." Complexity alone is not a reason.
3. Inability to make meaningful progress.

Two invalid triggers

The exam guide calls sentiment-based escalation and self-reported confidence scores "unreliable proxies for actual case complexity." Sample question 3 (an agent that escalates easy cases and attempts hard ones) rejects both: confidence fails because "the agent is already incorrectly confident on hard cases," and sentiment "solves a different problem entirely; sentiment doesn't correlate with case complexity."

Asking for a human vs being frustrated

Two rules apply side by side:

• If the customer explicitly asks for a human, escalate immediately, without investigating first.
• If the customer is frustrated but hasn't asked for a human, acknowledge the frustration and offer to resolve the issue if you can. Escalate only if they then ask again.

What matters is whether they asked for a human, not how upset they are.

Policy gaps

Escalate when the policy is ambiguous or silent about the specific request. The exam guide's example: a customer asks for a price match against a competitor, but the policy only covers price adjustments on the company's own site. The policy not covering something doesn't mean it's forbidden; a human should decide.

Several matching customers

When a lookup returns multiple customer matches, the agent should ask for additional identifiers, not pick one by guessing.

Fixing an agent that escalates the wrong cases

Sample question 3's answer: write explicit escalation criteria in the system prompt, with few-shot examples of when to escalate and when to resolve. It is the proportionate first fix. A trained classifier, sentiment analysis and confidence thresholds are all wrong.

5.3 Error propagation across multi-agent systems

What a subagent's error report should contain

The type of failure · what was attempted · any partial results · possible alternative approaches. With these four, the coordinator can choose a recovery: retry with a changed query, try another approach, or carry on with the partial results.

Two anti-patterns the exam guide names

• Silently suppressing errors: returning an empty result as if it were a success. Nothing can recover from it, and the final output may be incomplete without anyone knowing.
• Stopping the whole workflow because one step failed.

A third, milder one: generic error statuses such as "search unavailable" hides what the coordinator needs to know. Sample question 8 (a search subagent timing out) rejects an option that retries with backoff and then returns a generic status on this basis, even though the retry part is fine.

Recover locally first

Subagents implement local recovery for transient failures, and pass up only what they can't resolve, including what was attempted and partial results.

Access failure vs valid empty result

The same distinction as in task 2.2, applied to multi-agent systems. A timeout needs a retry decision. A query that succeeded but matched nothing does not.

Coverage annotations

Mark in the synthesis output which findings are well supported and which topics have gaps because a source was unavailable. Partial results are acceptable; partial results presented as complete are not.

5.4 Context in large codebase exploration

Context degradation

The exam guide describes the symptom: in long sessions, the model starts giving inconsistent answers and refers to "typical patterns" instead of the specific classes it found earlier. If a question describes an agent that has drifted into generic advice about a codebase it explored an hour ago, this is the diagnosis.

Five techniques

Current product (not tested): the memory tool closely matches the scratchpad and manifest patterns: a persistent /memories directory Claude reads and writes across sessions. Server-side compaction (compact_20260112) accepts an instructions field, so you can tell it to keep dollar amounts and dates verbatim instead of relying only on a case-facts block.

5.5 Human review workflows and confidence calibration

The main risk

An overall accuracy figure can hide poor accuracy on particular document types or fields. 97% overall can include 60% on one class of document. Check accuracy per document type and per field before letting high-confidence extractions skip human review.

Four techniques

• Stratified random sampling of high-confidence extractions: keep reviewing a random sample from each segment of the extractions the model is sure about, to measure the ongoing error rate and detect novel error patterns. You sample the ones you believe are correct because that is where unnoticed errors collect.
• Accuracy broken down by document type and field, to confirm performance holds in every segment before cutting human review.
• Field-level confidence scores, calibrated against a labeled validation set, to decide where reviewers look. Calibrated means you have checked that, say, 90% confidence really is right about 90% of the time.
• Send to human review any extraction with low confidence, or whose source document is ambiguous or contradictory, so limited reviewer time goes where it's needed.

Confidence is legitimate here because it is per field, calibrated against labeled data, and used to prioritize review, not to make a decision on its own (compare 5.2 and 4.6).

5.6 Provenance and uncertainty in multi-source synthesis

Provenance means being able to trace each claim in the output back to its source.

How it gets lost

Source attribution is lost during summarization steps when findings are compressed without preserving claim-source mappings. The fix is structural: require structured claim-source mappings (source URL, document name, relevant excerpt) that every later agent must preserve and merge through synthesis.

Sources that disagree

Report both values with their sources instead of picking one. If two credible sources give different figures, show both, attributed. More fully: finish the document analysis with the conflicting values included and marked as conflicting, and let the coordinator decide how to reconcile them before synthesis.

Dates

Require publication or data collection dates in structured outputs, to prevent temporal differences from being misinterpreted as contradictions. Two "conflicting" figures may just be from different years.

Current product (not tested): the citations API (citations: {"enabled": true} on a document content block) does claim-to-source mapping automatically when Claude reads a document directly, returning the cited text and its location. You still need your own mapping for handoffs between subagents, where there is no document block, only the previous subagent's findings.

Report structure

• Separate sections distinguishing well-established findings from contested ones, keeping each source's own wording and methodological context.
• Present each kind of content in a suitable form: financial data as tables, news as prose, technical findings as structured lists, rather than one uniform format.

Domain 5 decision table

Ten things to have cold

1. Repeated summarization blurs numbers, dates and stated expectations; keep them in a persistent facts block.
2. Lost in the middle: the start and end of a long input are used reliably, the middle is not. Order inputs accordingly.
3. Trim long tool output at the source, before it builds up.
4. Three valid escalation triggers: customer asks · policy gap · no progress. Sentiment and self-reported confidence are not triggers.
5. Customer explicitly asks for a human → escalate immediately, without investigating first.
6. Several matching customers → ask for another identifier; never guess.
7. A structured error report = failure type + what was attempted + partial results + alternatives.
8. Never return an error as an empty success; never stop the whole workflow over one failure. Mark the coverage gaps instead.
9. Long sessions degrade → scratchpad files, subagent delegation, phase summaries, manifests, /compact.
10. Sources survive synthesis only if claim-to-source mappings are structured and kept at every step. Mark conflicts rather than resolving them, and require dates.

Practice Questions

Forty-five original scenario questions written against the v1.0 task statements and in the style of the exam guide's own samples — D1 ×11, D2 ×8, D3 ×9, D4 ×10, D5 ×7. Two items are multiple-response, as v1.0 says to expect.

These are not Anthropic's questions. The twelve official sample questions are in §9 of the exam guide PDF; do those too, because their answer explanations show best how the exam chooses between options.

Answers and explanations follow all the questions, at the end of the page.

Domain 1 — Agentic Architecture & Orchestration

Q1. (Customer Support Resolution Agent) Your agentic loop terminates as soon as the assistant response contains any text, on the theory that Claude only writes prose when it has a final answer. Support engineers report the agent frequently replies "Let me look that up for you" and then stops. What is the correct fix?

A. Add a system prompt instruction telling Claude not to narrate before calling tools. B. Continue the loop while stop_reason is "tool_use" and terminate only on "end_turn". C. Set a minimum of three iterations before the loop is allowed to exit. D. Strip text content blocks from the response before evaluating the termination condition.

Q2. (Multi-Agent Research System) Your coordinator delegates to a web search subagent and then a synthesis subagent. The synthesis subagent's prompt is "Synthesize the research findings into a report." Its output is generic and cites nothing. What is the root cause?

A. The synthesis subagent's model is too small for synthesis work. B. The synthesis subagent needs the WebSearch tool so it can gather its own material. C. The coordinator did not include the prior agents' findings in the synthesis subagent's prompt; subagents do not inherit parent context. D. The coordinator should invoke synthesis in the same turn as web search so context is shared.

Q3. (Multi-Agent Research System) You want four subagents to run concurrently rather than one after another. How does the coordinator achieve this?

A. Emit multiple Task tool calls in a single coordinator response. B. Set parallel: true in each AgentDefinition. C. Issue one Task call per turn and rely on the SDK to overlap them. D. Set disable_parallel_tool_use to false on the coordinator's requests.

Q4. (Customer Support Resolution Agent) Company policy is that refunds above $500 require human approval. Your system prompt says so clearly. Over three months, four refunds above $500 were processed autonomously. Choose the most appropriate remediation.

A. Restate the rule at both the beginning and end of the system prompt for position emphasis. B. Add few-shot examples showing the agent escalating a $600 refund. C. Intercept outgoing process_refund calls and block any exceeding $500, redirecting to escalation. D. Add a PostToolUse hook on process_refund that flags refunds above $500 for after-the-fact audit.

Q5. (Developer Productivity with Claude) Different MCP servers return timestamps as Unix epochs, ISO 8601 strings, and human-readable dates. The agent frequently miscompares dates. What is the cleanest fix?

A. A PostToolUse hook that normalizes timestamp formats before the model processes the results. B. A system prompt section explaining all three formats and how to convert between them. C. Rewrite each MCP server to emit ISO 8601. D. A tool-call interception hook that rejects calls to tools known to return non-ISO timestamps.

Q6. (Code Generation with Claude Code) You analyzed a codebase in a session two days ago. Since then, six files in the authentication module were rewritten. You want to continue the investigation. What is the best approach?

A. Resume the session with --resume and say nothing; Claude will re-read files as needed. B. Resume the session with --resume and tell it specifically which files changed so it re-analyzes those targets. C. Fork the session so the original analysis is preserved. D. Always start fresh; resumed sessions cannot be trusted after code changes.

Q7. (Developer Productivity with Claude) You want to compare two refactoring strategies, both starting from the same completed codebase analysis, and keep the analysis session intact. Which mechanism?

A. --resume twice with different follow-up prompts. B. fork_session to create independent branches from the shared baseline. C. /compact with instructions focused on each strategy. D. Two new sessions, each seeded with a copy of the analysis summary.

Q8. (Multi-Agent Research System) The coordinator always runs the full pipeline — web search, document analysis, synthesis, report generation — even for a query like "what year was the GDPR enacted?" Latency and cost are high for trivial queries. What should change?

A. Reduce each subagent's max_tokens so trivial queries cost less. B. Have the coordinator analyze query requirements and dynamically select which subagents to invoke. C. Add a caching layer in front of the web search subagent. D. Merge web search and document analysis into one subagent to halve the pipeline.

Q9. (Code Generation with Claude Code) A task reads: "Add comprehensive tests to this legacy service." You do not know the module structure, the existing coverage, or the dependency graph. Which decomposition strategy fits?

A. A fixed prompt chain: generate unit tests, then integration tests, then end-to-end tests. B. Dynamic adaptive decomposition — map structure first, identify high-impact areas, then build a prioritized plan that adapts as dependencies surface. C. Parallel subagents, one per directory, each writing tests independently. D. A single prompt with the whole repository in context.

Q10. (Customer Support Resolution Agent) A customer writes: "My order arrived damaged, I was double-charged, and I want to change my shipping address." Your agent addresses the damage and ignores the other two. Which design change follows the exam guide's recommended pattern?

A. Instruct the agent to ask the customer to submit one issue per message. B. Decompose the request into distinct items, investigate each in parallel using shared context, then synthesize a unified resolution. C. Escalate any message containing more than one issue to a human. D. Process the issues strictly in the order stated, one conversation turn each.

Q11. (Customer Support Resolution Agent) Select TWO. Your agent is escalating to human agents who cannot see the conversation transcript. Which two elements belong in the handoff?

A. The complete raw conversation transcript, verbatim. B. Root cause analysis. C. The agent's self-reported confidence score for the escalation decision. D. Customer ID, refund amount, and recommended action. E. A list of all tools the agent called, with timings.

Domain 2 — Tool Design & MCP Integration

Q12. (Multi-Agent Research System) Two tools, analyze_content ("Analyzes content") and analyze_document ("Analyzes documents"), are selected almost at random. What is the most effective first step?

A. Consolidate them into one analyze tool that branches internally. B. Rename and rewrite descriptions to eliminate overlap — e.g. analyze_content → extract_web_results with a web-specific description. C. Add a deterministic keyword router in front of the model. D. Set tool_choice: "any" so the model must commit to one of them.

Q13. (Customer Support Resolution Agent) lookup_order times out against a flaky backend. Which error response best enables agent recovery?

A. {"error": "Operation failed"} B. isError: true, errorCategory: "transient", isRetryable: true, plus a human-readable description of what was attempted. C. Return an empty order list so the conversation can continue. D. Raise the exception to the top level and end the session.

Q14. (Customer Support Resolution Agent) A customer requests a refund on an item outside the 30-day return window. The tool correctly refuses. What should it return?

A. errorCategory: "transient" with isRetryable: true, so the agent can retry after backoff. B. errorCategory: "business", retriable: false, and a customer-friendly explanation of the policy. C. A generic failure so the agent escalates automatically. D. A successful empty result, since no refund was created.

Q15. (Multi-Agent Research System) Your synthesis agent has been given all 18 tools available in the system "so it never gets stuck." It now attempts web searches mid-synthesis and produces inconsistent reports. What is the best correction?

A. Add a system prompt line telling the synthesis agent not to search. B. Scope the synthesis agent to its role's tools, and add one narrow cross-role tool (verify_fact) for the high-frequency simple case, routing complex verification through the coordinator. C. Remove all tools from the synthesis agent so it can only write. D. Increase the synthesis agent's context window so it retains its instructions better.

Q16. (Developer Productivity with Claude) Your team's custom MCP server exposes a search_internal_docs tool with the description "Searches docs." The agent keeps using Grep on the local repo instead. What fixes this?

A. Move the server from user scope to project scope in .mcp.json. B. Remove Grep from allowedTools. C. Expand the MCP tool's description to explain its capabilities, coverage, and outputs in detail. D. Set tool_choice: "any" to force a tool call.

Q17. (Developer Productivity with Claude) You need a shared GitHub MCP server available to the whole team, authenticated with a token that must not be committed. What do you do?

A. Add it to ~/.claude.json with the token inline and tell everyone to copy the file. B. Add it to .mcp.json with ${GITHUB_TOKEN} and commit that file. C. Add it to .mcp.json with the token inline and add .mcp.json to .gitignore. D. Write a custom MCP server that reads the token from a vault at runtime.

Q18. (Developer Productivity with Claude) You need to find every caller of calculatePremium() across a large TypeScript monorepo. Which tool?

A. Glob with **/*.ts B. Grep C. Read on each file, following imports D. Bash with find

Domain 3 — Claude Code Configuration & Workflows

Q19. (Code Generation with Claude Code) You documented your team's commit message conventions and Claude follows them perfectly for you. Two new hires report Claude ignores them entirely. Where are the conventions?

A. In .claude/rules/ without a paths: field. B. In ~/.claude/CLAUDE.md. C. In .claude/CLAUDE.md, but the new hires haven't run /memory. D. In a skill that requires manual invocation.

Q20. (Code Generation with Claude Code) Your repository has React components, API handlers, and database models, each with different conventions. Test files sit next to the code they test, scattered across every directory, and must all follow one shared testing convention. What is the most maintainable configuration?

A. A CLAUDE.md in each subdirectory. B. All conventions in the root CLAUDE.md under clear headers. C. .claude/rules/ files with YAML paths: glob patterns, including **/*.test.tsx for tests. D. A skill per code type containing the relevant conventions.

Q21. (Code Generation with Claude Code) A /codebase-map skill produces 8,000 tokens of directory listings and dependency graphs every time it runs, crowding out the actual work. Which frontmatter option addresses this?

A. allowed-tools: Read, Grep B. argument-hint: [directory] C. context: fork D. model: haiku

Q22. (Code Generation with Claude Code) Which statement correctly distinguishes skills from CLAUDE.md?

A. Skills are always loaded; CLAUDE.md is loaded on demand. B. Skills are for on-demand task-specific workflows; CLAUDE.md is for always-loaded universal standards. C. Skills are project-scoped only; CLAUDE.md can be user- or project-scoped. D. Skills replace CLAUDE.md in projects that use them.

Q23. (Code Generation with Claude Code) You need to migrate the application from one HTTP client library to another. It touches roughly 45 files, and there are two viable migration strategies with different testing implications. What is the right approach?

A. Direct execution with a very detailed instruction listing every file. B. Plan mode to explore and design, then direct execution to implement the chosen plan. C. Direct execution, switching to plan mode only if something unexpected appears. D. Split the work into 45 single-file direct-execution tasks.

Q24. (Claude Code for CI) Your pipeline runs claude "Review this PR" and the job times out with no output. What is the fix?

A. claude -p "Review this PR" B. export CLAUDE_HEADLESS=true C. claude --batch "Review this PR" D. claude "Review this PR" < /dev/null

Q25. (Claude Code for CI) You want Claude Code's review findings posted as inline PR comments by a downstream script. Which combination produces reliably parseable output?

A. --output-format json together with --json-schema B. A prompt instructing Claude to reply in JSON only C. --output-format text and a regex parser D. --json-schema alone, since it implies JSON output

Q26. (Claude Code for CI) Every time a developer pushes a new commit, the review bot re-posts the same twelve comments from the previous run. What change fixes this?

A. Only run the review on the first push of a PR. B. Include the prior review findings in context and instruct Claude to report only new or still-unaddressed issues. C. Deduplicate comments in the posting script by hashing their text. D. Reduce the review scope to only the files changed in the latest commit.

Q27. (Code Generation with Claude Code) You describe a data transformation in prose three times and get three different interpretations. What is the exam guide's recommended technique?

A. Move the description into CLAUDE.md so it is always loaded. B. Provide 2–3 concrete input/output examples. C. Ask Claude to restate the requirement before implementing. D. Increase specificity by adding more prose constraints.

Domain 4 — Prompt Engineering & Structured Output

Q28. (Claude Code for CI) Your review bot flags too many false positives on the "misleading comment" category. Your current instruction is "check that comments are accurate." Which change most improves precision?

A. Append "be conservative and only report high-confidence findings." B. Replace it with "flag comments only when the claimed behavior contradicts the actual code behavior." C. Require the model to output a confidence score and filter below 0.8. D. Run the review three times and report only issues appearing in all three.

Q29. (Claude Code for CI) Six months in, the "security" category is 92% precise and the "performance" category is 30% precise. Developers have started ignoring all bot comments. What is a legitimate immediate action?

A. Temporarily disable the performance category while you improve its prompt. B. Lower the overall reporting threshold so fewer findings surface across all categories. C. Switch to a larger model for the performance pass only. D. Post all findings but mark performance ones as "low confidence."

Q30. (Structured Data Extraction) Your extractor returns well-formed JSON that validates against the schema, but on 8% of invoices the line items do not sum to the stated total. What addresses this?

A. Enable strict mode on the tool schema. B. Extract calculated_total alongside stated_total and flag discrepancies in a validation step. C. Add additionalProperties: false to the schema. D. Switch tool_choice from "auto" to "any".

Q31. (Structured Data Extraction) Roughly a fifth of your source contracts have no governing-law clause. The model invents a plausible jurisdiction for those. The governing_law field is currently required. What is the fix?

A. Add a few-shot example showing a contract with no governing law. B. Make governing_law optional/nullable so the model can return null. C. Add "do not guess" to the system prompt. D. Lower the temperature to 0.

Q32. (Structured Data Extraction) You receive three document types and cannot tell which type a document is until you read it. You have a distinct extraction tool per type and must never get conversational prose back. Which tool_choice?

A. "auto" B. "any" C. {"type": "tool", "name": "extract_invoice"} D. "none" with a prefilled response

Q33. (Structured Data Extraction) Validation fails because total_paid is absent. Investigation shows the figure lives in a payment remittance file that was never supplied to the model. What should the pipeline do?

A. Retry with the validation error appended; the model will locate it. B. Retry up to five times with exponential backoff. C. Recognize that retry cannot recover absent information, and route the document to human review. D. Make the field nullable and accept null as correct.

Q34. (Structured Data Extraction) Select TWO. Which statements about the Message Batches API are accurate?

A. It offers 50% cost savings. B. It guarantees completion within one hour. C. It supports multi-turn tool calling within a single request. D. custom_id correlates requests with responses. E. It is appropriate for blocking pre-merge checks because it is usually fast.

Q35. (Structured Data Extraction) A batch of 100 documents completes with 7 failures, all because the documents exceeded context limits. What is the right recovery?

A. Resubmit the entire batch with a larger model. B. Identify the 7 by custom_id, chunk them, and resubmit only those. C. Process all 100 synchronously instead. D. Reduce max_tokens and resubmit the whole batch.

Q36. (Claude Code for CI) Claude generates a module, then you ask it in the same session to review its own work. It finds nothing. What is the most effective architecture?

A. Enable extended thinking on the review request. B. Ask it to review the code three times and take the union of findings. C. Send the code to a second, independent Claude instance without the generator's reasoning context. D. Add "be critical of your own work" to the review prompt.

Domain 5 — Context Management & Reliability

Q37. (Customer Support Resolution Agent) Forty turns into a billing dispute, the agent refers to "the disputed amount" and "the charge from last month" instead of $247.80 and 14 August. Summarization has been running throughout. What fixes it?

A. Truncate the history to the last ten turns and drop everything earlier. B. Extract transactional facts — amounts, dates, order numbers, statuses — into a persistent case-facts block included in each prompt, outside the summarized history. C. Instruct the summarizer to be more detailed. D. Move to a model with a larger context window.

Q38. (Customer Support Resolution Agent) Your agent responds to "I've had enough, put me through to a person" by first running lookup_order and offering a replacement. What is correct?

A. Escalate immediately, without first attempting investigation. B. Attempt one resolution; escalate if the customer repeats the request. C. Run sentiment analysis to confirm the frustration is genuine before escalating. D. Escalate only if the agent's confidence in resolving the case is below threshold.

Q39. (Multi-Agent Research System) A document analysis subagent times out after processing 3 of 5 papers. Which propagation approach best enables coordinator recovery?

A. Return structured error context: failure type, what was attempted, the 3 completed analyses as partial results, and possible alternative approaches. B. Retry internally with backoff, then return "document analysis unavailable." C. Return the 3 completed analyses marked as a successful, complete result. D. Propagate the timeout to a top-level handler that aborts the research run.

Q40. (Multi-Agent Research System) Two credible sources report the market size as $4.2B and $6.8B. Your synthesis agent silently picks $4.2B. What should it do instead, and what would have prevented the misreading in the first place?

A. Average the two figures; require confidence scores from each source. B. Pick the higher figure as a conservative upper bound; require source URLs. C. Annotate both values with source attribution and let the report present the conflict; require publication or data-collection dates so temporal differences are not mistaken for contradictions. D. Escalate to a human reviewer; require the coordinator to deduplicate sources before synthesis.

Additional items — under-drilled task statements

Q41. (D4 · Claude Code for CI) Your review criteria are specific and well-defined, and precision is good on clear-cut cases. But on borderline code — a broad catch that is arguably intentional, a magic number that may be a domain constant — the bot's verdicts swing run to run. What most improves consistency?

A. Add "when uncertain, do not report the issue" to the criteria. B. Add 2–4 few-shot examples of ambiguous cases showing the reasoning for why one verdict was chosen over the plausible alternative. C. Enumerate every borderline pattern in the codebase as an explicit exception list. D. Increase the number of review passes and report only findings that recur.

Q42. (D5 · Developer Productivity with Claude) Three hours into exploring a legacy payments service, the agent starts answering questions with "typically, services like this validate at the controller layer" instead of naming the PaymentValidator class it identified earlier. What is happening, and what fixes it?

A. The system prompt has been diluted; restate it every ten turns. B. Context degradation in an extended session; have the agent maintain a scratchpad file recording key findings and reference it for subsequent questions. C. The model is hallucinating; lower the temperature. D. Tool results were malformed; add a PostToolUse normalization hook.

Q43. (D5 · Structured Data Extraction) Your extraction pipeline reports 97% overall accuracy and you want to cut human review. What must you verify first?

A. That the 3% of errors are evenly distributed across reviewers. B. That accuracy is consistent when segmented by document type and by field, since an aggregate figure can mask poor performance on specific segments. C. That the model's self-reported confidence exceeds 0.9 on the 97%. D. That the validation set is at least 10,000 documents.

Q44. (D5 · Structured Data Extraction) Having reduced review to low-confidence extractions only, how do you keep detecting new error patterns in the high-confidence bucket you no longer inspect?

A. Wait for downstream systems to report anomalies. B. Stratified random sampling of high-confidence extractions for ongoing error-rate measurement and novel pattern detection. C. Re-run every high-confidence extraction with a second model and compare. D. Raise the confidence threshold each quarter.

Q45. (D2 · Developer Productivity with Claude) Your internal MCP server fronts a documentation system with 4,000 pages across a deep hierarchy. Agents burn six or seven exploratory search_docs calls just working out what topics exist before asking a useful question. What is the appropriate MCP mechanism?

A. Increase the result count returned by search_docs. B. Expose the documentation hierarchy as an MCP resource so agents can see the content catalog without exploratory tool calls. C. Add a list_topics tool and instruct the agent to always call it first. D. Cache search_docs results so repeat exploration is cheap.

---

Answers

Q1 — B. Loop control flow must be driven by stop_reason. The exam guide names "checking for assistant text content as a completion indicator" as an anti-pattern: a tool_use response can and often does also contain text. A treats a structural bug as a prompting problem; C substitutes an arbitrary iteration count for a real signal; D still uses the wrong signal, just more carefully.

Q2 — C. Subagents operate with isolated context and do not inherit the coordinator's conversation history. The coordinator must include complete findings from prior agents directly in the synthesis prompt. B would work around the problem by making synthesis re-do the research — over-provisioning and violating separation of concerns. D misunderstands the mechanism; there is no shared turn context.

Q3 — A. Parallel execution means emitting multiple Task tool calls in a single coordinator response. B invents a field. C is exactly the sequential pattern. D names a real Messages API parameter but it governs parallel tool calls, not subagent orchestration — and it appears nowhere in the exam guide's API appendix.

Q4 — C. Deterministic compliance is required — the consequence is financial — and prompt instructions have a non-zero failure rate. Tool-call interception blocks the action before it happens. A and B are both probabilistic. D is the right hook family but the wrong direction: PostToolUse fires after execution, so the refund has already gone out.

Q5 — A. PostToolUse hooks intercepting tool results for transformation before the model processes them is the exam guide's named pattern for exactly this — normalizing heterogeneous formats from different MCP tools. B is probabilistic and burns tokens on every turn. C may be impossible for third-party servers and is more work. D blocks useful tools rather than fixing their output.

Q6 — B. Resume when prior context is mostly valid, and inform the resumed session about specific file changes for targeted re-analysis rather than full re-exploration. A wastes the benefit and risks stale reasoning. C solves a different problem (divergent branches). D overstates: the exam guide prefers a fresh session with a structured summary only when prior tool results are stale, which six changed files out of a whole codebase does not necessarily make everything.

Q7 — B. fork_session creates independent branches from a shared analysis baseline for exploring divergent approaches — the use case the exam guide states. A pollutes one session with two strategies. C compresses, it doesn't branch. D discards the detailed analysis you already paid for.

Q8 — B. The coordinator's role includes deciding which subagents to invoke based on query complexity, dynamically selecting rather than always routing through the full pipeline. A degrades quality without addressing the structure. C and D are narrower optimizations that leave the routing problem intact.

Q9 — B. Open-ended investigation calls for dynamic adaptive decomposition: map structure, identify high-impact areas, build a prioritized plan that adapts as dependencies are discovered. This is the exam guide's own worked example for "add comprehensive tests to a legacy codebase." A imposes a fixed chain on an unpredictable task. C guarantees duplication and no coherent prioritization. D invites attention dilution.

Q10 — B. Decompose multi-concern requests into distinct items, investigate each in parallel using shared context, then synthesize a unified resolution. A and C push work back onto the customer or a human for a case the agent can handle. D is serial and slow, and risks the same drop-off.

Q11 — B and D. The exam guide specifies structured handoff summaries containing customer ID, root cause, refund amount, and recommended action, for human agents who lack transcript access. A is the raw dump the pattern exists to replace. C is an unreliable signal. E is operational telemetry, not what the human needs to act.

Q12 — B. Tool descriptions are the primary mechanism for tool selection; the exam guide's own repair for this exact pair is renaming and rewriting to eliminate functional overlap. A (consolidation) is a defensible architecture but more work than a first step. C over-engineers and bypasses the model's language understanding. D forces a tool call without improving which tool.

Q13 — B. Structured error metadata — category, retryable flag, human-readable description of what was attempted — is what lets the agent make an appropriate recovery decision. A is the named anti-pattern ("Operation failed"). C silently suppresses a failure as success. D terminates the workflow over a recoverable fault.

Q14 — B. Business errors are non-retryable and need a customer-friendly explanation so the agent can communicate rather than retry. A would cause pointless retries of a policy decision. C throws away the reason. D conflates a policy refusal with an empty result set.

Q15 — B. Too many tools degrades selection, and out-of-specialization tools get misused. The exam guide's pattern is scoped tool access plus a limited cross-role tool for a specific high-frequency need. A is probabilistic. C removes a real capability the agent needs 85% of the time. D misdiagnoses the cause.

Q16 — C. The named skill is enhancing MCP tool descriptions to explain capabilities and outputs in detail, to prevent the agent from preferring built-in tools like Grep. A confuses scope with priority — scope controls sharing, not selection. B cripples a useful tool. D forces a call without steering which.

Q17 — B. Project-scoped .mcp.json for shared team tooling, with environment variable expansion for credentials so nothing secret is committed. A is user scope and leaks the token. C loses the sharing that motivated the change. D reinvents what env var expansion already does.

Q18 — B. Grep searches file contents for patterns such as function names. Glob matches file paths, which finds the files but not the callers. C is the read-everything-upfront anti-pattern. D shells out where a dedicated tool exists.

Q19 — B. User-level ~/.claude/CLAUDE.md applies only to that user and is not shared via version control — the exam guide names this exact diagnostic. A would still be committed and would still load (a rule without paths: just isn't conditionally scoped). C is contradicted by "ignores them entirely." D doesn't match "follows them perfectly for you" without manual invocation.

Q20 — C. .claude/rules/ with glob patterns applies conventions by file type regardless of directory location — essential when test files are spread throughout. A is directory-bound and can't cover scattered tests. B relies on inference rather than explicit matching. D requires invocation and is not automatic.

Q21 — C. context: fork runs the skill in an isolated sub-agent context, preventing verbose skill output from polluting the main conversation. A restricts tools, not context. B prompts for arguments. D is not one of the frontmatter options the exam guide tests, and a cheaper model would still emit 8,000 tokens.

Q22 — B. Skills are on-demand invocation for task-specific workflows; CLAUDE.md is always-loaded universal standards. A inverts it. C is false — personal skills live in ~/.claude/skills/. D is not a trade-off the exam guide draws.

Q23 — B. Plan mode is for large-scale changes with multiple valid approaches and architectural implications; combining plan mode for investigation with direct execution for implementation is a named skill. A assumes you already know the answer. C ignores that the complexity is stated in the requirements, not something that might emerge. D loses the cross-file coherence the migration needs.

Q24 — A. -p / --print is the documented non-interactive mode. CLAUDE_HEADLESS and --batch do not exist. Redirecting stdin is a Unix workaround that doesn't address the command syntax.

Q25 — A. --output-format json with --json-schema produces machine-parseable structured findings for automated posting as inline PR comments. B is unenforced. C is fragile. D breaks the exam guide's pairing in half — it only ever names the two flags together.

Q26 — B. Include prior review findings in context and instruct Claude to report only new or still-unaddressed issues. A abandons the feature. C hides the symptom and breaks when wording shifts slightly. D loses issues in unchanged files that interact with the change.

Q27 — B. Concrete input/output examples are the exam guide's named remedy when prose descriptions are interpreted inconsistently, and it specifies 2–3 of them. A changes where the ambiguity lives, not whether it's ambiguous. C is the interview pattern, aimed at surfacing unknown considerations rather than pinning down a known transformation. D is more of what already failed.

Q28 — B. Explicit categorical criteria beat vague instructions — this is the exam guide's own before/after pair. A is named as a technique that fails to improve precision. C relies on poorly calibrated self-reported confidence. D suppresses real bugs that only some passes catch (the exam guide's reason for rejecting 2-of-3 consensus in sample question 12, the 14-file review).

Q29 — A. Temporarily disabling high false-positive categories to restore developer trust while improving those prompts is a named skill. High false positives in one category undermine confidence in the accurate ones, so protecting trust is the priority. B degrades the good category too. C treats a criteria problem as a capability problem. D still spends the developer's attention.

Q30 — B. Schemas eliminate syntax errors but not semantic ones; the exam guide's named self-correction flow is extracting calculated_total alongside stated_total to flag discrepancies. A, C and D are all schema-level, and the JSON already validates.

Q31 — B. Designing fields as optional/nullable where source documents may not contain the information is the exam guide's stated way to prevent the model fabricating values to satisfy required fields. A helps at the margin but leaves the structural pressure in place. C is a prompt instruction against a schema-level incentive. D is not a lever the exam guide tests and wouldn't stop the fabrication.

Q32 — B. tool_choice: "any" guarantees the model calls a tool while letting it choose which — the exam guide's named use for exactly this case (multiple extraction schemas, unknown document type). A permits a text reply. C forces the wrong schema two thirds of the time. D disallows tools entirely.

Q33 — C. Retries are ineffective when the required information is simply absent from the source; the skill tested is identifying when retry will not help. A and B burn cost on an unrecoverable case. D would silently record a null for a value that exists — wrong, and it hides a real data-pipeline gap.

Q34 — A and D. 50% cost savings and custom_id correlation are both stated. B is wrong — up to 24 hours with no guaranteed latency SLA. C is wrong — batch does not support multi-turn tool calling within a request. E is the argument sample question 11 (batch vs a blocking pre-merge check) rejects.

Q35 — B. Handle batch failures by resubmitting only the failed documents, identified by custom_id, with appropriate modifications such as chunking oversized documents. A, C and D all reprocess 93 documents that already succeeded.

Q36 — C. A model retains its reasoning context from generation and is less likely to question its own decisions in the same session; independent review instances are more effective than self-review instructions or extended thinking. A is named as inferior. B is still self-review, three times. D is a self-review instruction.

Q37 — B. Extracting transactional facts into a persistent case-facts block included in each prompt, outside the summarized history, is the exam guide's named fix for progressive summarization condensing numbers and dates. A discards exactly the early transactional detail the customer will hold you to, and the exam guide is explicit that complete conversation history matters for coherence. C asks the summarizer to stop being a summarizer. D is a capacity answer to a curation problem.

Q38 — A. Honour explicit customer requests for a human immediately, without first attempting investigation. B is the correct behaviour for frustration without an explicit request — the customer here asked. C and D are the two named unreliable proxies.

Q39 — A. Structured error context — failure type, what was attempted, partial results, potential alternatives — lets the coordinator decide whether to retry, substitute, or proceed with coverage annotations. B's generic status hides that context. C is silent suppression, the named anti-pattern. D terminates the whole workflow over one recoverable failure.

Q40 — C. Conflicting statistics from credible sources should be annotated with source attribution rather than arbitrarily resolved, and requiring publication or data-collection dates in structured outputs prevents temporal differences from being misread as contradictions. A fabricates a figure neither source reported. B is arbitrary selection with a rationalization. D escalates a case the system is designed to represent, and deduplication doesn't address really different measurements.

Q41 — B. Few-shot examples are the exam guide's named technique for demonstrating ambiguous-case handling, and it specifies 2–4 targeted examples that show the reasoning for why one action was chosen over a plausible alternative. A is a "be conservative"-class instruction, which the exam guide says fails to improve precision. C doesn't generalize to novel patterns — the exact benefit few-shot is credited with. D suppresses real intermittent findings.

Q42 — B. The exam guide describes context degradation in extended sessions as models "giving inconsistent answers and referencing typical patterns rather than specific classes discovered earlier" — this stem is that symptom verbatim. The named remedy is scratchpad files persisting key findings across context boundaries. A, C and D all misdiagnose a context problem as a prompting, sampling, or data-format problem.

Q43 — B. Aggregate accuracy metrics can mask poor performance on specific document types or fields; the exam guide requires analyzing accuracy by document type and field before reducing human review. A misreads whose errors matter. C leans on uncalibrated confidence. D is volume without segmentation, which is exactly what hides the problem.

Q44 — B. Stratified random sampling of high-confidence extractions is the named technique for ongoing error-rate measurement and novel pattern detection. A is reactive and slow. C costs as much as the review you just eliminated. D trades recall for a false sense of safety without ever looking at what's in the bucket.

Q45 — B. MCP resources expose content catalogs — the exam guide's own examples are issue summaries, documentation hierarchies, and database schemas — specifically to reduce exploratory tool calls. Remember the split: resources for content catalogs, tools for actions. A returns more noise per call. C adds a tool where a resource is the designed mechanism. D makes an unnecessary pattern cheaper rather than removing it.

Scoring yourself

Out of 45. Below 34 and you have real gaps — go back to the domain page for whichever section you missed most. Above 39 and you are in good shape; spend remaining time on the domain pages' "ten things to have cold" lists and on the twelve official sample questions.

Quick Reference — final-hour review

One page to read on the morning of the exam.

Exam mechanics

60 items · 120 min (2 min/item) · 4 of 6 scenarios · pass 720/1000 scaled · multiple-choice and multiple-response — read how many to select.

D1 27% · D2 18% · D3 20% · D4 20% · D5 15%.

Five rules that answer most questions

1. When a mistake has real consequences, enforce the rule in code. Money, compliance, security, identity checks, required order of steps → prerequisite gate or hook. Not a system prompt instruction, not few-shot examples.
2. Otherwise, pick the smallest fix that addresses the root cause. The exam guide's answers reject "over-engineered" options (routing classifiers, trained models, extra infrastructure) when rewriting a description or writing explicit criteria would work.
3. Fix the root cause, not the symptom. If every downstream component works, the problem is upstream. If tool descriptions are thin, fix the descriptions rather than adding examples.
4. Anything that must survive a handoff goes in structured fields. Sources, error details, case facts and escalation summaries should be fields, not prose the next step has to interpret.
5. A bigger model or a bigger context window is almost never the answer.

Phrases in the question that decide the answer

Fixed facts

stop_reason — "tool_use" → continue the loop · "end_turn" → stop. On the exam, nothing else ends a loop. (Current API also has stop_sequence and, on Claude 4.5+, model_context_window_exceeded; not tested.)

Spawning subagents — the Task tool; allowedTools must include "Task"; parallel = multiple Task calls in one response; subagents inherit nothing. (The current SDK calls it the Agent tool, with Task kept as an alias; the exam says Task.)

tool_choice — "auto" may return text · "any" must call some tool · {"type":"tool","name":"x"} forces one. (Current API also has {"type":"none"}, forcing a text-only reply; not tested.)

Error categories — transient · validation · business · permission. Transient is the retryable one; business violations carry retriable: false plus a customer-friendly explanation. Payload: errorCategory, isRetryable, human-readable description. MCP flag is isError.

Empty result ≠ error. Timeout = error. Zero matches = success.

Tool count — ~4–5 per agent; 18 degrades selection.

Batch API — 50% off · ≤24h · no SLA · no multi-turn tool calling · custom_id correlates and drives selective resubmission.

CI flags — -p / --print · --output-format json · --json-schema. (CLAUDE_HEADLESS and --batch don't exist.)

File and directory map

~/.claude/CLAUDE.md          user-level, NOT shared
~/.claude/commands/          personal slash commands
~/.claude/skills/            personal skills (rename to avoid clashing)
~/.claude.json               user-scope MCP servers (personal/experimental)

CLAUDE.md  or  .claude/CLAUDE.md    project-level, committed
<subdir>/CLAUDE.md                  directory-scoped conventions
.claude/rules/*.md                  topic files; YAML `paths:` globs for conditional loading
.claude/commands/*.md               team slash commands, version-controlled
.claude/skills/<name>/SKILL.md      team skills
.mcp.json                           project-scope MCP servers, ${ENV_VAR} for secrets

SKILL.md frontmatter tested: context: fork · allowed-tools · argument-hint. Commands used: /memory (check which memory files are loaded; exam answer. Current product uses /context for this) · /compact · --resume <session-name> · fork_session.

Current product (not tested): there is also a managed (enterprise) policy CLAUDE.md, a gitignored project-root CLAUDE.local.md (personal, one project only), and a third MCP scope, local (one project, private, not in git).

Built-in tools

Read · Write · Edit · Bash · Grep · Glob. Task 2.5 names all six.

Grep = search file contents. Glob = match file paths/names. Read/Write = whole file. Edit = targeted, needs unique anchor text → on failure fall back to Read + Write. Bash exists, but don't shell out where a dedicated tool does the job. Exploration order: Grep for entry points → Read to follow imports. Don't read everything up front.

Confidence scores: when they are valid

Few-shot vs explicit criteria vs enforcement

Escalation triggers

Valid: customer asks for a human (escalate immediately, no investigation first) · policy exception or gap · no meaningful progress

Invalid: sentiment analysis · self-reported confidence · "the case seems complex" · turn count

Provenance checklist for multi-source synthesis

Structured claim-to-source mappings (URL, document name, excerpt) kept at every step · publication or collection dates required · conflicts reported with both sources, never silently resolved · separate report sections for well-established and contested findings · coverage annotations for gaps · each content type in a suitable form (tables for financial data, prose for news, lists for technical findings).

Off the table

Fine-tuning and training custom models · Claude API auth, billing, account management · language- and framework-specific implementation detail · deploying or hosting MCP servers (infrastructure, networking, containers) · Claude's internal architecture and weights · Constitutional AI / RLHF · embedding models and vector DBs · computer use · vision · streaming / SSE · rate limits, quotas, pricing calculations · OAuth and key rotation · cloud provider config · performance benchmarking · prompt caching beyond knowing it exists · tokenization.

An option that depends on one of these is very likely a distractor.

Preface — from using Claude Code to building with Claude

You drive Claude Code well: CLAUDE.md, plan mode, slash commands, MCP servers, and good instincts about when Claude will recover from a failing test. Many of the exam's judgement questions reward exactly those instincts.

What you haven't done is build the thing. You haven't sent a Messages API request, written the while loop that turns a stateless call into an agent, authored an MCP server, or designed a schema that stops a model fabricating values. The exam assumes that work and asks about stop_reason, tool_result, AgentDefinition, and tool_choice, the machinery the product hides from you.

The gap is smaller than it looks. You have already operated every architecture the exam tests, from the outside. The fastest route is to attach each exam term to behaviour you've already seen.

What you already know, and what it is called here

Use the right column to find the chapter for a row you're shaky on. Use the left column when a chapter feels abstract.

The four things you have never done

1. Written the loop. Chapter 2 is the most important chapter for you. Do its exercises, don't just read it. About forty lines turn half of Domain 1 from reasoning into recognition.
2. Authored a tool. A tool description is a prompt competing for the model's attention, not documentation (Chapter 3). Chapter 8 makes you an MCP author, not just a consumer.
3. Designed for structured output. tool_use with a JSON schema eliminates malformed JSON and does nothing about line items that don't sum (Chapter 13).
4. Made behaviour reliable at volume. Explicit categorical criteria, few-shot examples that show reasoning, and why "be more conservative" never fixes precision (Chapter 12).

How the book is arranged

Fifteen chapters. Each of the 30 task statements has exactly one home chapter.

• 1–8 build the agent: patterns, loop, tools, errors, orchestration, hooks, sessions, MCP.
• 9–11 cover Claude Code configuration. You'll move fastest here, because you need the vocabulary and exact paths, not the concepts.
• 12–15 cover prompting, structured output, context, and Domain 5's reliability judgements.

Short on time? Chapter 2 is non-negotiable, Chapter 6 has the highest-yield idea on the exam, and Chapters 9–11 can be skimmed for the configuration details.

Each chapter ends with What the exam tests (task statements and judgement calls) and Exercises. For the four gaps above, the exercises are worth more than the text.

Where the exam guide and the product disagree

The guide dates from July 2026, and Claude Code has moved since. Eight points no longer match current docs:

• Wrong semantics (3): allowedTools gating subagent delegation; allowed-tools restricting a skill's tools; Read + Write as the fallback when Edit hits a non-unique match.
• Overtaken (4): Task renamed to Agent; --resume taking a session name; /memory reporting what's loaded; MCP having two scopes rather than three.
• Cosmetic (1).

Your instincts will be right about the product and wrong about the exam on these. Answer the guide's version. Chapters keep the guide's answer in the main text and add a marked note on current reality. The full list with sources is in REVIEW-FINDINGS.md at the repo root. Read it once before the exam.

Seven of the eight sit in the Claude Code / Agent SDK configuration surface. Domains 4 and 5 are essentially unaffected. The material that feels most familiar is the most likely to have drifted.

A note on the code

Substantive examples come in Python then TypeScript, using the exam scenarios' tool names (get_customer, lookup_order, process_refund, escalate_to_human, extract_invoice). Four of the six scenarios appear on your form, and the same mechanic recurs across them in different costumes. A deterministic gate on a refund is the same idea as a gate in a CI pipeline, and the exam will test whichever costume you're less comfortable with.

Chapter 1 — Agents, workflows, and knowing which one you need

Most production failures with Claude are architecture failures, not model failures. A support agent refunds the wrong account because identity verification was requested in a prompt rather than enforced in code. A 14-file review contradicts itself because one call was asked to do 14 jobs. The central question is how much control flow lives in your code and how much in the model's judgement. Too little structure gives you non-determinism where you needed guarantees. Too much gives you a brittle, expensive state machine.

Domain 1 is 27% of the exam, the heaviest domain. Its questions are diagnostic: a system misbehaves in a described way, and you pick the fix. Distractors are usually sound engineering aimed at the wrong problem, or more machinery than the failure needs.

Claude Code is itself an agent built from the primitives this chapter covers: a loop, tools, a system prompt, and gates. The vocabulary comes from Anthropic's "Building effective agents" (workflow vs agent, the pattern catalogue) and the Agent SDK work (the loop).

The augmented LLM

The building block is the augmented LLM: a model plus tools, retrieval, and memory.

Underneath it is the Messages API, a single stateless call. It takes a system prompt, alternating user and assistant messages, and optional tool definitions, and returns one assistant message. It has no memory, cannot run anything, and cannot read files. Everything Claude Code does beyond that (reading the repo, editing, running tests, remembering earlier steps) is scaffolding built around this call. This book teaches you to build that scaffolding.

Each augmentation is a decision surface:

• Tools set what the model can do, and their descriptions steer what it will do. A description is a prompt (Chapter 3). When Claude Code picks the wrong MCP tool, a description has underperformed.
• Retrieval sets what the model knows when it acts: pre-fetched (CLAUDE.md, injected up front) or fetched on demand (the file Claude opens on turn six).
• Memory sets what survives between turns and sessions. Subagents inherit nothing except their prompt, which is why a Claude Code subagent can't see your main thread.

Read each exam scenario by which augmentation carries the weight. The support agent's refund depends on memory of a verified customer ID. The research system's hard problem is what the coordinator carries between subagents. Extraction barely needs tools and lives or dies on schema design.

Workflows and agents

Exam options are written against Anthropic's exact definitions:

• Workflow: LLMs and tools orchestrated through predefined code paths. You write the sequence, and the model fills in the steps.
• Agent: the LLM dynamically directs its own process and tool usage. You give it a goal and tools, and it decides each next step from what it just learned.
• Agentic systems: the umbrella term for both. It does not imply autonomy.

In Claude Code, a bare request is an agent: nobody told it to grep, read three files, edit, then test. A slash command whose body lists the steps in order is a workflow. Same model, same tools; only where the control flow lives differs.

The distinction is architectural, not a measure of intelligence. Task statement 1.1 tests it as model-driven decision-making versus pre-configured decision trees or fixed tool sequences. Neither is the default answer. The deciding question is whether you can enumerate the paths in advance.

The agentic loop

Anthropic's loop: gather context → take action → verify work → repeat.

• Gather: fetch what's needed rather than being handed everything, e.g. grep/tail on a log instead of reading all of it.
• Act: call a tool that changes something.
• Verify: check the result against the goal.

Teams skip verification because in most domains it is expensive or impossible to automate. Nothing cheaply tells you whether a summary is good or a report has gaps. The loop decays into gather-act-gather-act, and errors compound unnoticed.

Coding became the first strong agent domain because it has free, fast, deterministic verifiers: compiler, type checker, tests. Outside coding, ask what is my test suite? It might be a JSON Schema validator (Chapter 13), or a second model call with a rubric (evaluator-optimizer, below). If no verifier exists, keep the design a workflow with human review rather than an autonomous agent.

Mechanically, the loop is a while you write. The API is stateless, so you own the message list, resending it every iteration. Each response's stop_reason says whether Claude is done or wants a tool run. Termination is a protocol signal, not a judgement call. Chapter 2 builds the loop.

The five workflow patterns

Five workflows plus the autonomous agent. For each, learn the condition that makes it right.

Details the exam turns on:

• Chaining: the gates are what matter. Without them you have a long prompt split across calls. With them, a malformed extraction is caught at step 2 instead of step 5. Plan mode is a gate: nothing executes until the plan clears a check. The blueprint's example is per-file review passes plus a separate cross-file integration pass.
• Parallelization has two forms. Sectioning runs independent subtasks at once, e.g. one instance answers while another screens for policy violations. Voting runs the same task several times to raise recall. Trap: requiring 2-of-3 agreement suppresses real findings rather than filtering false ones.
• Parallelization vs orchestrator-workers: can you write the subtask list before seeing the input? If yes, it's parallelization and the split belongs in code. If no, it's orchestrator-workers and a model must decompose. When Claude Code decides for itself to spawn subagents, that is orchestrator-workers.
• Orchestrator-workers' signature failure: every worker succeeds and the aggregate is still wrong. The defect is upstream, in the decomposition. Every subagent returns a correct answer about the wrong three files.
• Evaluator-optimizer loops uselessly when no reliable criterion exists, because the critic can't tell good from bad.
• Autonomous agents trade cost and latency for adaptability, so they need guardrails: sandboxing, checkpoints, human review at consequential boundaries. Claude Code's permission prompts are those guardrails. In your own agent, you build them.

Choosing

Ask in order:

1. Can you draw the decision tree? If it's finite on a whiteboard, build the workflow. Readable code is cheaper to debug than a reasoning trace and doesn't vary between runs.
2. Are the subtasks predictable? That is, can you enumerate them before seeing the input? This settles parallelization vs orchestrator-workers, and fixed pipeline vs dynamic decomposition.
3. Is the task worth the tokens? An agent resends its growing history every iteration, so cost grows super-linearly with steps, and subagents multiply it. No exact multiplier is tested, only the shape: more steps and more agents cost more.
4. What does an error cost, and would you notice? Cost and discoverability are separate axes. A wrong refund is expensive and visible. A silent coverage gap is cheap per incident and invisible. Low discoverability argues for structure and explicit verification even when errors look cheap.

Overriding rule: pick the simplest solution that works and add complexity only when it demonstrably helps. Often a single well-specified call with good examples beats any agentic design.

Task statement 1.6, concretely

Task 1.6 (design task decomposition strategies) is one choice: fixed sequential pipeline (prompt chaining) versus dynamic adaptive decomposition based on intermediate findings.

Legacy tests shows what "adaptive" means: map the structure, identify high-impact areas from that map, then build a prioritised plan that keeps adapting as dependencies surface. The plan can't be written up front because it depends on the codebase.

The same task, two ways

Triaging a support ticket. The workflow fixes the path: classify, fetch the records the class implies, draft. Each helper is one Messages API call, and the control flow between them is your code.

Python

async def triage_workflow(ticket: str) -> Resolution:
    # Step 1 — classify. Forced tool call guarantees structured output.
    category = await classify_ticket(ticket)  # "billing" | "shipping" | "account"

    # Programmatic gate: unroutable tickets never reach the drafting step.
    if category.confidence < 0.7:
        return escalate_to_human(ticket, reason="low_confidence_classification")

    # Step 2 — predetermined context fetch per category.
    if category.name == "billing":
        context = await get_customer(category.customer_id)
    elif category.name == "shipping":
        context = await lookup_order(category.order_id)
    else:
        context = await get_customer(category.customer_id)

    # Step 3 — draft against fetched context only.
    return await draft_resolution(ticket, context)

TypeScript

async function triageWorkflow(ticket: string): Promise<Resolution> {
  const category = await classifyTicket(ticket);

  if (category.confidence < 0.7) {
    return escalateToHuman(ticket, "low_confidence_classification");
  }

  const context =
    category.name === "shipping"
      ? await lookupOrder(category.orderId)
      : await getCustomer(category.customerId);

  return draftResolution(ticket, context);
}

The agent version gives a goal and constraints, then loops on stop_reason. It is the smallest version of what Claude Code runs. The message list is the conversation. SYSTEM plays CLAUDE.md's role. TRIAGE_TOOLS stands for tool definitions (name, description, input_schema), which is what a .mcp.json entry becomes at the API boundary. Chapter 2 shows their real shape.

Python

TRIAGE_TOOLS = [get_customer, lookup_order, process_refund, escalate_to_human]

SYSTEM = """Resolve the customer's ticket. Verify the customer's identity with
get_customer before any financial operation. Escalate when the request falls
outside documented policy or the customer asks for a human."""

async def triage_agent(ticket: str) -> Resolution:
    messages = [{"role": "user", "content": ticket}]

    while True:
        response = await client.messages.create(
            model="claude-sonnet-5",
            system=SYSTEM,
            tools=TRIAGE_TOOLS,
            messages=messages,
            max_tokens=2048,
        )
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason == "end_turn":
            return Resolution.from_content(response.content)

        results = [await execute_tool(block) for block in response.content
                   if block.type == "tool_use"]
        messages.append({"role": "user", "content": results})

TypeScript

async function triageAgent(ticket: string): Promise<Resolution> {
  const messages: MessageParam[] = [{ role: "user", content: ticket }];

  for (;;) {
    const response = await client.messages.create({
      model: "claude-sonnet-5",
      system: SYSTEM,
      tools: TRIAGE_TOOLS,
      messages,
      max_tokens: 2048,
    });
    messages.push({ role: "assistant", content: response.content });

    if (response.stop_reason === "end_turn") {
      return resolutionFrom(response.content);
    }

    const toolResults = await Promise.all(
      response.content
        .filter((b): b is ToolUseBlock => b.type === "tool_use")
        .map(executeTool),
    );
    messages.push({ role: "user", content: toolResults });
  }
}

The tools and model are unchanged. Only the decomposition moved, from if statements into the model. That handles a ticket that is both a billing dispute and a shipping complaint, which the workflow can't express without a new branch. The cost is non-deterministic ordering. So the identity-verification rule in the prompt is not sufficient: prompts have a non-zero failure rate, just as CLAUDE.md instructions are usually but not always followed. A rule with financial consequences needs a programmatic prerequisite gate or an interception hook (Chapter 6, the highest-yield idea on the exam).

In production the answer is usually a composite: an agent loop for the investigative middle, with hard gates around operations that must not go wrong. Patterns compose (routing feeds a chain; an orchestrator wraps evaluator-optimizer workers). Reach for a composite only after a single pattern has demonstrably failed.

What the exam tests

• Task 1.6: choose fixed pipeline or dynamic decomposition and justify it from the scenario's predictability. The two blueprint cases are per-file plus cross-file passes for a large PR (the stated cause is attention dilution), and map-then-prioritise for legacy-code tests (the plan can't exist until the structure is known).
• Task 1.1: model-driven decisions versus pre-configured tool sequences.
• Task 1.2: every subagent succeeds but aggregate coverage is wrong, so fix the decomposition.
• Distractors are competent engineering at the wrong layer: a bigger context window for an attention problem, a consensus rule that suppresses real findings, an extra agent where a gate would do.
• Scoring rule: proportionality. Name the described failure and pick the least complex fix. When two options both work, the simpler one is correct.

Exercises

1. Build both triage versions against four mock MCP tools. Run 20 tickets (10 single-concern, 10 with two concerns), logging tokens and tool calls per ticket, and find the ticket the workflow mishandles. Do this first: 40 lines of hand-written tool-use loop teaches more than reading.
2. For the 14-file review, implement one call over all files, then per-file passes plus an integration pass. Feed both a PR with one planted cross-file bug and three single-file bugs, and record what each catches.
3. Write an orchestrator's decomposition step for "research the impact of AI on creative industries" and inspect the subtasks before any worker runs. If they're all visual arts, rewrite the coordinator prompt to specify coverage criteria rather than procedure. Claude Code hides this step from you.
4. For one exam scenario, write its verifier in one sentence. If you can't, write what would have to be true for one to exist, and decide whether it should be an agent at all.

Chapter 2 — The agentic loop in code

Claude Code is an agentic loop — about forty lines of code — wrapped around a good tool set. The backtracking and recovery you see in it are not orchestration logic: Claude decides each next step by reasoning over the conversation history, and the loop only ferries messages and checks one field. This chapter builds that loop by hand (task statement 1.1) against the customer-support scenario: get_customer, lookup_order, process_refund, escalate_to_human.

The Messages API, from zero

One endpoint, POST /v1/messages. The request fields that matter:

content can be a plain string, but its real form is a list of typed blocks:

• text — prose.
• image — an image.
• tool_use — assistant messages only. Claude asking you to run something.
• tool_result — user messages only. You handing back what happened.

One message can mix several block types. Many plausible-looking loops break on that fact.

The response is an assistant message: role: "assistant", a content list of blocks, and a stop_reason saying why generation ended. Because it is already shaped like a message, you append it straight onto messages.

tools is a declaration, not an installation. Nothing runs on Anthropic's side: Claude emits a tool_use block and stops, you run the function, and you send the result back. Execution is local, and that is why Claude Code can show a permission prompt before a tool runs.

The API is stateless. There is no session and no server-side memory. Every request re-sends the whole messages array, and the model knows only what is in it. /compact rewrites that array.

One turn of the cycle

1. Send system, the full messages array, and tools.
2. Claude replies with an assistant message: text blocks, possibly tool_use blocks, and a stop_reason. The stop_reason, not the content, decides what happens next.
3. If stop_reason == "tool_use", append the assistant message in full, run every requested tool, append one user message holding all the tool_result blocks, and loop.
4. If stop_reason == "end_turn", stop. The text blocks are the answer.

The tool output you see indented in a Claude Code transcript is how it renders tool_result blocks. Memorise both shapes:

Pairing rules. The API enforces all three, and a violation returns a 400, not a degraded answer:

• Every tool_use must be answered by a tool_result in the immediately following message. Return two results for three calls and the request is rejected.
• Every tool_result needs a preceding tool_use with that id. This is what breaks when you truncate history to save context and the conversation ends up opening on a tool_result.
• All results go in one user message, and the tool_result blocks must come first in its content. Any extra text goes after them.

Human-in-the-loop sits between tool_use and execution. Claude has asked, but nothing has run yet. That is where Claude Code's permission prompt lives. Approving runs the tool. Denying sends back a tool_result saying so.

The loop, written out

Python

import json
import time
import anthropic

client = anthropic.Anthropic()

TOOLS = [
    {
        "name": "get_customer",
        "description": (
            "Retrieve a customer record by email or account ID. Returns the "
            "verified customer_id, tier, and account status. Call this before "
            "any order lookup or refund."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"identifier": {"type": "string"}},
            "required": ["identifier"],
        },
    },
    {
        "name": "lookup_order",
        "description": "Fetch an order by order_id, including line items, "
                       "ship date, and current refund eligibility.",
        "input_schema": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
    {
        "name": "process_refund",
        "description": "Issue a refund against a verified customer and order. "
                       "Amount is in USD cents.",
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string"},
                "order_id": {"type": "string"},
                "amount_cents": {"type": "integer"},
                "reason": {"type": "string"},
            },
            "required": ["customer_id", "order_id", "amount_cents", "reason"],
        },
    },
    {
        "name": "escalate_to_human",
        "description": "Hand the case to a human agent with a structured "
                       "summary. Use when policy blocks resolution or the "
                       "customer asks for a person.",
        "input_schema": {
            "type": "object",
            "properties": {
                "customer_id": {"type": "string"},
                "root_cause": {"type": "string"},
                "recommended_action": {"type": "string"},
            },
            "required": ["customer_id", "root_cause", "recommended_action"],
        },
    },
]

TOOL_IMPLS = {
    "get_customer": get_customer,
    "lookup_order": lookup_order,
    "process_refund": process_refund,
    "escalate_to_human": escalate_to_human,
}


def execute_tool(block):
    """Return a tool_result block for one tool_use block."""
    try:
        result = TOOL_IMPLS[block.name](**block.input)
        return {
            "type": "tool_result",
            "tool_use_id": block.id,
            "content": json.dumps(result),
        }
    except Exception as exc:
        return {
            "type": "tool_result",
            "tool_use_id": block.id,
            "content": f"{type(exc).__name__}: {exc}",
            "is_error": True,
        }


def run_agent(user_message, max_iterations=25, budget_seconds=120):
    messages = [{"role": "user", "content": user_message}]
    deadline = time.monotonic() + budget_seconds

    for iteration in range(max_iterations):
        if time.monotonic() > deadline:
            raise TimeoutError("wall-clock budget exhausted")

        response = client.messages.create(
            model="claude-sonnet-5",
            max_tokens=2048,
            system=SUPPORT_SYSTEM_PROMPT,
            tools=TOOLS,
            messages=messages,
        )
        log.info("iteration=%d stop_reason=%s", iteration, response.stop_reason)

        # The assistant message goes back verbatim, text blocks and all.
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason == "tool_use":
            tool_uses = [b for b in response.content if b.type == "tool_use"]
            results = [execute_tool(b) for b in tool_uses]
            messages.append({"role": "user", "content": results})
            continue

        if response.stop_reason == "end_turn":
            return response

        if response.stop_reason == "max_tokens":
            raise RuntimeError("response truncated; raise max_tokens")

        raise RuntimeError(f"unhandled stop_reason: {response.stop_reason}")

    raise RuntimeError("iteration cap hit — investigate, do not just raise it")

TypeScript

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

type Impl = (input: any) => Promise<unknown>;
const toolImpls: Record<string, Impl> = {
  get_customer: getCustomer,
  lookup_order: lookupOrder,
  process_refund: processRefund,
  escalate_to_human: escalateToHuman,
};

async function executeTool(
  block: Anthropic.ToolUseBlock,
): Promise<Anthropic.ToolResultBlockParam> {
  try {
    const result = await toolImpls[block.name](block.input);
    return {
      type: "tool_result",
      tool_use_id: block.id,
      content: JSON.stringify(result),
    };
  } catch (err) {
    return {
      type: "tool_result",
      tool_use_id: block.id,
      content: err instanceof Error ? err.message : String(err),
      is_error: true,
    };
  }
}

export async function runAgent(
  userMessage: string,
  { maxIterations = 25, budgetMs = 120_000 } = {},
): Promise<Anthropic.Message> {
  const messages: Anthropic.MessageParam[] = [
    { role: "user", content: userMessage },
  ];
  const deadline = Date.now() + budgetMs;

  for (let iteration = 0; iteration < maxIterations; iteration++) {
    if (Date.now() > deadline) throw new Error("wall-clock budget exhausted");

    const response = await client.messages.create({
      model: "claude-sonnet-5",
      max_tokens: 2048,
      system: SUPPORT_SYSTEM_PROMPT,
      tools: TOOLS,
      messages,
    });
    log.info({ iteration, stop_reason: response.stop_reason });

    messages.push({ role: "assistant", content: response.content });

    if (response.stop_reason === "tool_use") {
      const toolUses = response.content.filter(
        (b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
      );
      const results = await Promise.all(toolUses.map(executeTool));
      messages.push({ role: "user", content: results });
      continue;
    }

    if (response.stop_reason === "end_turn") return response;
    if (response.stop_reason === "max_tokens") {
      throw new Error("response truncated; raise max_tokens");
    }
    throw new Error(`unhandled stop_reason: ${response.stop_reason}`);
  }

  throw new Error("iteration cap hit — investigate, do not just raise it");
}

The only branch is on stop_reason. The loop never inspects Claude's text, keeps no whitelist of which tool may follow which, and has no workflow state machine. Swap in Read/Bash/Edit/Grep, put a permission gate before execute_tool, and you have Claude Code's shape.

stop_reason is the control signal

The exam focuses on "tool_use" (continue) and "end_turn" (terminate). Still handle unknown values in an explicit else that raises. A loop that falls through will one day treat a new stop reason as completion and return half an answer.

Strict tool use and thinking blocks

• "strict": true next to a tool's input_schema makes schema-valid arguments guaranteed rather than merely very likely. The only cost is that one field. Chapter 3 covers the JSON Schema subset it supports.
• Extended thinking: when a tool_use block sits alongside thinking or redacted_thinking blocks, send those thinking blocks back unmodified and in the same position on the next turn. Editing, reordering, or dropping one breaks its signature, and the request fails.

Why history is the agent's memory

Tool results are appended to history so the model can reason about the next action. That is how the agent accumulates knowledge, not a serialisation detail. In the support case:

1. get_customer("[email protected]") → history now holds customer_id: "cus_8812", tier: "enterprise".
2. Claude reads that and calls lookup_order → shipped 19 days ago, inside a 30-day window.
3. Claude calls process_refund with arguments it derived from history. You did not route them.

Ways to break it:

• Drop the assistant message, keep the results. 400: a tool_result with no tool_use.
• Keep the assistant message, drop the results. 400. If it somehow got through, Claude would re-request the same tool forever.
• Summarise the result into your own prose. You have quietly become the decision-maker, filtering what the model can reason over.

All three treat history as a log when it is state. The same applies to /compact: whatever it drops, the agent no longer knows. That is why a long session "forgets" a decision from an hour ago.

Model-driven, not a decision tree

The exam contrasts Claude choosing the next tool from context with a pre-configured decision tree or fixed sequence. A fixed sequence (always get_customer → lookup_order → branch on amount) assumes every case has the same shape. Real cases don't:

• Customer gives an order number → no email lookup needed.
• Three disputed charges → three order lookups.
• "Just let me talk to someone" → escalate_to_human, nothing else.

The loop above handles all of these with no routing code. You control the envelope: which tools exist, what their schemas allow, and which calls a hook may block (Chapter 6). Claude chooses within it. Plan mode is an example: it narrows the envelope to read-only tools, and Claude still decides everything inside it.

When ordering must be guaranteed, use a programmatic prerequisite gate (task 1.4, e.g. block process_refund until get_customer returns a verified id). Don't hard-code the call order in the loop.

Three anti-patterns the guide names

These three from task 1.1's skills list are the standard distractors.

Parsing natural language for termination

Python

# WRONG
text = "".join(b.text for b in response.content if b.type == "text")
if "let me know if" in text.lower() or "anything else" in text.lower():
    return response

TypeScript

// WRONG
const text = response.content
  .filter((b) => b.type === "text")
  .map((b) => (b as Anthropic.TextBlock).text)
  .join("");
if (/let me know if|anything else/i.test(text)) return response;

This fails both ways. "Let me know if you need anything else" can arrive alongside a pending tool_use, so the loop exits early and the refund never happens. "Your refund of $84.99 has been submitted." matches no phrase, so a finished conversation keeps looping.

An iteration cap as the stopping mechanism

Python

# WRONG
for _ in range(5):
    response = client.messages.create(...)
    messages.append({"role": "assistant", "content": response.content})
    tool_uses = [b for b in response.content if b.type == "tool_use"]
    if not tool_uses:
        break
    messages.append({"role": "user", "content": [execute_tool(b) for b in tool_uses]})
return response   # after 5 turns, whatever we have

TypeScript

// WRONG
for (let i = 0; i < 5; i++) {
  response = await client.messages.create({ /* ... */ });
  messages.push({ role: "assistant", content: response.content });
  const toolUses = response.content.filter((b) => b.type === "tool_use");
  if (toolUses.length === 0) break;
  messages.push({
    role: "user",
    content: await Promise.all(toolUses.map(executeTool)),
  });
}
return response; // after 5 turns, whatever we have

A 5-turn finish and a runaway both leave through the same return, and nothing tells them apart. A three-issue billing dispute that needs 8–9 turns is silently cut off. Keep a cap, but exceeding it must raise an error, not return.

Assistant text as a completion indicator

Python

# WRONG
if any(b.type == "text" for b in response.content):
    return response

TypeScript

// WRONG
if (response.content.some((b) => b.type === "text")) return response;

This sounds structural, but a tool_use response often carries text too. It is the sentence of narration you see above every tool call in Claude Code:

{
  "stop_reason": "tool_use",
  "content": [
    { "type": "text",
      "text": "I've confirmed Dana's account. Let me pull up order SO-44120." },
    { "type": "tool_use", "id": "toolu_01Xy…", "name": "lookup_order",
      "input": { "order_id": "SO-44120" } }
  ]
}

Rule: any completion check based on something other than stop_reason is the wrong answer.

tool_choice and how it bends the loop

Forcing applies per turn, not to the whole loop. Force get_customer on iteration one, then go back to auto. If you leave it forced, the loop can never reach "end_turn".

Parallel tool calls

One assistant turn can hold several tool_use blocks, for example two lookup_order calls for two disputed orders. It is the same as Claude Code firing three Reads at once. Run them concurrently (Promise.all, or asyncio.gather for slow I/O; the Python listing runs them sequentially for clarity) and return every result in one user message.

Claude parallelises only calls it believes are independent, and it can be wrong when one call's input should come from another's output. Safeguards:

• State prerequisites in descriptions, e.g. "process_refund needs a verified customer_id from get_customer". This nudges the model to sequence the calls itself.
• Gate the dependent tool programmatically when correctness can't rely on the model. A premature call then gets a structured error it can recover from.
• disable_parallel_tool_use alongside tool_choice caps a response at one tool call. The cost is latency.

tool_use_id binds each result to its call. Keeping results in order is good hygiene but not required for validity.

Guardrails that are not termination logic

Only stop_reason decides that the task is complete. Separate operational guardrails decide that the loop is no longer allowed to run:

• Max iterations bounds cost and catches cycles, such as retrying a tool that fails the same way every time. Exceeding it should raise, alert, and keep the transcript. Never return a partial answer as final.
• Wall-clock budget covers what an iteration count can't. One slow MCP call can take 90 s inside a single iteration.
• Structured logs record iteration, stop_reason, tool names, and durations, so an incident has a trace to read. Chapter 5 makes the same observability argument for routing subagent traffic through a coordinator.

The same loop, at a higher level

The Claude Agent SDK's query() runs this loop for you and yields typed messages. It is the engine Claude Code runs on.

MCP tool names are namespaced mcp__<server>__<tool>. <server> is the key in your config, not the server's internal name. Server key support plus tool get_customer gives mcp__support__get_customer. The namespacing exists because two servers can both expose search. Rename the config key and every allowlist entry for it breaks, which is the usual cause of a tool that "exists but Claude never calls".

Python

from claude_agent_sdk import query, ClaudeAgentOptions

options = ClaudeAgentOptions(
    system_prompt=SUPPORT_SYSTEM_PROMPT,
    mcp_servers={"support": {"command": "node", "args": ["./support-mcp.js"]}},
    allowed_tools=[
        "mcp__support__get_customer",
        "mcp__support__lookup_order",
        "mcp__support__process_refund",
        "mcp__support__escalate_to_human",
    ],
)

async for message in query(
    prompt="Dana ([email protected]) says order SO-44120 arrived damaged.",
    options=options,
):
    log.info("%s", message)

TypeScript

import { query } from "@anthropic-ai/claude-agent-sdk";

for await (const message of query({
  prompt: "Dana ([email protected]) says order SO-44120 arrived damaged.",
  options: {
    systemPrompt: SUPPORT_SYSTEM_PROMPT,
    mcpServers: {
      support: { command: "node", args: ["./support-mcp.js"] },
    },
    allowedTools: [
      "mcp__support__get_customer",
      "mcp__support__lookup_order",
      "mcp__support__process_refund",
      "mcp__support__escalate_to_human",
    ],
  },
})) {
  log.info(message);
}

The SDK builds tools from the MCP servers, branches on stop_reason, dispatches each tool_use, and batches the results, all as you did by hand. It also adds permission modes, hooks, and subagents. Two consequences:

• A custom handler that throws kills the loop, and Claude never sees the error. Returning an error result (isError) puts the failure into history, where the agent can react. It is is_error one level up.
• maxTurns/max_turns defaults to unlimited, so the iteration cap and wall-clock budget are still your job. Retrying failed HTTP requests belongs to the client, not the loop.

What the exam tests

• Task 1.1, mostly diagnostic: "this loop misbehaves; why?" It has three knowledge bullets: the stop_reason lifecycle, results appended to history, and model-driven decisions versus decision trees.
• Completion determined by scanning text, hitting an iteration cap, or the presence of prose is wrong. Continue on "tool_use", terminate on "end_turn".
• Expect an item that turns on a tool_use response that also contains narration text.
• Task 2.3 tool_choice: "any" guarantees an action instead of chat. A forced named tool pins the first step, then you return to auto.
• Task 1.4: enforcing order with a gate is a different design from hard-coding the call order in the loop.
• Streaming, token counting, and rate limits are out of scope.

Exercises

1. Send one request with no tools and print the raw response: find role, content, the block types, and stop_reason. Add one tool and a prompt that needs it, then diff the two responses.
2. Implement run_agent from scratch, in your less-used language, against stub tools. Don't look at the listing until it runs, then diff and account for every difference.
3. Break it four ways and record what happens: omit the assistant message; return one result for two tool_use blocks; change one character of a tool_use_id; set max_tokens to 64. Which give a 400, which give a silent behaviour change, which loop forever?
4. Add a stdin approval prompt before process_refund. On refusal, return a tool_result saying a human denied it, and watch what Claude does. That is Claude Code's permission prompt in about eight lines.
5. Send a message disputing three orders, and log the tool_use blocks per turn. Then set disable_parallel_tool_use and compare iterations and wall-clock time.
6. Write the smallest conversation that makes the iteration-cap anti-pattern silently return an incomplete resolution. Keep the transcript.

Chapter 3 — Designing tools Claude can actually choose between

Claude Code picks Grep, Glob, and Edit reliably because its built-in tools are described with unusual care, not because selection is intrinsic to the model. Write two tools you think are obviously distinct and you may see coin-flip selection between them. The tempting explanations (the model is confused, you need a routing layer, add few-shot examples) are wrong. The exam's claim is causal: tool descriptions are the primary mechanism for tool selection, and minimal descriptions produce unreliable selection among similar tools.

What a tool definition actually is

Three fields: name, description, input_schema (a JSON Schema object). The model sees nothing else. Anything you know about the tool (which database it hits, what ID it needs, overlapping fields) is invisible unless it's in the description.

The description is prompt text. It is injected into context at selection time and read like any other instruction. It is not documentation for someone who can read the source. Treat it as an API contract and you'll write two lines where you needed twenty.

Python

extract_data_points = {
    "name": "extract_data_points",
    "description": (
        "Extract structured numeric and categorical data points from a research "
        "document that has already been loaded. Returns a list of records, each with "
        "a field name, a value, a unit where applicable, and the page and paragraph "
        "the value was read from.\n\n"
        "Input: document_id from load_document, plus the field names to extract.\n"
        "Use this when you need specific values out of a document — revenue figures, "
        "sample sizes, publication dates, dosage tables.\n"
        "Do NOT use this to understand what a document argues; use summarize_content. "
        "Do NOT use this to check whether an external claim is supported; use "
        "verify_claim_against_source.\n"
        "Edge case: if a requested field is absent, the record is returned with a null "
        "value and a reason, rather than being omitted."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "document_id": {
                "type": "string",
                "description": "Identifier returned by load_document.",
            },
            "field_names": {
                "type": "array",
                "items": {"type": "string"},
                "description": "Field labels to extract, e.g. ['sample_size', 'p_value'].",
            },
        },
        "required": ["document_id", "field_names"],
    },
}

TypeScript

const extractDataPoints: Anthropic.Tool = {
  name: "extract_data_points",
  description: [
    "Extract structured numeric and categorical data points from a research document",
    "that has already been loaded. Returns a list of records, each with a field name, a",
    "value, a unit where applicable, and the page and paragraph the value was read from.",
    "",
    "Input: document_id from load_document, plus the field names to extract.",
    "Use this when you need specific values out of a document — revenue figures, sample",
    "sizes, publication dates, dosage tables.",
    "Do NOT use this to understand what a document argues; use summarize_content.",
    "Do NOT use this to check whether an external claim is supported; use",
    "verify_claim_against_source.",
    "Edge case: if a requested field is absent, the record is returned with a null value",
    "and a reason, rather than being omitted.",
  ].join("\n"),
  input_schema: {
    type: "object",
    properties: {
      document_id: { type: "string", description: "Identifier returned by load_document." },
      field_names: {
        type: "array",
        items: { type: "string" },
        description: "Field labels to extract, e.g. ['sample_size', 'p_value'].",
      },
    },
    required: ["document_id", "field_names"],
  },
};

• The field is input_schema, and tools are a flat list. OpenAI's shape (type: "function" wrapper, nested function.parameters) is not accepted, and parameters appears in distractors.
• Per-property description strings are prompt text too. Put format constraints there that the schema can't express, e.g. "#-prefixed, eight digits" or "must point at a document, not an arbitrary page".

What a good description contains

Audit any description against five elements:

1. What it does and returns: the shape of the result, not the implementation.
2. Accepted input formats, including ones that look valid but aren't.
3. Example queries in the user's language.
4. Edge cases, especially "failed" versus "succeeded and found nothing".
5. Boundaries against sibling tools, by name.

With near-neighbour tools, #5 outweighs the other four, because it's the only information the model can't infer from the tool in isolation.

Bad:

name: lookup_order
description: Retrieves order details.

Good:

name: lookup_order
description: >
  Look up a single order by its order number and return status, line items,
  fulfilment and delivery dates, carrier tracking, and the refund state of each
  line. Accepts an order number with or without the leading '#' (e.g. "#48812",
  "48812"). Does NOT accept email addresses, names, or account IDs.

  Use for queries like "where is my order 48812", "was #48812 delivered",
  "which items shipped in my last order".

  Use get_customer instead when the request identifies a person rather than an
  order ("look up my account", "what's on file for [email protected]") or when
  you need the customer's order history rather than one order.

  If the order number is well-formed but not found, returns an empty result with
  found: false — this is a successful call, not an error, and usually means a
  typo or an order belonging to a different account.

What each part of the good version prevents:

• Return description: the model no longer calls get_customer just to find out whether tracking data exists.
• "Does NOT accept" clause: no email passed in, no turn burned on a validation error.
• Example queries: they do the job of system-prompt few-shot examples, for fewer tokens, and they're maintained with the tool.
• Boundary sentence naming get_customer: turns an overlap into a divide.
• Empty-result note: stops the agent treating "not found" as a failure and retrying or escalating, when it should ask the customer to re-check the number.

150–250 tokens per description is normal, which is what Claude Code's own tools run to, and cheaper than a misrouted call plus the recovery turn.

strict: true guarantees the schema, not the selection

Descriptions fix which tool is called. A separate failure survives a perfect description: the right tool with a malformed argument, such as a string for a number or a missing required field. "strict": true beside input_schema constrains generation so the input matches the schema exactly, much as strict JSON output does in Chapter 13. Add it wherever a malformed input is costly (e.g. process_refund.amount_cents). It doesn't help selection, so a strict tool still needs a good description.

input_examples lets you attach example input objects as structured data. It is an alternative to writing example queries in prose, but prose still carries when to use the tool, which bare inputs don't.

Diagnosing misrouting

Misrouting has a signature: consistently the wrong member of a specific pair, on a consistent class of query. For example, get_customer gets called on order queries, or analyze_content and analyze_document (near-identical descriptions) are picked interchangeably regardless of input type.

Diagnose in order of cost:

1. Compare the two descriptions. Could a new engineer, given only those strings, pick correctly for every failing query? If not, the descriptions are the cause.
2. Check the schemas. If both take a bare string query, the schema offers no signal either.
3. Only if the descriptions already discriminate, read the system prompt for keyword-sensitive instructions. "Always begin by understanding the customer" pulls calls toward get_customer whatever its description says. It overrides good descriptions and is invisible unless you look.

Non-fixes:

• A deterministic routing layer that pre-selects tools from the input. It discards the model's language understanding and fails on every phrasing you didn't anticipate.
• 5–8 few-shot examples in the system prompt. They add tokens to every request and leave the cause untouched. The model memorises surface patterns but still can't tell the tools apart.

The three repair moves

1. Rename and rewrite to remove overlap. analyze_content becomes extract_web_results: it takes search snippets and URLs, returns passages with source attribution, and explicitly routes loaded documents to the document tools. The rename matters as much as the rewrite, because the name is what the model sees first.
2. Split a generic tool into purpose-specific tools. If one tool does unrelated jobs, no description can fix it, because the ambiguity is in the tool. An analyze_document that summarises, extracts, or fact-checks depending on phrasing becomes extract_data_points, summarize_content, and verify_claim_against_source, each with defined I/O contracts and each naming the other two in its boundaries.
3. Review the system prompt. It's the cheapest fix and the one most often skipped. Remove keywords that collide with tool names, and move real sequencing out of prose and into tool_choice or a programmatic prerequisite.

Naming: one convention across the inventory, snake_case verb plus resource, namespaced by service when there are several backends. billing_get_invoice and fulfilment_get_shipment are easy to tell apart; two get_records are not.

Distributing tools across agents

Selection reliability is a property of the tool set, not the individual tool. The exam's numbers: an agent with 18 tools selects less reliably than one with 4–5.

Agents also misuse tools outside their specialisation. Give a synthesis agent web search and it will search, because the tool is there and a gap seems to fit it. You get mediocre research instead of good synthesis, and blurred responsibility.

Default: scoped tool access, where each agent holds only what its role needs. In the research system:

• Search agent: search and fetch tools.
• Document agent: load_document plus the three extraction tools.
• Synthesis agent: none of those.
• Report agent: formatting and citation tools.
• Coordinator: delegation tools only.

The verify_fact case. Strict scoping means synthesis must go through the coordinator to reach search, which adds 2–3 round trips and about 40% latency. Measurement shows 85% of verifications are simple (dates, names, statistics) and 15% need real investigation. Fix: give synthesis a limited, scoped verify_fact tool for the 85%, and keep routing the 15% through the coordinator. Wrong answers:

• The full search toolkit for synthesis: over-provisioned, and it dissolves the separation of concerns.
• Batching verifications to the end: later steps depend on earlier facts, so it creates blocking dependencies.

Same instinct for individual tools: replace generic with constrained. fetch_url (fetches anything) becomes load_document (validates that the URL is a document and rejects the rest).

The tool search tool, as an alternative to manual scoping

For an agent that legitimately needs many tools of one kind, the tool search tool (tool_search_tool_bm25_20251119 for keyword search, tool_search_tool_regex_20251119 for patterns) defers most definitions and lets Claude search for the relevant one on demand. It solves the context and selection-load problem, but not separation of concerns: a synthesis agent that shouldn't research still shouldn't have search. Scoping answers "who should be able to do this". Tool search answers "how do I keep a large legitimate inventory usable".

tool_choice as a sequencing instrument

When enrichment depends on extract_metadata's output, don't write "always call extract_metadata first" in the prompt. Force it on turn one, then return to "auto".

Python

first = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=2048,
    tools=[extract_metadata, enrich_company, enrich_filing],
    tool_choice={"type": "tool", "name": "extract_metadata"},
    messages=[{"role": "user", "content": "Process the Q3 filing at doc_8814."}],
)
# ... append the tool_use block and its tool_result, then continue:
followup = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=2048,
    tools=[extract_metadata, enrich_company, enrich_filing],
    tool_choice={"type": "auto"},
    messages=conversation,
)

TypeScript

const first = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 2048,
  tools: [extractMetadata, enrichCompany, enrichFiling],
  tool_choice: { type: "tool", name: "extract_metadata" },
  messages: [{ role: "user", content: "Process the Q3 filing at doc_8814." }],
});

const followup = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 2048,
  tools: [extractMetadata, enrichCompany, enrichFiling],
  tool_choice: { type: "auto" },
  messages: conversation,
});

tool_choice applies to one request. It can't enforce a prerequisite across an open-ended multi-turn loop. For business-critical sequencing (verify the customer before refunding), block the call in your own code. Strength ranking: prompt < tool_choice < programmatic gate. Sample question 1 turns on this. Chapter 6 covers gates.

The built-in Claude Code tools

You know these; here are the exam's distinctions.

• Edit fallback (the guide's answer): when Edit can't find unique text, use Read + Write: load the whole file, reconstruct it, write it back. It's slower and costs more tokens, but it's deterministic. Answer this on the exam. Current Claude Code documents a different first move: widen old_string with context, or set replace_all: true.
• Incremental exploration: Grep for entry points, then Read to follow imports. Don't read everything up front. Forty files for one code path wastes context and pushes the relevant part into the weak-attention middle.
• Tracing through wrapper modules: re-exports rename things, so one Grep for the original name misses callers. First identify the exported names (Grep the barrel file or export statements), then search for each.

What the exam tests

• Task 2.1: descriptions are the primary selection mechanism. Given a misrouting symptom (get_customer/lookup_order, analyze_content/analyze_document), the best first step is expanding the descriptions: formats, examples, edge cases, sibling boundaries. Not few-shot examples, not a routing layer, not consolidating tools.
• Match each repair to its example: rename + rewrite for overlap; split with defined I/O contracts for a generic tool; system prompt review when good descriptions are overridden by keywords.
• Task 2.3: 18 tools vs 4–5; out-of-specialisation misuse; scoped access by default; a limited verify_fact when a measured majority of cases are simple, with the rest via the coordinator; fetch_url → load_document. tool_choice: forced to run extract_metadata first, "any" to guarantee a call rather than text.
• Task 2.5: Grep = contents, Glob = paths, Read/Write = whole files, Edit = unique anchor with Read + Write fallback, Bash only when no dedicated tool exists. Two strategies: incremental exploration from Grep, and exported-names-first tracing through wrappers.
• MCP deployment, auth, and transports are out of scope.

Exercises

1. Give get_customer and lookup_order one-line descriptions. Run 20 queries (10 each) with tool_choice: "auto" and record accuracy. Rewrite both with all five elements and re-run. For queries still failing, decide whether the description or the schema is at fault.
2. Write a generic analyze_document with a mode parameter, then split it into extract_data_points, summarize_content, and verify_claim_against_source with distinct schemas, in Python and TypeScript. Check that each description names the other two.
3. Build a two-turn harness: force extract_metadata, append the tool_use/tool_result, then follow up with "auto". Change the follow-up to "any" and see what happens when nothing useful is left to call. That shows where "any" is a trap.
4. Audit a Claude Code built-in tool's description against the five elements, then write one of your own to that standard.

Chapter 4 — Errors, recovery, and telling the agent the truth

When a Bash command fails in Claude Code, Claude reads stderr, adjusts, and tries something else. No handler knows what command not found: uv means. Recovery works because the failure arrives as content the model can read, not as an exception that ends the turn.

In an agentic system, an error is a message to the model, and the model is its primary consumer, not the operator or the log. Raise an exception out of the loop and you remove the one participant that could narrow the query, try another source, or continue with what it has. Given "the pricing service did not respond within 30 seconds; this is usually transient", the model often plans recovery better than hand-written branches, because it holds the whole task in context. Backend reflexes like "fail fast" and "let it crash" are right for failures people read. They are harmful here, where failing fast kills the reasoning process that was about to solve the problem.

Two task statements share this material:

• 2.2: structured error responses from MCP tools. The recipient is the calling agent.
• 5.3: error propagation in multi-agent systems. The recipient is a coordinator handling a subagent that couldn't finish.

Both use the same taxonomy, the same access-failure versus empty-result distinction, and the same rule that partial results travel with the error.

Scope. Rate limits, quotas, and pricing are out of scope. Retry strategy is in scope as design judgement (attempt budgets, idempotency).

Layers not to conflate:

• isError/is_error and the categories below cover your tool failing while the API is healthy.
• overloaded_error (HTTP 529) and rate_limit_error (HTTP 429) mean the API call itself failed. They never reach the model as a tool_result. The SDK retries them beneath your loop with a small default budget, exponential backoff, and retry-after. Don't build a competing retry policy for this layer.
• stop_reason: "model_context_window_exceeded" (Claude 4.5+) means the conversation ran out of room. Fix it by trimming or compacting history, not by retrying.

The two spellings of the same flag

A tool result is content plus one boolean saying "this is a failure":

• MCP: isError on the tool result.
• Messages API: is_error on a tool_result block.

isError is not a thrown exception. The tool returns normally with the flag set and a description. The model reads it as the outcome of its call, and the turn continues.

Routing ordinary tool failures out as protocol errors, or letting an unhandled exception kill the turn, is the most common way to build an agent that can't recover from anything.

Four categories, one retryable

Only transient errors are retryable. Retrying the others fails the same way and wastes tokens and latency. "Not retryable" doesn't mean hopeless: after a validation error, the agent should call again with corrected input. That is a different act from a retry.

Guide inconsistency: the knowledge statements name four categories, but task 2.2's skills bullet lists errorCategory as only transient/validation/permission. Business-rule violations are instead described as the case needing retriable: false plus a customer-friendly message. Default to four categories, recognise the three-value list, and remember that retriable: false is attached to business violations by name.

Why "Operation failed" is the expensive answer

Generic, uniform error text is a named anti-pattern. If every failure reads Operation failed, the model can't tell a network blip from a policy refusal. Its only strategy is to retry, so it retries a refund that is out of policy on attempt one and still out of policy on attempt four. Structured metadata prevents wasted retries.

Payload shape:

• errorCategory
• isRetryable boolean
• A description written for a reader who must decide what to do next
• For business violations: retriable: false plus a customer-friendly explanation the agent can relay

Python

import json
from dataclasses import dataclass, asdict

@dataclass
class ToolFailure:
    errorCategory: str          # transient | validation | business | permission
    isRetryable: bool
    description: str            # for the model: what happened, what to try
    customerMessage: str | None = None   # for business errors: safe to relay verbatim

def process_refund(order_id: str, amount_cents: int) -> dict:
    order = orders.get(order_id)
    if order is None:
        return error_result(ToolFailure(
            errorCategory="validation",
            isRetryable=False,
            description=(
                f"No order matches id '{order_id}'. Order ids look like 'ORD-8842911'. "
                "Confirm the id with the customer before calling again."
            ),
        ))
    if order.days_since_delivery > 30:
        return error_result(ToolFailure(
            errorCategory="business",
            isRetryable=False,
            description="Refund window closed; policy REFUND-30D. Do not retry.",
            customerMessage=(
                "This order was delivered 47 days ago and our refund window is 30 days, "
                "so I can't process a refund automatically — but I can raise it with a specialist."
            ),
        ))
    ...

def error_result(failure: ToolFailure) -> dict:
    return {
        "isError": True,
        "content": [{"type": "text", "text": json.dumps(asdict(failure))}],
    }

TypeScript

type ErrorCategory = "transient" | "validation" | "business" | "permission";

interface ToolFailure {
  errorCategory: ErrorCategory;
  isRetryable: boolean;
  description: string;
  customerMessage?: string;
}

const errorResult = (failure: ToolFailure) => ({
  isError: true,
  content: [{ type: "text" as const, text: JSON.stringify(failure) }],
});

export async function processRefund(orderId: string, amountCents: number) {
  const order = await orders.get(orderId);
  if (!order) {
    return errorResult({
      errorCategory: "validation",
      isRetryable: false,
      description:
        `No order matches id '${orderId}'. Order ids look like 'ORD-8842911'. ` +
        "Confirm the id with the customer before calling again.",
    });
  }
  if (order.daysSinceDelivery > 30) {
    return errorResult({
      errorCategory: "business",
      isRetryable: false,
      description: "Refund window closed; policy REFUND-30D. Do not retry.",
      customerMessage:
        "This order was delivered 47 days ago and our refund window is 30 days, so I can't " +
        "process a refund automatically — but I can raise it with a specialist.",
    });
  }
  // ...
}

The description is a prompt fragment, not a log line. It carries the same weight as CLAUDE.md text, so write it as an instruction.

Access failure is not an empty result

This appears in both Domain 2 and Domain 5.

• Access failure: couldn't reach the data. It failed and needs a retry decision.
• Valid empty result: reached the data, the query ran, nothing matched. That is a success and the answer is "none".

Collapse them and you lose information both ways. A timeout marked as an empty success produces a confident "no records" with no basis. An empty match marked as an error burns the retry budget re-asking an answered question. Make them structurally different, not just differently worded.

Python

def search_incident_reports(query: str, since: str) -> dict:
    try:
        rows = incident_index.search(query, since=since, timeout=20)
    except SearchTimeout:
        return error_result(ToolFailure(
            errorCategory="transient",
            isRetryable=True,
            description=f"Incident index timed out after 20s for query {query!r}.",
        ))
    payload = {
        "isError": False,          # explicit, not the absence of the field
        "resultCount": len(rows),  # 0 is a real, trustworthy answer
        "results": rows,
        "queryExecuted": query,
    }
    return {"isError": False, "content": [{"type": "text", "text": json.dumps(payload)}]}

TypeScript

export async function searchIncidentReports(query: string, since: string) {
  try {
    const rows = await incidentIndex.search(query, { since, timeoutMs: 20_000 });
    const payload = { isError: false, resultCount: rows.length, results: rows, queryExecuted: query };
    return { isError: false, content: [{ type: "text" as const, text: JSON.stringify(payload) }] };
  } catch (err) {
    if (err instanceof SearchTimeout) {
      return errorResult({
        errorCategory: "transient",
        isRetryable: true,
        description: `Incident index timed out after 20s for query "${query}".`,
      });
    }
    throw err;
  }
}

resultCount: 0 with no error flag means someone looked.

Local recovery, then honest escalation

The subagent is the right place to absorb transient failure. It is closest to the tool, knows expected latency, and retrying there costs the coordinator nothing. Handle transients locally and propagate only what you couldn't resolve, always with partial results and what was attempted.

Task 5.3's structured error context has four parts, and each drives a coordinator decision:

Python

async def market_research_subagent(topic: str) -> dict:
    attempts, partial = [], []
    for query in expand_topic(topic):           # e.g. three narrower sub-queries
        attempts.append(query)
        try:
            partial.extend(await web_search(query, timeout=25, retries=2))
        except SearchTimeout:
            continue                             # local recovery: keep going
    if not partial:
        return {
            "status": "failed",
            "failureType": "transient",          # timeout, not "no such topic"
            "attempted": attempts,
            "partialResults": [],
            "alternatives": [
                "Retry with a single narrowed query such as 'EU battery recycling mandates 2026'",
                "Route to the internal-archive subagent, which has cached analyst notes",
            ],
        }
    if len(partial) < expected_minimum(topic):
        return {
            "status": "partial",
            "failureType": "transient",
            "attempted": attempts,
            "partialResults": partial,
            "coverageGaps": ["regulatory outlook — all three source queries timed out"],
            "alternatives": ["Re-run only the regulatory sub-query"],
        }
    return {"status": "complete", "results": partial}

TypeScript

export async function marketResearchSubagent(topic: string) {
  const attempted: string[] = [];
  const partial: Finding[] = [];

  for (const query of expandTopic(topic)) {
    attempted.push(query);
    try {
      partial.push(...(await webSearch(query, { timeoutMs: 25_000, retries: 2 })));
    } catch (err) {
      if (!(err instanceof SearchTimeout)) throw err;   // local recovery only for transient
    }
  }

  if (partial.length === 0) {
    return {
      status: "failed",
      failureType: "transient",
      attempted,
      partialResults: [],
      alternatives: [
        "Retry with a single narrowed query such as 'EU battery recycling mandates 2026'",
        "Route to the internal-archive subagent, which has cached analyst notes",
      ],
    };
  }
  if (partial.length < expectedMinimum(topic)) {
    return {
      status: "partial",
      failureType: "transient",
      attempted,
      partialResults: partial,
      coverageGaps: ["regulatory outlook — all three source queries timed out"],
      alternatives: ["Re-run only the regulatory sub-query"],
    };
  }
  return { status: "complete", results: partial };
}

Walking sample question 8

A web-search subagent times out mid-research. Correct: the four-part structured payload above. The distractors:

• Retry with backoff, then return generic "search unavailable". The retry is fine and belongs there. The generic status is the defect: it throws away the queries tried and the data collected.
• Catch the timeout, return an empty result marked successful. This suppresses the error, so no recovery is possible, and synthesis treats absence of findings as evidence of absence.
• Terminate the workflow from a top-level handler. Honest but disproportionate: it kills the whole run when a narrowed retry or another source would likely have worked.

The last two are task 5.3's named anti-patterns: suppressing errors as empty success, and terminating an entire workflow on one subtask failure.

Coverage annotations: partial is fine, undisclosed is not

Synthesis output must mark which findings are well supported and which topic areas have gaps because sources were unavailable. "Competitor pricing: three independent sources; regulatory outlook: no sources retrieved, all queries timed out" is useful. The same report with the regulatory section silently dropped is a hazard, because the reader can't tell "investigated, found nothing" from "never reached".

Partial results are acceptable. Undisclosed partial results are not.

Idempotency, and the refund you paid twice

process_refund posts the refund, then the response times out. The tool correctly reports a retryable transient error, the agent retries, and the customer is refunded twice. The category was right, but the operation wasn't safe to retry.

Retryability is a property of the operation as well as the error. Make mutating tools idempotent: the client supplies a key derived from the request's intent, not the attempt, and the server returns the original outcome for a repeated key.

Python

def refund_result(refund_id: str, replayed: bool) -> dict:
    payload = {"isError": False, "refundId": refund_id, "replayed": replayed}
    return {"isError": False, "content": [{"type": "text", "text": json.dumps(payload)}]}

def process_refund(order_id: str, amount_cents: int, idempotency_key: str) -> dict:
    prior = refund_ledger.get(idempotency_key)
    if prior is not None:
        return refund_result(prior.id, replayed=True)
    receipt = payments.refund(order_id, amount_cents, key=idempotency_key)
    refund_ledger.put(idempotency_key, receipt)
    return refund_result(receipt.id, replayed=False)

TypeScript

const refundResult = (refundId: string, replayed: boolean) => {
  const payload = { isError: false, refundId, replayed };
  return { isError: false, content: [{ type: "text" as const, text: JSON.stringify(payload) }] };
};

export async function processRefund(
  orderId: string, amountCents: number, idempotencyKey: string,
) {
  const prior = await refundLedger.get(idempotencyKey);
  if (prior) return refundResult(prior.id, true);

  const receipt = await payments.refund(orderId, amountCents, { key: idempotencyKey });
  await refundLedger.put(idempotencyKey, receipt);
  return refundResult(receipt.id, false);
}

Make the key a required schema parameter and describe it as identifying the refund attempt, so the model reuses it on retry. The caller is a model choosing arguments from prose, so the description is all that stands between you and a duplicate payment. A key regenerated per call protects nothing.

Retry strategy as a design decision

• Exponential backoff with jitter for transients. Backoff gives the dependency room to recover. Jitter stops parallel subagents retrying in lockstep (thundering herd).
• Bound attempts: 2–3 when a human is waiting, more for batch.
• Retries the model sees cost tokens. Each failed attempt stays in context for the rest of the session, crowding out useful material. Absorb retries inside the subagent and surface one summarised outcome.
• Set a per-turn budget alongside the per-call one. When it's exhausted, escalate with the four-part payload.

What the exam tests

• The four categories, and that only transient is retryable.
• isError (MCP) and is_error (Messages API) by name. Their purpose is to make failure visible to the model.
• Access failure versus valid empty result (Domains 2 and 5). A timeout reported as empty success, or an empty match reported as an error, is wrong on sight.
• The four-part payload for multi-agent failures. A generic "search unavailable" is always the trap, even with sound retry logic around it.
• The two named anti-patterns: suppressing as empty success, and terminating the whole workflow on one failure.
• Local recovery first. Escalation carries partial results rather than replacing them.
• Coverage annotations (task 5.3): partial is fine, undisclosed is not.

Exercises

1. Write an MCP tool wrapping a service you have credentials for. Give every exit path an errorCategory, an isRetryable, and a model-oriented description. In Claude Code, trigger each failure: it should retry the timeout and stop on the policy refusal. Then change every description to Operation failed and compare.
2. Build a search tool with structurally distinct empty and timeout results. With the source unreachable, confirm the agent retries. With the source reachable and the query unmatchable, confirm it reports "no matches" with zero retries.
3. Implement the market-research subagent and force one sub-query to time out permanently. Check that the coordinator gets partial results plus coverageGaps, and that the synthesis names the gap. Then make the subagent return {"status": "complete", "results": []} and record what the synthesis claims.
4. Add a required idempotency key to a mutating tool and simulate a timeout after commit. Confirm the retry replays the receipt. Then reword the key's description to sound per-call and watch the model mint a fresh one.

Chapter 5 — Orchestration

When Claude Code fans a broad request out to subagents, it is running a small distributed system. A coordinator decides to delegate, isolated workers each see only what they are told, and a synthesis step reconciles results that never talked to each other. This chapter covers building that system yourself with the Agent SDK (task statements 1.2 and 1.3), so you can diagnose it and use it where no Claude Code exists.

Running example: the exam's Multi-Agent Research System. A coordinator delegates to search and document-analysis subagents, and a synthesis subagent writes the report. Customer support is the secondary example.

Why orchestration is a separate problem from the loop

Chapter 2's loop is one agent with one growing messages array. Orchestration applies when one context is the wrong shape for the whole problem:

• Tool load: 18 tools select worse than 4–5, and out-of-specialisation tools get misused (Chapter 3). One agent holding search, document loading, extraction, and formatting will sometimes half-read a document instead of delegating.
• Context load: a single conversation accumulates every document, result, and draft, so by synthesis time the signal is buried. Separate bounded contexts are a context-management technique as well as a division of labour (Chapter 14).

Hub-and-spoke: the coordinator owns everything

Default architecture: one coordinator at the centre. Subagents talk only to the coordinator, never to each other. The coordinator decomposes, delegates, and aggregates. A finished search subagent returns to the coordinator, which decides what synthesis sees.

Giving up direct subagent-to-subagent communication buys three things the exam names:

1. Observability: everything passes through one place, so one log tells the whole story.
2. Consistent error handling: one place decides whether to retry, substitute, or surface partial results.
3. Controlled information flow: the coordinator can filter, summarise, or redact before passing data on, e.g. keep sensitive document content away from the report agent.

In support: a case needing order history and a fraud check before process_refund routes both through a coordinator, not a fraud agent reading the order agent's output directly. One place sees the case and decides what an inconclusive fraud check means.

Subagents have no memory of the coordinator, or of each other

A subagent's context is isolated. It does not inherit the coordinator's history, and separate invocations of "the same" subagent share nothing. The Task tool starts a subagent with only the prompt you write. Anything the coordinator or an earlier subagent learned exists for it only if it is in that prompt.

The exam's example: a synthesis subagent prompted with just "Synthesize the research findings into a report." It has no findings, because they live in the coordinator's history, so the output is generic and uncited. The fix is to put the findings in the prompt, not to use a bigger model or give synthesis a WebSearch tool. That would move research into the wrong agent and break separation of concerns.

Python

# WRONG — the coordinator assumes shared memory that doesn't exist
synthesis_prompt = "Synthesize the research findings into a report."

# RIGHT — the coordinator passes what it actually collected
synthesis_prompt = f"""
Write a research report answering: {research_question}

You have no access to the search or document tools. Work only from the
findings below. Cite every claim using the bracketed source ids.

## Web search findings
{format_search_findings(search_results)}

## Document analysis findings
{format_document_findings(document_results)}

Quality criteria: every factual claim must carry a citation id. Flag any
subtopic in the original question that the findings below do not cover.
"""

TypeScript

// WRONG — the coordinator assumes shared memory that doesn't exist
const synthesisPrompt = "Synthesize the research findings into a report.";

// RIGHT — the coordinator passes what it actually collected
const synthesisPrompt = `
Write a research report answering: ${researchQuestion}

You have no access to the search or document tools. Work only from the
findings below. Cite every claim using the bracketed source ids.

## Web search findings
${formatSearchFindings(searchResults)}

## Document analysis findings
${formatDocumentFindings(documentResults)}

Quality criteria: every factual claim must carry a citation id. Flag any
subtopic in the original question that the findings below do not cover.
`;

The corrected prompt ends with a goal and quality bar (cite every claim, flag coverage gaps), not a numbered procedure. That is the second skill tested: coordinator prompts specify what a good result looks like, leaving the subagent free to structure it to fit the material.

Structured data over prose blobs, so attribution survives the handoff

If a subagent returns prose that mixes claims with URLs, synthesis has to re-parse attribution from running text. Sometimes it drops or garbles it, and you get confident, untraceable claims. Separate content from metadata at return time: every finding carries its source URL, or document name and page, as a field alongside the content.

Python

search_findings = [
    {
        "claim": "The library dropped support for callback-style APIs in v5.",
        "source_url": "https://example.com/library/changelog/v5",
        "retrieved_at": "2026-08-30",
    },
    {
        "claim": "Migration guides recommend the async adapter for v4 codebases.",
        "source_url": "https://example.com/library/migration-guide",
        "retrieved_at": "2026-08-30",
    },
]

document_findings = [
    {
        "claim": "Internal usage of the callback API appears in 14 call sites.",
        "document_name": "internal-audit-q3.pdf",
        "page": 6,
    },
]

TypeScript

const searchFindings = [
  {
    claim: "The library dropped support for callback-style APIs in v5.",
    sourceUrl: "https://example.com/library/changelog/v5",
    retrievedAt: "2026-08-30",
  },
  {
    claim: "Migration guides recommend the async adapter for v4 codebases.",
    sourceUrl: "https://example.com/library/migration-guide",
    retrievedAt: "2026-08-30",
  },
];

const documentFindings = [
  {
    claim: "Internal usage of the callback API appears in 14 call sites.",
    documentName: "internal-audit-q3.pdf",
    page: 6,
  },
];

format_search_findings / format_document_findings render these with metadata still attached to each claim. It is the same content/metadata discipline as tool outputs in Chapter 3, applied between agents.

The Task tool and the AgentDefinition

The Task tool looks like any tool (name, description, schema), but calling it starts a new agentic loop with its own messages array and returns that loop's final result as the tool result. Claude Code spawns its subagents this way.

Two configuration facts:

1. allowedTools must include "Task" for the coordinator to delegate. A coordinator that never spawns subagents despite its prompt is structurally unable to, not reasoning badly. Check this first. (Claude Code renamed Task to Agent in v2.1.63 and keeps Task as an alias. The guide and exam say Task, so answer Task.)
2. Each spawnable role is an AgentDefinition: name, description (when the coordinator should use it), prompt (scopes behaviour), and tools (restriction). It is the API analogue of Claude Code subagent definitions.

Python

from claude_agent_sdk import ClaudeAgentOptions, AgentDefinition

options = ClaudeAgentOptions(
    system_prompt=COORDINATOR_SYSTEM_PROMPT,
    allowed_tools=["Task"],
    agents={
        "search_agent": AgentDefinition(
            description="Runs web searches for a specific subtopic and "
                        "returns findings with source URLs attached.",
            prompt=SEARCH_AGENT_SYSTEM_PROMPT,
            tools=["WebSearch", "WebFetch"],
        ),
        "document_agent": AgentDefinition(
            description="Loads and extracts claims from a specific set of "
                        "documents, returning findings with document name "
                        "and page number attached.",
            prompt=DOCUMENT_AGENT_SYSTEM_PROMPT,
            tools=["load_document", "extract_data_points"],
        ),
        "synthesis_agent": AgentDefinition(
            description="Writes a cited research report from findings "
                        "supplied in its prompt. Has no research tools.",
            prompt=SYNTHESIS_AGENT_SYSTEM_PROMPT,
            tools=[],
        ),
    },
)

TypeScript

import { AgentDefinition } from "@anthropic-ai/claude-agent-sdk";

const options = {
  systemPrompt: COORDINATOR_SYSTEM_PROMPT,
  allowedTools: ["Task"],
  agents: {
    search_agent: {
      description: "Runs web searches for a specific subtopic and returns " +
        "findings with source URLs attached.",
      prompt: SEARCH_AGENT_SYSTEM_PROMPT,
      tools: ["WebSearch", "WebFetch"],
    } satisfies AgentDefinition,
    document_agent: {
      description: "Loads and extracts claims from a specific set of " +
        "documents, returning findings with document name and page " +
        "number attached.",
      prompt: DOCUMENT_AGENT_SYSTEM_PROMPT,
      tools: ["load_document", "extract_data_points"],
    } satisfies AgentDefinition,
    synthesis_agent: {
      description: "Writes a cited research report from findings supplied " +
        "in its prompt. Has no research tools.",
      prompt: SYNTHESIS_AGENT_SYSTEM_PROMPT,
      tools: [],
    } satisfies AgentDefinition,
  },
};

• Caveat on allowedTools: the SDK documents it as an auto-approve list that "does not restrict Claude to only these tools". Omitting "Task" won't reliably block delegation in real code, but it is still the exam's answer to "why won't my coordinator spawn subagents?". By contrast, AgentDefinition.tools is a real fence: an omitted tool "isn't in the subagent's session at all."
• description is what the coordinator routes on. A vague one misroutes between subagent types, just as a vague tool description does between tools.
• synthesis_agent has no tools by design. That enforces the Chapter 3 scoping argument at the config level.

Other AgentDefinition fields (not tested field by field):

Resuming a subagent

A Task/Agent result carries a session_id (surfaced as agentId in current tooling) for that subagent's transcript. Pass it back as resume: sessionId on a later Task call to continue that subagent's history instead of starting from nothing. This helps when a document subagent has built up expensive context on a document set. Resume when the prior context is still valid and worth its cost; otherwise spawn fresh (the same judgement as Chapter 7).

Spawning in parallel: multiple Task calls, one turn

Parallel delegation = multiple Task calls in a single coordinator response. Four Task calls in one turn run four subagents side by side. One per turn over four turns runs them in sequence, at much higher wall-clock cost, since each is its own multi-step loop.

There is no parallelism flag. Distractors offer real parameters that govern something else, e.g. disable_parallel_tool_use, which controls batching of ordinary tool calls in a turn.

With query(), the SDK harness sees the four tool_use blocks and runs the four loops concurrently. You supply options and read typed messages.

Python

from claude_agent_sdk import query

async for message in query(
    prompt="Research the market impact of generative AI on creative "
           "industries: visual arts, music, writing, and film/video, "
           "each as a distinct workstream.",
    options=options,  # the ClaudeAgentOptions defined above
):
    log.info("%s", message)
# Internally, the coordinator's turn carries four Task tool_use blocks
# naming "search_agent", each with a distinct, non-overlapping subtopic
# in its input — the SDK runs the four resulting subagent loops concurrently.

TypeScript

import { query } from "@anthropic-ai/claude-agent-sdk";

for await (const message of query({
  prompt: "Research the market impact of generative AI on creative " +
    "industries: visual arts, music, writing, and film/video, each as " +
    "a distinct workstream.",
  options, // the ClaudeAgentOptions defined above
})) {
  log.info(message);
}
// Internally, the coordinator's turn carries four Task tool_use blocks
// naming "search_agent", each with a distinct, non-overlapping subtopic
// in its input — the SDK runs the four resulting subagent loops concurrently.

Fan-out ceilings exist because anything that can spawn more of itself needs a hard cap:

• CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH: nesting depth (default 3).
• CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS: concurrent loops (default 20).
• maxBudgetUsd / max_budget_usd: total spend across the delegation tree. Exceeding it stops with result subtype error_max_budget_usd.

Partitioning scope without narrowing the topic

Splitting "creative industries" into visual arts, music, writing, and film/video partitions scope correctly: no duplication, and full coverage. The exam calls this partitioning research scope to minimise duplication: assign each subagent distinct subtopics or source types (e.g. one agent on regulatory filings, one on news, one on vendor pages).

Now split it into gallery representation, digital art marketplaces, and museum curation, which are all visual arts. Every subagent succeeds with well-cited findings. Synthesis faithfully produces a confident report covering about a quarter of the topic. Nothing errored, every log is green, and the answer is wrong.

When every subagent succeeds and aggregate coverage is still wrong, the defect is in the coordinator's decomposition. Error handling can't catch it. You catch it only by comparing the decomposition against the original question's breadth, before delegating and again after synthesis.

Iterative refinement: the coordinator checks its own coverage

A one-pass pipeline (decompose → delegate → synthesise) can't catch narrow decomposition after the fact. Make it a loop: evaluate synthesis for gaps against the original question, re-delegate targeted queries for the gaps, re-synthesise, and repeat until coverage is sufficient.

Python

def run_research_coordinator(question, max_refinement_rounds=3):
    findings = delegate_initial_research(question)   # parallel Task calls
    report = invoke_synthesis(question, findings)

    for round_num in range(max_refinement_rounds):
        gaps = evaluate_coverage(question, report)    # coordinator's own check
        if not gaps:
            return report
        log.info("refinement round=%d gaps=%s", round_num, gaps)
        targeted_findings = delegate_targeted_research(gaps)  # more Task calls
        findings = findings + targeted_findings
        report = invoke_synthesis(question, findings)

    report["coverage_warning"] = (
        "Refinement rounds exhausted; known gaps may remain unresolved."
    )
    return report

TypeScript

async function runResearchCoordinator(
  question: string,
  maxRefinementRounds = 3,
) {
  let findings = await delegateInitialResearch(question); // parallel Task calls
  let report = await invokeSynthesis(question, findings);

  for (let round = 0; round < maxRefinementRounds; round++) {
    const gaps = await evaluateCoverage(question, report); // coordinator's own check
    if (gaps.length === 0) return report;
    log.info({ round, gaps });
    const targetedFindings = await delegateTargetedResearch(gaps); // more Task calls
    findings = [...findings, ...targetedFindings];
    report = await invokeSynthesis(question, findings);
  }

  report.coverageWarning =
    "Refinement rounds exhausted; known gaps may remain unresolved.";
  return report;
}

evaluate_coverage must look for named subtopics that never got a subagent, not just check for non-empty output. The three visual-arts subagents all returned good, non-empty results. The loop is bounded and warns loudly when exhausted. It never silently returns incomplete work as complete.

Query complexity: not every question needs the full pipeline

The opposite failure is over-processing. "What year was [a well-known data regulation] enacted?" doesn't need search, then document analysis, then synthesis. The coordinator's prompt should have it assess complexity first (simple lookup, comparison, or open investigation) and invoke only the subagents needed. Left at "always run the full team", it spends four invocations on a one-agent question. In support, a plain order-status request doesn't need a fraud check.

Fork-based sessions, briefly

Forking branches a session into independent copies from a shared baseline, so you can explore two divergent approaches (e.g. two report framings) without re-deriving the shared analysis. It is a session mechanism, covered in Chapter 7.

Scaling past hub-and-spoke, and the default subagent

• Workflow tool (separate from Task/Agent) orchestrates dozens to hundreds of agents outside the coordinator's conversation. Hub-and-spoke keeps everything in the coordinator's context, which is what buys observability and controlled flow, and that context would overflow at large scale. They answer different problem sizes. Hub-and-spoke still applies at small to moderate scale.
• general-purpose is a built-in subagent always available even with no agents defined, unless disabled with CLAUDE_AGENT_SDK_DISABLE_BUILTIN_AGENTS=1. A coordinator with an empty agents map can still delegate.

What the exam tests

Tasks 1.2 and 1.3 are diagnostic: name the cause of a symptom.

• Coordinator never spawns subagents: check allowedTools for "Task".
• Generic or uncited subagent output: put the findings in its prompt. Don't add tools or restructure turns.
• Parallel delegation: multiple Task calls in one coordinator response. Distractors are real parameters that govern adjacent things.
• Narrow decomposition: clean logs, wrong coverage. The defect is upstream in decomposition, found by comparing the question's breadth with what was delegated.
• Complexity assessment versus routing everything through the full pipeline.
• Hub-and-spoke benefits (observability, consistent error handling, controlled information flow) versus direct subagent communication.
• Right for a reason: structured findings with separated content and metadata; coordinator prompts stating goals and quality criteria, not procedure.

Exercises

1. Build the research coordinator with search_agent, document_agent, and synthesis_agent (tools scoped per Chapter 3), with the synthesis prompt as just "Synthesize the research findings into a report." Observe the generic output, then pass structured, attributed findings and compare.
2. Have a coordinator decompose "the effects of remote work on urban commercial real estate" twice: once narrowly (three subtopics, same angle) and once broadly. Compare reports and write one sentence on what a reader of the narrow one would wrongly believe is covered.
3. Add the refinement loop to exercise 2's narrow decomposition. Confirm it finds the gap, re-delegates, and closes it. Then remove the cap and make evaluate_coverage impossible to satisfy.
4. Have the coordinator assess complexity first. Run a single-fact lookup, a two-way comparison, and an open investigation, and log which subagents run for each.

Chapter 6 — Hooks and gates

Claude Code's permission prompt is an architectural pattern. The model emits a tool_use block, and before the call reaches the real world, code outside the model inspects it and can refuse. Nobody asked Claude nicely not to run rm -rf. The loop was stopped at a point the model has no say over.

This is arguably the highest-yield idea on the exam: prompt instructions have a non-zero failure rate. However carefully you write "always verify the customer before refunding", enough conversations, context pressure, and adversarial phrasing will produce some runs that refund anyway. An instruction competes with every other token for attention. A gate doesn't compete: it runs, checks, and returns a fixed answer.

In the support scenario, most decisions (which order, which policy, what to say) are fine left to the model. Two properties must never be false:

1. A refund is never issued against an unverified identity.
2. A refund never exceeds the policy threshold without a human.

Claude Code's hook system exposes the two seams: PreToolUse (before a call executes) and PostToolUse (after a result returns). The permission prompt is a PreToolUse hook with a UI. This chapter builds both seams by hand, because the exam tests the pattern, not the config file (Chapters 9–10 cover that).

The enforcement ladder

The exam's trap: forcing get_customer guarantees only that the next response calls it. Three turns later you're back on "auto", and nothing stops an unverified process_refund. A prerequisite across an open-ended conversation must live in code that persists and inspects every call. That is a gate.

When to use deterministic enforcement: the consequence is financial, compliance, or security, or an ordering must never be skipped (identity before payment is the canonical case). Below that tier, a good prompt is fine, and a hook is a mistake (see the counterweight below).

Prerequisite gates: blocking the call, not asking nicely

A prerequisite gate sits between the model's request and execution and refuses the call until an earlier condition holds. The exam's instance: block process_refund until get_customer has returned a verified customer ID. It's a dict lookup and an if, but three details matter:

1. State is per conversation, not process-global. One shared boolean across concurrent sessions lets session B refund on session A's verification. That fails silently and looks like success, which is worse than no gate. Key the state by conversation ID.
2. Return, don't raise. A raised block kills the loop like any uncaught exception (Chapter 4). Return a normal tool_result with is_error and a description of what's missing, and the model recovers: it calls get_customer, then retries.
3. Gate the mutation, not neighbouring reads. Gating lookup_order as well gains no safety and stops the agent answering "what's in my order?" for the majority of contacts that never refund. A sample question offers "gate both lookup_order and process_refund" as a wrong answer because it is disproportionate, not unsafe.

Python

verified_customers: dict[str, str] = {}  # conversation_id -> customer_id

def gated_process_refund(conversation_id: str, order_id: str,
                          amount_cents: int, reason: str) -> dict:
    customer_id = verified_customers.get(conversation_id)
    if customer_id is None:
        return {
            "is_error": True,
            "content": (
                "process_refund blocked: no verified customer on this "
                "conversation. Call get_customer first, then retry with "
                "the returned customer_id."
            ),
        }
    return process_refund(customer_id, order_id, amount_cents, reason)

def gated_get_customer(conversation_id: str, identifier: str) -> dict:
    result = get_customer(identifier)
    if result.get("verified"):
        verified_customers[conversation_id] = result["customer_id"]
    return result

TypeScript

const verifiedCustomers = new Map<string, string>(); // conversationId -> customerId

async function gatedProcessRefund(
  conversationId: string, orderId: string, amountCents: number, reason: string,
) {
  const customerId = verifiedCustomers.get(conversationId);
  if (!customerId) {
    return {
      is_error: true,
      content:
        "process_refund blocked: no verified customer on this conversation. " +
        "Call get_customer first, then retry with the returned customer_id.",
    };
  }
  return processRefund(customerId, orderId, amountCents, reason);
}

async function gatedGetCustomer(conversationId: string, identifier: string) {
  const result = await getCustomer(identifier);
  if (result.verified) verifiedCustomers.set(conversationId, result.customerId);
  return result;
}

Direction is not a detail: interception versus PostToolUse

This is an easy distractor, because the two look alike in a diagram:

• Interception (PreToolUse, pre-call) inspects a call before dispatch and can block it.
• PostToolUse inspects a result after it returns. The action has already happened.

Once process_refund returns, the money has moved. PostToolUse can log, flag, or page, but it can't un-refund. "Refunds over $500 never go out without a human" can only live before the call, in interception.

Python

REFUND_THRESHOLD_CENTS = 50_000

def intercept_process_refund(conversation_id: str, order_id: str,
                              amount_cents: int, reason: str) -> dict | None:
    """Return a blocking result if the call must not proceed, else None."""
    if amount_cents > REFUND_THRESHOLD_CENTS:
        return {
            "is_error": True,
            "content": (
                f"process_refund blocked: ${amount_cents/100:.2f} exceeds the "
                "$500 auto-approval threshold. Call escalate_to_human with a "
                "root cause and recommended action instead."
            ),
        }
    return None

def dispatch_process_refund(conversation_id, order_id, amount_cents, reason):
    blocked = intercept_process_refund(conversation_id, order_id, amount_cents, reason)
    if blocked is not None:
        return blocked
    return gated_process_refund(conversation_id, order_id, amount_cents, reason)

TypeScript

const REFUND_THRESHOLD_CENTS = 50_000;

function interceptProcessRefund(
  conversationId: string, orderId: string, amountCents: number, reason: string,
) {
  if (amountCents > REFUND_THRESHOLD_CENTS) {
    return {
      is_error: true,
      content:
        `process_refund blocked: $${(amountCents / 100).toFixed(2)} exceeds the ` +
        "$500 auto-approval threshold. Call escalate_to_human with a root cause " +
        "and recommended action instead.",
    };
  }
  return null;
}

async function dispatchProcessRefund(
  conversationId: string, orderId: string, amountCents: number, reason: string,
) {
  const blocked = interceptProcessRefund(conversationId, orderId, amountCents, reason);
  if (blocked) return blocked;
  return gatedProcessRefund(conversationId, orderId, amountCents, reason);
}

The block redirects: it names escalate_to_human, so the agent has a productive next step instead of retrying or giving up. It sits in the same seam as Claude Code's permission prompt, between tool_use and execution.

Typical refund-threshold question:

PostToolUse: normalizing what comes back

PostToolUse solves data quality, not compliance. Real support tooling spans systems: lookup_order returns Unix epoch timestamps, get_customer returns ISO 8601, a third returns status codes 0/1/2 with the meaning in an unattached lookup table. Making the model translate formats as a side job is the kind of mechanical task a probabilistic reader gets wrong on the odd input.

A PostToolUse hook rewrites every result into one shape (field names, date format, status vocabulary) before the model sees it.

Python

from datetime import datetime, timezone

STATUS_CODES = {0: "active", 1: "suspended", 2: "closed"}

def normalize_tool_result(tool_name: str, raw: dict) -> dict:
    result = dict(raw)
    if tool_name == "lookup_order" and isinstance(result.get("ship_date"), int):
        result["ship_date"] = datetime.fromtimestamp(
            result["ship_date"], tz=timezone.utc
        ).date().isoformat()
    if tool_name == "get_customer" and isinstance(result.get("status"), int):
        result["status"] = STATUS_CODES.get(result["status"], "unknown")
    return result

TypeScript

const STATUS_CODES: Record<number, string> = { 0: "active", 1: "suspended", 2: "closed" };

function normalizeToolResult(toolName: string, raw: Record<string, unknown>) {
  const result = { ...raw };
  if (toolName === "lookup_order" && typeof result.ship_date === "number") {
    result.ship_date = new Date(result.ship_date * 1000).toISOString().slice(0, 10);
  }
  if (toolName === "get_customer" && typeof result.status === "number") {
    result.status = STATUS_CODES[result.status as number] ?? "unknown";
  }
  return result;
}

Wire it into execute_tool from Chapter 2: call the tool, normalise, then serialise into content.

Agent SDK names (task 1.5 is phrased in SDK terms):

• PreToolUse callback returns hookSpecificOutput with permissionDecision ("allow", "deny", "ask", "defer") and permissionDecisionReason. It can rewrite arguments via updatedInput instead of blocking.
• PostToolUse callback returns additionalContext to append, or updatedToolOutput to replace the output before Claude sees it. That is the SDK capability behind normalisation.

Why a hook rather than "dates may be Unix or ISO; interpret carefully" in the prompt? The prompt usually works but spends tokens every turn re-deriving a fixed fact. And if lookup_order is a third-party MCP server, prompting the model can't change what it returns. You can only change what the model is shown. The hook converts once, and every turn gets clean data.

The counterweight: not every rule needs a hook

Hooks are code: they have bugs, need maintenance, and add failure seams. For soft preferences (phrase refunds politely, prefer store credit, mention the loyalty programme), a prompt is adequate, misses are cheap, and a hook is over-engineering.

Test: does a probabilistic miss justify code that can never be skipped? Money, skipped identity checks, compliance: yes. Tone, phrasing, a choice between two acceptable orderings: no. Over-gating brings back the rigidity of the decision trees criticised in Chapter 2.

Multi-concern decomposition: one customer, several problems

"SO-44120 arrived damaged, and I was also charged twice for last month's subscription" is two concerns in one message. Named failures:

• Resolving one and telling the customer to open a second ticket.
• Handling independent concerns serially.

Both hurt first-contact resolution, the scenario's headline metric.

Correct pattern: decompose into distinct items, investigate in parallel on shared context, and synthesise one unified resolution. Shared context means fetching the customer once and passing the same verified customer_id to both branches, not having each call get_customer (and re-verify through the gate). The two lookups are independent once the customer is known, so they can be parallel tool_use blocks.

Python

async def resolve_multi_concern(conversation_id: str, customer_identifier: str,
                                 concerns: list[dict]) -> dict:
    customer = gated_get_customer(conversation_id, customer_identifier)
    findings = await asyncio.gather(*(
        investigate(concern, customer_id=customer["customer_id"])
        for concern in concerns
    ))
    return synthesize_resolution(customer, concerns, findings)

TypeScript

async function resolveMultiConcern(
  conversationId: string, customerIdentifier: string, concerns: Concern[],
) {
  const customer = await gatedGetCustomer(conversationId, customerIdentifier);
  const findings = await Promise.all(
    concerns.map((concern) => investigate(concern, customer.customerId)),
  );
  return synthesizeResolution(customer, concerns, findings);
}

Synthesis produces one reply resolving both concerns, not two answers stapled together. The judgement call is recognising a compound message as decomposition, not escalating it merely for being compound.

Structured handoff: what the human on the other end actually needs

escalate_to_human fires when interception blocks a refund, policy is silent, or the customer asks for a person. The human who receives it has no access to the transcript. They open the ticket cold. A raw transcript dump hands them a research task under time pressure.

A structured handoff carries four fields:

Python

def escalate_to_human(customer_id: str, root_cause: str, amount_cents: int,
                       recommended_action: str) -> dict:
    return {
        "is_error": False,
        "content": json.dumps({
            "customer_id": customer_id,
            "root_cause": root_cause,
            "refund_amount_cents": amount_cents,
            "recommended_action": recommended_action,
        }),
    }

TypeScript

function escalateToHuman(
  customerId: string, rootCause: string, amountCents: number, recommendedAction: string,
) {
  return {
    is_error: false,
    content: JSON.stringify({
      customerId,
      rootCause,
      refundAmountCents: amountCents,
      recommendedAction,
    }),
  };
}

Wrong answers in the select-two version:

• Raw transcript: the thing this structure replaces.
• Agent's self-reported confidence: the same unreliable signal rejected for autonomous escalation (Chapter 15), and just as unreliable here.
• Tool calls with timings: debugging telemetry, not something the human can act on now.

What the exam tests

• Tasks 1.4/1.5 spine: prompts have a non-zero failure rate, so enforcement for financial, compliance, or security misses belongs in code. The standing example is identity before refund.
• Agent occasionally skips a mandatory step: add a programmatic prerequisite gate. Not a stronger instruction, more few-shot examples, or an iteration-cap workaround.
• Direction (the most tested 1.5 distractor): PostToolUse can't stop what already happened. Interception blocks before dispatch. PostToolUse for a refund threshold is the right family, wrong direction.
• Interception redirects to escalate_to_human rather than issuing a bare denial.
• Heterogeneous formats (Unix, ISO 8601, numeric codes): PostToolUse normalisation, not "interpret carefully".
• Multi-concern: decompose, parallel on shared context, one resolution. Not serial, not escalated for being compound.
• Handoff fields: customer ID, root cause, refund amount, recommended action. Reject transcript, confidence score, and telemetry.
• Counterweight: hooks are for guarantees, not preferences. Gating read-only lookups is disproportionate.

Exercises

1. Add verified_customers to Chapter 2's run_agent, wire in gated_get_customer/gated_process_refund, and push the model to refund before verifying. Confirm the block arrives as a normal tool_result and the model recovers.
2. Put the $500 interception in front of gated_process_refund. Run a $200 and a $650 request, and confirm only the second is blocked and redirected to escalate_to_human.
3. Build two lookup_order stubs (Unix and ISO ship_date) and make normalize_tool_result produce identical output. Then replace it with "interpret dates carefully" in the prompt, run 20 conversations, and compare delivery-window reasoning accuracy.
4. Write resolve_multi_concern with two stub investigations sharing one customer fetch. Then make each call get_customer itself and observe the extra round trip the gate forces.

Chapter 7 — Sessions

You resume a two-day-old Claude Code session and Claude confidently describes a file you rewrote yesterday. It isn't hallucinating. It is reading its memory correctly, but the memory is old. Task statement 1.7 is about the gap between "history is intact" and "history is true": when to resume, when to fork, and when resuming is worse than starting over.

What a session actually is

A session is the running list of blocks: user turns, the model's text and tool calls, and the tool_result blocks that came back. It is replayed to the model every turn. Resuming reloads that list. It revalidates nothing. Every tool result in it is treated as still describing the world.

That's what lets a resumed refactor remember the plan and naming decisions. It is also a cache, and it is only correct if the world hasn't moved since it was written.

Resuming a named session

The skill: --resume <session-name> to continue a specific investigation, not just the latest one, because you usually have several lines of work going.

Example: on Monday Claude explored a legacy billing module, designed a split of process_refund out of a monolith, and started the new module. On Tuesday, if nothing changed overnight, resuming is clearly right, and re-deriving it would be waste.

What the exam tests is the condition, not the flag: resume when prior context is mostly still valid.

resume versus continue

With several lines of work in one project, continue may pick the wrong one.

When resuming lies to you

Overnight, a teammate rewrote six files in the auth module your session analysed. You resume. The transcript still contains Monday's contents and analysis verbatim, and nothing marks them as stale. Staying silent means Claude reasons from Monday's files, and every downstream decision inherits the error. It is Chapter 4's failure again: an unlabelled stale result reads as current, just as an unlabelled failure read as empty.

Fix: when you resume, tell the agent specifically what changed. Naming the six files by path makes the problem explicit and bounded. Claude re-reads exactly those, keeping Monday's reasoning and discarding only the facts that went bad. This is the guide's skill of "informing a resumed session about specific file changes for targeted re-analysis rather than requiring full re-exploration."

Python

resume_prompt = """
Resuming the refund-module refactor session.
Since we last worked on this, the following files were rewritten
upstream and your prior analysis of them is stale — re-read and
re-analyze these before continuing, everything else in your prior
context still holds:

  - src/billing/auth_context.py
  - src/billing/session_guard.py
  - src/billing/token_refresh.py
  - src/billing/scopes.py
  - src/billing/middleware.py
  - src/billing/audit_hooks.py

Do not rely on your previous reading of these six files.
"""

TypeScript

const resumePrompt = `
Resuming the refund-module refactor session.
Since we last worked on this, the following files were rewritten
upstream and your prior analysis of them is stale — re-read and
re-analyze these before continuing, everything else in your prior
context still holds:

  - src/billing/auth-context.ts
  - src/billing/session-guard.ts
  - src/billing/token-refresh.ts
  - src/billing/scopes.ts
  - src/billing/middleware.ts
  - src/billing/audit-hooks.ts

Do not rely on your previous reading of these six files.
`;

The instruction states what is still valid (design, plan) and pins exactly what is stale. "Some files changed, please double-check" triggers the unbounded re-exploration you resumed to avoid.

When resuming is the wrong tool entirely

A research session crawled competitor pricing, a regulatory filing, and news. By the time you return, prices have updated, the filing was amended, and three links 404. Most of what the session knows is suspect, and no "here's what changed" note can rescue it.

The guide states it bluntly: starting a new session with a structured summary is more reliable than resuming with stale tool results. A resumed history is tool_result blocks that were true when fetched, with no signal of decay. A fresh session seeded with a summary of only facts you've just re-confirmed (the questions, the conclusions that don't depend on today's prices, the sources worth revisiting) reasons from something you vouched for. Current Agent SDK guidance says the same: once tool results have plausibly gone stale, capture what you've learned as application state and pass it into a fresh session.

Python

structured_summary = """
Prior research session (competitor pricing analysis) concluded with
these durable findings — re-verify nothing below unless noted:

  - Research questions: (1) how does competitor X price the enterprise
    tier, (2) what regulatory constraints apply in the EU market.
  - Methodology that worked: querying the vendor's public pricing page
    plus the EU filing register directly outperformed general web search.
  - Durable conclusion: competitor X's enterprise tier is seat-based,
    not usage-based — this is a structural fact unlikely to have moved.

Treat all specific prices, filing statuses, and news article contents
from the prior session as UNVERIFIED. Re-fetch before citing any figure.
"""

TypeScript

const structuredSummary = `
Prior research session (competitor pricing analysis) concluded with
these durable findings — re-verify nothing below unless noted:

  - Research questions: (1) how does competitor X price the enterprise
    tier, (2) what regulatory constraints apply in the EU market.
  - Methodology that worked: querying the vendor's public pricing page
    plus the EU filing register directly outperformed general web search.
  - Durable conclusion: competitor X's enterprise tier is seat-based,
    not usage-based — this is a structural fact unlikely to have moved.

Treat all specific prices, filing statuses, and news article contents
from the prior session as UNVERIFIED. Re-fetch before citing any figure.
`;

There is no threshold number. Place the scenario:

• A known, enumerable subset changed: resume and inform.
• Staleness is broad or unenumerable (a wide crawl aged out, the data source rotated, too much time passed): start fresh with a summary. A patch note would be as long and uncertain as redoing the work. The summary is small and hand-checked, while the transcript is large and unchecked.

Forking: one baseline, divergent explorations

fork_session answers a different question: you've paid for a shared baseline and want to explore several paths from it without them polluting each other, and without paying for the baseline twice.

Example: Claude has mapped a service's test setup (fixtures, mocks, flaky spots). Two strategies, "migrate to contract tests" and "add property-based tests", both need that map and then diverge.

• Both in one session: strategy A's dead ends stay in context and warp reasoning about B.
• Two fresh sessions: no pollution, but the careful exploration is repeated or reduced to a summary.
• fork_session: each branch inherits the full baseline once, then diverges in isolation.

The research scenario is the same: fork after the shared literature review into a regulatory branch and a pricing branch.

Forking is a resume with a flag: point at the baseline id and branch rather than continue.

Python

from claude_agent_sdk import ClaudeAgentOptions, query

contract_tests_branch = query(
    prompt="Propose a migration to contract tests for this service.",
    options=ClaudeAgentOptions(resume=baseline_session_id, fork_session=True),
)

incremental_branch = query(
    prompt="Propose adding property-based tests to the existing suite.",
    options=ClaudeAgentOptions(resume=baseline_session_id, fork_session=True),
)

TypeScript

import { query } from "@anthropic-ai/claude-agent-sdk";

const contractTestsBranch = query({
  prompt: "Propose a migration to contract tests for this service.",
  options: { resume: baselineSessionId, forkSession: true },
});

const incrementalBranch = query({
  prompt: "Propose adding property-based tests to the existing suite.",
  options: { resume: baselineSessionId, forkSession: true },
});

Both calls resume baseline_session_id, and fork_session=True gives each its own new session copied from the baseline. Plain --resume twice would put both prompts in one growing transcript. (Python's ClaudeSDKClient wraps resume/fork in a session-holding object, so real apps rarely pass ids by hand.)

What the exam tests

Task 1.7 stems ask you to pick one of the four rows above.

• A handful of files changed: resume and name the files. Traps: resume silently (stale results read as current), or discard everything (throws away valid context to fix a narrow problem).
• Broad staleness (long gap, wide crawl, external data): fresh session with an injected summary. A targeted note isn't enough because there's no small list to give. Know the reason: resumed tool_results carry no marker of decay.
• "Compare" / "divergent approaches" from "the same starting point": fork_session. Traps: repeated --resume into one transcript, or duplicated fresh sessions that discard the baseline.

Exercises

1. Run a multi-file exploration under a named session, then edit two or three of the files it read. Resume twice from the same point, once saying nothing and once naming the changed files, and compare how each reasons about them.
2. Fork that exploration into an aggressive-refactor branch and a minimal one. Confirm neither sees the other, and note what you'd lose running both as follow-ups in the original session.
3. After a wide web-research session, write a summary of only what you'd trust unchecked. Compare a fresh session seeded with it against resuming the original. Find where the resumed one cites a fetched detail as current.

Chapter 8 — MCP

As a consumer, MCP looks like a plugin system: add a config entry, get new tools. The exam wants the layer above that: what goes in the config, who else sees it, and whether a tool should exist at all.

MCP standardises the interface between an agent and an external capability. Before it, every agent needed its own bespoke integration for each tool. Now one GitHub server works identically from Claude Code, an Agent SDK app, or another host.

Scope of task 2.4: configuration, scoping, discovery, descriptions, build versus buy. Out of scope: transports (stdio vs network), hosting, containers, load balancing, OAuth. Answer options mentioning those are decoration on a wrong answer.

What MCP actually standardizes

Resources are for catalogs; tools are for actions. A resource fixes a specific reliability problem: an agent that doesn't know what exists will guess and call tools speculatively.

• Issue tracker: without a resource, the agent calls search_issues 2–3 times with guessed keywords just to learn which components and labels exist. A resource exposing the label taxonomy, component list, or open-epic summary lets it read once, then make one targeted call.
• Document store: a resource exposing folder hierarchy and titles stops repeated list_documents calls at different prefixes.

You won't implement resources by hand for the exam. You need the judgement: content the agent browses to orient itself is a resource, and things that act are tools.

Scoping: local, project, and user

Scope decides who shares the configuration, nothing about inference.

• The exam guide uses the two-scope model (project vs user). Local is real in current Claude Code, but assume project-vs-user unless a question names local.
• Don't confuse local MCP scope with .claude/settings.local.json, an unrelated use of "local".
• Set scope with claude mcp add <name> --scope local|project|user ... rather than hand-editing ~/.claude.json.
• The my-experimental-notes-server below fits local scope better than user scope today.

// .mcp.json — project scope, committed, shared with the team
{
  "mcpServers": {
    "jira": {
      "command": "npx",
      "args": ["-y", "@some-vendor/jira-mcp-server"],
      "env": {
        "JIRA_API_TOKEN": "${JIRA_API_TOKEN}",
        "JIRA_BASE_URL": "https://acme.atlassian.net"
      }
    }
  }
}

// ~/.claude.json — user scope, personal, not committed anywhere
{
  "mcpServers": {
    "my-experimental-notes-server": {
      "command": "node",
      "args": ["/Users/paulo/dev/notes-mcp/index.js"]
    }
  }
}

Environment variable expansion. ${JIRA_API_TOKEN} in .mcp.json is a placeholder resolved from the runner's environment at connection time. The team shares the server config while each engineer's token stays in their own shell or .env. Hardcoding a token in .mcp.json is always wrong, because a committed secret is a leaked secret.

• Fallbacks: ${JIRA_BASE_URL:-https://acme.atlassian.net}.
• Expansion works in env, in command/args (stdio), and in url/headers (HTTP).

Scope is not priority. Tools from every configured server, project and user alike, are discovered at connection time and available simultaneously, in one flat inventory with the built-ins. The model picks among them per turn, just as it picks between Grep and Glob. A question where a project server "takes priority" over a user server, or hides it, describes something that doesn't happen.

One narrow exception, which the exam doesn't test: when two scopes define a server with the same name, Claude Code resolves the collision with a fixed precedence (local → project → user → plugin → claude.ai connector) and connects once. That is scope resolution. Tool selection stays priority-free.

Why your MCP tool loses to Grep

You install a server with a better, structured search, and Claude still uses Grep or WebSearch. The cause is the Chapter 3 principle: the description is prompt text read at selection time. Grep has a careful paragraph. Your tool has "Search Jira issues". The model has more evidence for Grep every time.

Fix: enhance the MCP tool's description.

Python

search_issues = {
    "name": "search_issues",
    "description": (
        "Search Jira issues by text, status, assignee, or label within the "
        "acme-platform project. Returns issue key, summary, status, assignee, "
        "labels, and the last three comments for each match — structured data "
        "pulled live from Jira, not a text search over a local checkout.\n\n"
        "Use this for any question about tickets, bugs, or work items: "
        "'what's blocking the payments epic', 'find open bugs tagged "
        "checkout', 'who is assigned OPS-4471'.\n"
        "Do NOT use Grep or a general web search for these queries — issue "
        "text lives in Jira, not in the repository, and this tool returns "
        "current status and assignment that a code search cannot see.\n"
        "If no issues match the filters, returns an empty list; this is a "
        "successful call, not an error, and usually means the filters are "
        "too narrow rather than that Jira is unreachable."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Free-text search, e.g. 'refund timeout'."},
            "status": {"type": "string", "description": "Optional status filter, e.g. 'In Progress'."},
        },
        "required": ["query"],
    },
}

TypeScript

const searchIssues: Anthropic.Tool = {
  name: "search_issues",
  description: [
    "Search Jira issues by text, status, assignee, or label within the",
    "acme-platform project. Returns issue key, summary, status, assignee,",
    "labels, and the last three comments for each match — structured data",
    "pulled live from Jira, not a text search over a local checkout.",
    "",
    "Use this for any question about tickets, bugs, or work items: 'what's",
    "blocking the payments epic', 'find open bugs tagged checkout', 'who is",
    "assigned OPS-4471'.",
    "Do NOT use Grep or a general web search for these queries — issue text",
    "lives in Jira, not in the repository, and this tool returns current",
    "status and assignment that a code search cannot see.",
    "If no issues match the filters, returns an empty list; this is a",
    "successful call, not an error, and usually means the filters are too",
    "narrow rather than that Jira is unreachable.",
  ].join("\n"),
  input_schema: {
    type: "object",
    properties: {
      query: { type: "string", description: "Free-text search, e.g. 'refund timeout'." },
      status: { type: "string", description: "Optional status filter, e.g. 'In Progress'." },
    },
    required: ["query"],
  },
};

The key sentence is "do NOT use Grep": it names the competing built-in, just as boundary sentences name sibling tools. An MCP tool competes with the built-ins exactly as your own tools compete with each other. Community servers sometimes ship thin descriptions. Getting the protocol right and getting the prompt right are different skills, so when configuring one for a team, check the descriptions and wrap or override them if needed.

Build versus buy

• Standard integrations (Jira, GitHub, Slack, Drive, common DBs): use an existing community or vendor server, configured at project scope with env-expanded tokens. Someone has already handled pagination, custom-field mapping, rate limiting, and the API edge cases that only show up after months across many installations. A home-grown one handles what you tested and silently breaks on the rest.
• Team-specific workflows: build custom. Examples: one tool wrapping three internal systems, a resource exposing your own document taxonomy, business rules such as which refund reasons need a supervisor or which labels auto-escalate. A generic server can't know your rules exist. You also then own description quality from scratch, which is one more reason not to build by default.

Keep the question types apart. Scope and priority are configuration facts with one answer each. Build-vs-buy and description quality are judgement calls that depend on the scenario.

What the exam tests

• Scope recall: project .mcp.json (shared) versus user ~/.claude.json (personal/experimental), plus env var expansion. The wrong answer hardcodes a token in a committed file.
• Scope ≠ priority: with both scopes configured, every server's tools are discovered at connection time and sit together in one inventory, with no scope-based priority.
• Resources vs tools: an agent burning calls on exploratory discovery of a content catalog (issue summaries, doc hierarchy, schema) needs a resource, not a new tool or a bigger prompt.
• MCP tool loses to Grep: enhance its description. Don't change scope, restrict the built-in, or add a routing rule.
• Build vs buy: standard integration means a community server. Custom is only for team-specific needs.
• Transports, hosting, and OAuth are out of scope.

Exercises

1. In a real repo, write a project .mcp.json entry for a community server with ${VAR} for every credential, and a personal entry in ~/.claude.json. Justify each placement in one sentence.
2. Inspect a tool description from an MCP server you use and audit it against Chapter 3's five elements. If it's thin, rewrite it, adding a sentence naming the built-in (Grep, WebSearch) it should beat.
3. For an internal issue tracker, split capabilities into tools (create, transition, assign) and resources (label taxonomy, components, sprint summaries), justifying each as an action or a catalog.
4. In 2–3 sentences, argue why a "file ticket + tag per internal escalation rules + notify on-call" tool should be custom rather than assembled from Jira and Slack servers, and name the business logic a generic server couldn't know.

Chapter 9 — Configuring Claude Code

You already use CLAUDE.md. This chapter gives you the exam's vocabulary for it (task statements 3.1 and 3.3): which file wins, what "shared" means at each level, and the sharpest distinction in the domain. A rule can load because of where a file sits (directory-bound) or because of what kind of file it is (type-bound, via glob). Test files interleaved across 40 packages have no directory whose boundary is "every test file". No directory layout solves that. A glob in .claude/rules/ does.

Slash commands, skills, and plan mode are in Chapter 10.

The CLAUDE.md hierarchy

User, project, and directory are the three the exam leans on. CLAUDE.local.md suits personal-but-repo-specific preferences, like "I run this repo's tests with -k slow".

The key property is who sees the file. Project-level is for anything the team must follow. User-level is for your own preferences (commit verbosity, explain-before-code) and is never right for team conventions.

The diagnostic the exam tests most: a new engineer clones the repo and half the team's conventions don't apply for them. The conventions live in the tech lead's ~/.claude/CLAUDE.md, written as personal habit and never promoted. Engineers who copied a senior's dotfiles inherited them by accident. The fix is always to move the content from ~/.claude/CLAUDE.md to the project file. Not more docs, not an onboarding checklist. "Works for me but not for the team" means this misplacement.

Directory-level handles conventions for one subtree. A monorepo keeps org-wide conventions at root, DB access patterns in services/billing/CLAUDE.md, and component conventions in apps/web/CLAUDE.md. Each loads only when work touches its subtree. It activates on location.

Modular organization: @import and .claude/rules/

Two separate tools for breaking up a monolithic CLAUDE.md. The exam tests them as distinct facts.

@import pulls an external file into a CLAUDE.md by path. The package maintainer picks which standards apply:

# services/billing/CLAUDE.md

This service handles payment processing and refund issuance.

@docs/standards/python-style.md
@docs/standards/api-design.md

Refunds above $500 require the process_refund tool's manual-review path;
see the runbook in docs/refunds.md before touching that code path.

A Terraform-only package imports terraform-conventions.md instead and never loads the Python guide. The judgement sits with whoever knows the package.

• AGENTS.md interop: Claude Code reads CLAUDE.md, not AGENTS.md. Use @AGENTS.md inside a CLAUDE.md, or a symlink, rather than maintaining two drifting files.
• claudeMdExcludes (settings.json, any layer, arrays merge) excludes CLAUDE.md or rules files by glob on absolute paths. Use it to stop a subpackage inheriting a rule without moving the rule file.

.claude/rules/ splits one file doing too many jobs into topic files (testing.md, api-conventions.md, deployment.md), each small enough to own and review.

.claude/
  CLAUDE.md              # thin: project overview, links to rules
  rules/
    testing.md           # pytest conventions, fixture policy
    api-conventions.md   # REST error shapes, versioning policy
    deployment.md         # rollout steps, rollback procedure

Without frontmatter these always load. It is just organisation. With paths frontmatter (below), they load conditionally.

Diagnosing with /memory and /context

Instructions come from several levels, imports, and rule files, so "why did Claude do that?" needs tooling:

Exam answer: /memory verifies which memory files are loaded and diagnoses inconsistent behaviour across sessions. That is the guide's wording for task 3.1, and /context appears nowhere in the guide.

Current Claude Code differs. /memory now lists and opens CLAUDE.md and rules locations (including ones not yet created) and toggles auto memory. The docs point to /context for what actually loaded this session. In your own work, check /context first, then use /memory to open the file and find out why (wrong level, stale @import path, non-matching glob). On the exam, answer /memory.

Path-specific rules: scoping by file type, not by location

A .claude/rules/ file with YAML frontmatter paths: (a list of globs) loads only when the file being edited matches.

---
paths: ["terraform/**/*"]
---

# Terraform conventions

All resources must be tagged with `team` and `cost-center`. Modules under
terraform/modules/ are versioned by git tag, not by branch — never reference
a module by branch name in a `source` argument. Run `terraform fmt` before
every commit; CI will reject unformatted plans.

Mechanics:

• Rules are discovered recursively (.claude/rules/frontend/testing.md works) and can be symlinked in, so one rule can be shared across repos.
• ~/.claude/rules/ applies to all your projects and loads before the project's .claude/rules/. The project rule wins on conflict.
• Brace expansion works (**/*.{ts,tsx}), capped at 1000 expanded patterns and 4 MiB.

That Terraform example is directory-shaped, so a directory CLAUDE.md could also express it. The case only a glob can express is file type scattered across locations, which is the exam's sample question 6:

Button.tsx sits next to Button.test.tsx, and useAuth.ts next to useAuth.test.ts, across 40 component directories. You want one test rule: use the custom render wrapper, follow arrange-act-assert comments, mock the API client uniformly.

• Root CLAUDE.md: the rule applies to components and tests alike, written as an "if this is a test file" aside. The model has to infer scope from prose.
• CLAUDE.md in each directory: 40 copies, no single source of truth, and the rule is still keyed to address rather than suffix.
• Glob rule: scopes on the filename shape.

---
paths: ["**/*.test.tsx", "**/*.test.ts"]
---

# Frontend test conventions

Use `renderWithProviders` from src/test-utils, never the raw `render` export
from the testing library — our components depend on theme and auth context
that raw render does not supply. Structure test bodies as arrange, act,
assert, with a blank line between each section. Mock the API client via
`mockApiClient()` from src/test-utils; do not construct ad hoc fetch mocks.

Editing any *.test.tsx anywhere loads it.

Token economy (the second half of 3.3): conditional loading keeps Terraform rules out of context while you edit React, and the reverse. A comprehensive root CLAUDE.md carries everything on every turn.

Distractors for the scattered-tests scenario:

• "Consolidate into root CLAUDE.md with a testing section." Relies on the model inferring scope from prose. There is no mechanical match, so the advice can leak into non-test files or be missed in a long file.
• "Put it in a skill." Skills are invoked on demand, by command or by the model's judgement. A convention that must apply automatically every time can't depend on invocation.

Path rules are automatic (unlike skills) and location-agnostic (unlike CLAUDE.md). That combination is why they exist.

Choosing between subdirectory CLAUDE.md and glob rules

If the boundary is a place in the tree, use a subdirectory CLAUDE.md. If it is a kind of file that appears in many places, use a glob rule.

They compose: a thin root CLAUDE.md for orientation and org policy, directory files for the few location-bound conventions, and glob rules for type-bound ones (tests, migrations, generated-code markers, IaC).

Auto memory: a second system, distinct from CLAUDE.md

Auto memory is written by Claude, not you. It lives under ~/.claude/projects/<project>/memory/, is indexed by MEMORY.md, and loads at every session start in that project. Entries are typed (user, feedback, project, reference), capped around 200 lines / 25 KB.

• CLAUDE.md: conventions you guarantee, reviewed and versioned.
• Auto memory: what Claude learned about how you work (terse commits, a refactor already rejected, who owns deploys).

If Claude "remembers" something never written in any CLAUDE.md, that's auto memory, not a hierarchy level you missed.

What the exam tests

• 3.1 levels and paths: user, project, directory (plus managed above, CLAUDE.local.md below project).
• 3.1 diagnostic: new team member missing instructions because they sit in ~/.claude/CLAUDE.md. Fix: promote to the project file.
• @import ≠ .claude/rules/: @import selectively pulls standards files in, chosen by the maintainer. rules/ splits a monolith into topic files.
• /memory verifies which memory files are loaded (the guide's answer, even though current Claude Code uses /context for this).
• 3.3 discriminator: subdirectory CLAUDE.md is directory-bound (where the file lives). paths glob rules are type-bound (what the file is). For scattered conventions (interleaved tests, migrations across services), pick the glob and reject root consolidation (inference, not matching) and skills (invocation, not automatic).
• Secondary benefit: path scoping keeps irrelevant rules out of context and off the token budget.

Exercises

1. Run /context in a repo you maintain. Trace each loaded file to its level and to any @import or glob rule, and explain every surprise.
2. Split a repo's conventions (testing, API errors, deployment) into .claude/rules/ files, and give at least one a realistic paths glob (**/*.test.ts, **/*.tf, **/migrations/*.py).
3. Write one scenario where a subdirectory CLAUDE.md is right and one where it isn't, using the location-vs-type test. Write the glob rule for the second.
4. Put a team convention in ~/.claude/CLAUDE.md, confirm with /context in a fresh clone or scratch home that it's absent, then promote it to the project file and confirm it appears.

Chapter 10 — Extending Claude Code

Two halves (task statements 3.2 and 3.4). Slash commands and skills make a prompt durable and shareable: a file, version-controlled, identical every time. Plan mode decides whether a task needs a design phase before any edit. You've used both. The exam wants the exact mechanism (file, directory, frontmatter field) and the ability to pick the right one from a task description.

Slash commands: where the file lives decides who gets it

A custom slash command is a markdown file whose body becomes the prompt, and its filename is the invocation. There is no registration step.

"/review must run the same way for every developer on clone": the answer is .claude/commands/. Distractors:

• Put it in CLAUDE.md. CLAUDE.md carries context and standards, not invocable actions.
• A registration file listing commands. None is needed. The file's presence is the registration.

Personal shorthand that shouldn't reach teammates goes in ~/.claude/commands/.

.claude/commands/review.md          # project-scoped — every teammate gets /review
~/.claude/commands/my-triage.md     # user-scoped — only you get /my-triage

Skills: commands that carry configuration

A skill is a command plus frontmatter that Claude Code enforces around the invocation. Current Claude Code treats them as one mechanism: .claude/commands/foo.md and .claude/skills/foo/SKILL.md both give /foo, and a skill is the superset.

Locations and precedence:

• Enterprise/managed: org-controlled, applies to everyone.
• Personal: ~/.claude/skills/<name>/SKILL.md.
• Project: .claude/skills/<name>/SKILL.md, shared via git.
• Plugin: <plugin>/skills/, invoked as plugin:skill to avoid name collisions.
• Nested project dirs: apps/web/.claude/skills/deploy/ is invoked as /apps/web:deploy, so frontend and backend deploy skills can coexist in a monorepo.

The three exam fields:

• Caveat on allowed-tools, for real use and not the exam: current docs define it as tools usable without asking permission, a pre-approval, not a fence. The field that removes tools is disallowed-tools. On the exam, allowed-tools restricts tools. In your repos, use disallowed-tools for the guarantee.
• context: fork is the mechanism that keeps a skill's internal noise out of your transcript. You get the conclusion without the scaffolding.

---
name: analyze-refund-patterns
description: Scans recent process_refund calls for anomalies and reports a summary.
context: fork
allowed-tools: Read, Grep, Glob
argument-hint: "[date-range, e.g. 2026-08-01..2026-08-31]"
---

The body after the frontmatter is plain markdown. Skills are not a templating language.

Who may invoke a skill:

• disable-model-invocation: only a human can trigger it, never the model on its own judgement. Use it for side effects like /deploy.
• user-invocable: false: hidden from the command list, so the model invokes it automatically. Use it for background-knowledge skills.
• paths frontmatter: same glob scoping as .claude/rules/. The skill is considered for auto-invocation only when the current file matches.

Other frontmatter exists and isn't tested: when_to_use, arguments/$name, model, effort, background, agent, hooks, shell, metadata, license, compatibility.

Bundled skills (/doctor, /code-review, /run, /verify, /loop, /debug, …) ship out of the box. disableBundledSkills and skillOverrides turn them off or replace them.

Personal variants without stepping on teammates

To get different behaviour from a shared skill, create a second skill under a different name in ~/.claude/skills/. The project has .claude/skills/extract-invoice/SKILL.md, and you want currency normalisation too, so create ~/.claude/skills/extract-invoice-mine/SKILL.md. The shared skill is untouched and teammates are unaffected. The exam's answer is always rename-and-relocate, never an in-place edit of the shared file.

Skills versus CLAUDE.md

The test is not importance but whether the content applies to every interaction or only to one nameable task:

• CLAUDE.md: universal standards loaded every turn, e.g. the commit message convention.
• Skill: an on-demand, task-specific workflow, e.g. auditing process_refund call sites for compliance. Loading that every turn is pure context cost.

"Should apply always, to everyone, unasked" is a CLAUDE.md question dressed as a skills question.

Plan mode versus direct execution

The exam tests whether you can tell from the task description, before any exploration, whether plan mode was needed.

Plan mode gives safe exploration plus a human-reviewed design before any file changes. It prevents rework from committing to the wrong shape of change. Use it when there are several legitimate approaches and a wide blast radius. Canonical cases:

• Monolith → microservices.
• Migrating 45 files to a new library version.
• Choosing between integration approaches with different infrastructure requirements.

Direct execution fits tasks with one reasonable shape: a single-file fix where the stack trace names the line, or adding one validation conditional. Planning those produces a plan saying "add the null check". That is friction with no safety gain.

Timing (the point people get backwards): the complexity is stated in the request. "Restructure into microservices" announces it in the first sentence. The distractor is starting in direct execution "to see how far you get" and switching to plan mode once things get messy. By then you've made irreversible choices across many files. Choose plan mode at the outset.

Combine them. Plan the migration once (target API, order, special-case call sites), then directly execute the 45 edits. The plan is a durable spec the execution phase consumes. Treating it as a single either/or choice for the whole task is the trap.

The Explore subagent

Investigation can consume the context budget the rest of the task needs: mapping 45 call sites, surveying service boundaries, dead ends. The Explore subagent does the verbose discovery in its own isolated context and returns a summary. It is the same move as context: fork (Chapter 5 covers subagent mechanics). Use it in the investigation phase of a multi-phase task so exploration doesn't exhaust context before you act. A plan that had to read all 45 files in-session leaves no room to discuss it.

What the exam tests

• 3.2 placement: command or skill for every developer on clone goes in .claude/commands/ or .claude/skills/ (in git). A personal variant goes in ~/ under a new name, never as an edit to the shared file.
• Frontmatter by scenario: noisy analysis or brainstorming means context: fork. Must not destroy anything means scoped allowed-tools. Invoked with no parameters means argument-hint.
• Skill vs CLAUDE.md: always-loaded standards go in CLAUDE.md, on-demand workflows in a skill, regardless of importance.
• 3.4: complexity stated up front (microservices, 45-file migration, integration choice) means plan mode at the outset, not midway. Single-file fix or one conditional means direct execution.
• Combined: plan for investigation, direct execution for implementation. The Explore subagent keeps verbose discovery from exhausting context.

Exercises

1. Write .claude/skills/audit-refund-flow/SKILL.md (checks process_refund call sites for missing identity verification) with context: fork and read-only allowed-tools. Then make a date-range version with argument-hint. Confirm the forked exploration never appears in your transcript.
2. Before starting a real refactor, write one sentence on whether its complexity is declared up front. If it is, plan first and compare the plan with what direct execution would have done for the first few files.
3. Copy a team skill under a new name into ~/.claude/skills/ and modify it. Confirm the original still behaves as teammates expect.
4. On a small multi-call-site change, plan across all call sites, then execute. Note where the plan was wrong and whether an Explore pass first would have caught it.

Chapter 11 — Claude Code in CI

Interactive Claude Code waits for you when it's stuck or needs permission. In a pipeline nobody answers, so the job hangs until the runner times out. This chapter (task statement 3.6) covers what makes Claude Code a CI participant: a flag that never waits for input, flags for machine-parseable output, and CLAUDE.md as the way to tell an unattended instance what "good" looks like.

Running example: a PR review bot that checks a diff against team standards and posts inline comments. Secondary example: test generation for changed code.

The flag that prevents the hang: -p

-p / --print: non-interactive, single-shot. It takes the prompt, does the work, writes to stdout, and exits. No follow-up questions, no blocking on stdin. This is sample question 10: "a CI job hangs waiting for interactive input — what fixes it?"

claude -p "Review the diff between origin/main and HEAD for correctness issues, \
security concerns, and violations of the standards in CLAUDE.md" \
  --output-format json \
  --json-schema "$(cat ./ci/review-schema.json)"

Reject by name:

• CLAUDE_HEADLESS: doesn't exist.
• --batch: doesn't exist.
• < /dev/null: a generic Unix trick. It doesn't use Claude Code's non-interactive contract (write stdout, exit).

For GitHub Actions specifically, the official anthropics/claude-code-action wraps headless claude -p. It takes claude_args (e.g. --json-schema) and exposes structured_output as an Actions output. It is more idiomatic than hand-rolled subprocess calls, and the code below is what it does under the hood.

Structured findings: --output-format json and --json-schema

The consumer is a script that posts inline comments at file and line, or decides pass/fail. Parsing prose breaks as soon as a word changes.

• --output-format json: output is JSON.
• --json-schema: constrains that JSON to your schema (fields, types, required), so the bot can deserialise into a known type.

Use them together. An option naming --output-format json alone, or regex-parsing prose, is a distractor.

Two mechanical gotchas the CLI enforces (the guide doesn't mention them):

1. --json-schema takes the schema inline as a JSON string, not a file path. --json-schema ./ci/review-schema.json fails with --json-schema is not valid JSON. Keep the file in git, but pass its contents: "$(cat ./ci/review-schema.json)".
2. Response shape. --output-format json returns a single JSON object (result, session_id, usage, total_cost_usd, and structured_output when a schema is passed). Read json.loads(stdout)["structured_output"]. The array of events (system/assistant/user/result) belongs to --output-format stream-json, a newline-delimited stream for live progress (token-level needs --verbose --include-partial-messages). Treating the json output as an array gives a TypeError.

Finding schema:

{
  "type": "object",
  "properties": {
    "findings": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "file": { "type": "string" },
          "line": { "type": "integer" },
          "severity": { "type": "string", "enum": ["blocking", "warning", "nit"] },
          "message": { "type": "string" },
          "issueId": { "type": "string" }
        },
        "required": ["file", "line", "severity", "message", "issueId"]
      }
    }
  },
  "required": ["findings"]
}

Build in issueId from the start, because it's what makes duplicate-free re-review possible (below). Chapter 13 covers schema design in general.

Python

import json
import subprocess

def claude_structured(prompt: str, schema_path: str) -> dict:
    """Run Claude Code non-interactively and return the structured payload."""
    schema = open(schema_path).read()
    completed = subprocess.run(
        ["claude", "-p", prompt, "--output-format", "json", "--json-schema", schema],
        capture_output=True, text=True, check=True,
    )
    return json.loads(completed.stdout)["structured_output"]

def run_review(schema_path: str, base_ref: str = "origin/main") -> dict:
    return claude_structured(
        f"Review the diff between {base_ref} and HEAD against the standards "
        "in CLAUDE.md. Report only issues you are confident about.",
        schema_path,
    )

def post_inline_comments(findings: dict, pr_number: int) -> None:
    for finding in findings["findings"]:
        github_client.create_review_comment(
            pr_number,
            path=finding["file"],
            line=finding["line"],
            body=f"**{finding['severity']}**: {finding['message']}",
        )

TypeScript

import { execFile } from "node:child_process";
import { promises as fs } from "node:fs";
import { promisify } from "node:util";

const execFileAsync = promisify(execFile);

async function claudeStructured<T>(prompt: string, schemaPath: string): Promise<T> {
  const schema = await fs.readFile(schemaPath, "utf-8");
  const { stdout } = await execFileAsync("claude", [
    "-p", prompt, "--output-format", "json", "--json-schema", schema,
  ]);
  const response = JSON.parse(stdout) as { structured_output: T };
  return response.structured_output;
}

interface Finding {
  file: string;
  line: number;
  severity: "blocking" | "warning" | "nit";
  message: string;
  issueId: string;
}

async function runReview(schemaPath: string, baseRef = "origin/main"): Promise<{ findings: Finding[] }> {
  return claudeStructured(
    `Review the diff between ${baseRef} and HEAD against the standards in CLAUDE.md. ` +
      "Report only issues you are confident about.",
    schemaPath,
  );
}

async function postInlineComments(findings: { findings: Finding[] }, prNumber: number) {
  for (const finding of findings.findings) {
    await githubClient.createReviewComment(prNumber, {
      path: finding.file,
      line: finding.line,
      body: `**${finding.severity}**: ${finding.message}`,
    });
  }
}

"Report only issues you are confident about" is in the prompt because minimising false positives is a stated goal of this scenario. A bot that comments on every quibble gets muted within a week. The schema constrains the shape; the prompt and CLAUDE.md constrain what's worth saying.

Non-interactive execution: --bare, permissions, and no one left to ask

• --bare is recommended for CI and scripting. It skips auto-discovery of hooks, skills, commands, subagents, plugins, MCP servers, CLAUDE.md, and auto memory, for determinism across runners. The tension: CLAUDE.md doesn't load either. If you need both reproducibility and team standards, supply them explicitly, e.g. --append-system-prompt-file.

• Permission prompts stall a pipeline like unanswered stdin. Settle them up front:

  • --allowedTools pre-approves specific tools.
  • --permission-mode (auto, dontAsk, acceptEdits) sets the default for everything else.

  A read-only review bot needs little. A test-generation pass must have the write tool pre-approved.

CLAUDE.md as the CI context mechanism

A CI instance has no memory of your interactive sessions. It doesn't know "no magic numbers" or "use Fixtures.customer()". CLAUDE.md is the mechanism for supplying testing standards, fixture conventions, and review criteria, loaded the same as in interactive sessions (without --bare).

Write specifics, not platitudes. "Follow best practices" adds nothing. "Flag any process_refund call without an idempotency key" encodes what you've been burned by.

Test generation: documenting what a valuable test looks like

Asking for "tests for this" produces lots of low-value tests: restating the type checker, testing getters. The Code Generation scenario's fix: document testing standards, valuable-test criteria, and available fixtures in CLAUDE.md. For example:

• One test per distinct business rule, not per code path.
• Constructor and trivial-getter tests aren't valuable.
• Use the Fixtures module, not inline literals.
• Refund-window and partial-refund edge cases are always worth covering. Description-field formatting variants are not.

Second lever: include existing test files in context, not just the changed source. Otherwise the pass can't know test_refund_after_window_closed exists and will propose a near-duplicate.

Python

def generate_tests_for_pr(changed_file: str, existing_test_file: str, schema_path: str) -> dict:
    existing_tests = open(existing_test_file).read()
    return claude_structured(
        f"Generate tests for the changes in {changed_file}. "
        "Testing standards and valuable-test criteria are in CLAUDE.md. "
        "Here are the existing tests for this module — do not propose scenarios "
        f"already covered:\n\n{existing_tests}",
        schema_path,
    )

TypeScript

async function generateTestsForPr(
  changedFile: string, existingTestFile: string, schemaPath: string,
): Promise<{ tests: unknown[] }> {
  const existingTests = await fs.readFile(existingTestFile, "utf-8");
  return claudeStructured(
    `Generate tests for the changes in ${changedFile}. ` +
      "Testing standards and valuable-test criteria are in CLAUDE.md. " +
      "Here are the existing tests for this module — do not propose scenarios " +
      `already covered:\n\n${existingTests}`,
    schemaPath,
  );
}

Why the session that wrote the code is a poor judge of it

Having the session that just wrote the code review it seems efficient. It works worse than an independent instance, and the exam asks why:

• While writing, the session made many small calls (skip this unlikely edge case, this name is close enough, this shortcut is fine), each reasonable in context.
• Reviewing its own work, it grades decisions it just rationalised. Its context carries momentum toward "this is fine", and self-consistency makes it stick with earlier judgements rather than reopen them.
• A fresh claude -p reads the diff cold against CLAUDE.md with no investment in those choices, so it is better placed to catch the skipped null check or the misused fixture.

The CI pipeline should run a separate review step. "Have the generating session review it" is the trap. Chapter 15 develops the general principle.

Avoiding duplicate comments across re-reviews

A bot that re-runs on every push reposts the same comments, because a fresh claude -p has no memory of the last run. Fix: pass the prior findings in the prompt and instruct it to report only new or still-unaddressed issues. The bot's own state (a file, the comment thread, a DB row) holds the last run's issueIds.

Python

def run_incremental_review(schema_path: str, prior_findings: list[dict], base_ref: str) -> dict:
    prior_summary = json.dumps(prior_findings)
    return claude_structured(
        f"Review the diff between {base_ref} and HEAD against CLAUDE.md standards. "
        f"Here are findings from the previous review of this PR: {prior_summary}. "
        "Report only issues that are new since that review, or that were previously "
        "flagged and remain unaddressed in the current diff. Reuse the same issueId "
        "for a still-unaddressed issue rather than minting a new one.",
        schema_path,
    )

TypeScript

async function runIncrementalReview(
  schemaPath: string, priorFindings: Finding[], baseRef: string,
): Promise<{ findings: Finding[] }> {
  return claudeStructured(
    `Review the diff between ${baseRef} and HEAD against CLAUDE.md standards. ` +
      `Here are findings from the previous review of this PR: ${JSON.stringify(priorFindings)}. ` +
      "Report only issues that are new since that review, or that were previously " +
      "flagged and remain unaddressed in the current diff. Reuse the same issueId " +
      "for a still-unaddressed issue rather than minting a new one.",
    schemaPath,
  );
}

Rejected alternatives:

• Don't re-run after new commits. You lose the safety net exactly when a fix introduces a new problem.
• Diff raw comment text to suppress repeats. Fragile, because slight rewording breaks the match.

What the exam tests

• -p / --print is the only fix for a CI hang on interactive input. CLAUDE_HEADLESS and --batch are invented. /dev/null doesn't address the command contract. This is sample question 10.
• --output-format json + --json-schema together give structured findings for inline PR comments. The payload is one object with structured_output. The event array is stream-json.
• CLAUDE.md supplies testing standards, fixture conventions, and review criteria to CI runs. --bare skips its auto-loading, so "reproducible and must follow CLAUDE.md" means supplying it explicitly.
• --permission-mode / --allowedTools pre-clear actions so the pipeline doesn't stall on a permission prompt.
• Independent review instance beats self-review. Be able to explain momentum and self-consistency, not just state the conclusion.
• Avoid duplicate work, as a pair: prior findings plus "report only new or unaddressed" for re-review; existing test files in context for generation.

Exercises

1. Add a claude -p review step to a GitHub Actions workflow on every PR, with --output-format json and your own --json-schema, posting each finding inline via the GitHub API. Confirm no hang on a large diff.
2. Write a specific CLAUDE.md review and testing section (real fixture names, real "not worth flagging" examples). Run the same diff with and without it and compare, including false positives that disappeared.
3. Run the bot, capture issueIds, fix one issue and leave one, push, and re-run with prior findings. The fixed one should drop off and the other persist with the same id.
4. Have a session review its own nontrivial change to process_refund or extract_invoice, then run a fresh claude -p review of the same diff. Compare.

Chapter 12 — Prompt engineering

Every CLAUDE.md line you tightened because Claude kept getting it wrong was prompt engineering. The exam doesn't ask for eloquent prompts. It asks you to diagnose the failure and pick the matching fix.

The key distinction (tasks 4.1 and 4.2): explicit criteria fix a rule that was never written down. Few-shot examples fix a rule that is written down but applied inconsistently. Both add words to a prompt aimed at the same symptom, and they treat different diseases.

Task 3.5 (iterative refinement) is separate. It covers the process you run with Claude Code to converge on a working prompt or implementation, not what the prompt contains.

The running example: a code review bot with a trust problem

A review bot (the Code Generation scenario) has run on every PR for a month. About one comment in three is noise: style nits labelled "medium", or false alarms on patterns standard in this codebase. Developers stop reading it, including the comment that would have caught a real bug.

False positives in one category erode trust in every category. The reader can't tell categories apart, so trust is one shared reservoir and one leaky category drains it.

The instinct is "be conservative" or "only report high-confidence findings". That is a named trap. General appeals to caution don't improve precision compared with specific categorical criteria. The model was already guessing at an undefined boundary. "Be conservative" makes it guess less often, in both directions, in a ratio you didn't choose.

4.1 — When the rule itself does not exist

Write the rule down.

• Vague: "Check that comments are accurate." A fuzzy standard, judged differently on Monday and Friday.
• Explicit: "Flag comments only when claimed behavior contradicts actual code behavior." An operational test: read the claim, read the code, check for contradiction.

The general skill is to state which categories to report and which to skip, not to filter by confidence. Confidence is the model's self-assessment of an undefined question, and it is no more reliable than the question. Put bugs and security in scope, minor style and locally idiomatic patterns out, stated as rules about the code.

Python

REVIEW_SYSTEM_PROMPT = """
You are reviewing a pull request diff for get_customer, lookup_order, and
process_refund. Report an issue only if it falls into one of these categories:

- BUG: the code will produce an incorrect result or crash on some input that
  the diff's tests do not cover. Do not report a bug for behavior that matches
  the existing tests, even if you would have designed it differently.
- SECURITY: the change introduces an injection risk, a missing authorization
  check on a path that mutates customer data, or logs a secret or PII field.
- COMMENT_MISMATCH: a comment or docstring makes a factual claim about
  behavior that contradicts what the code actually does. Do not report a
  comment merely because it is vague, outdated in tone, or could be phrased
  better — only contradiction counts.

Do NOT report: formatting, naming conventions, import order, or any pattern
that already appears elsewhere in this file. If a pattern is already in the
codebase, it is a local convention, not a bug, even if you would write it
differently.
"""

TypeScript

const REVIEW_SYSTEM_PROMPT = `
You are reviewing a pull request diff for getCustomer, lookupOrder, and
processRefund. Report an issue only if it falls into one of these categories:

- BUG: the code will produce an incorrect result or crash on some input that
  the diff's tests do not cover. Do not report a bug for behavior that matches
  the existing tests, even if you would have designed it differently.
- SECURITY: the change introduces an injection risk, a missing authorization
  check on a path that mutates customer data, or logs a secret or PII field.
- COMMENT_MISMATCH: a comment or docstring makes a factual claim about
  behavior that contradicts what the code actually does. Do not report a
  comment merely because it is vague, outdated in tone, or could be phrased
  better -- only contradiction counts.

Do NOT report: formatting, naming conventions, import order, or any pattern
that already appears elsewhere in this file. If a pattern is already in the
codebase, it is a local convention, not a bug, even if you would write it
differently.
`;

Severity: if labels vary with no anchor, define each level with a concrete code example. "High: an unauthenticated endpoint returning another customer's order history, like lookup_order(order_id) with no ownership check" can be pattern-matched. "High: something important" can't.

Disable a collapsing category. When one category's false positives are destroying trust and you haven't found the right criteria yet, turn it off. Ship BUG + SECURITY only, disable COMMENT_MISMATCH, fix its prompt offline against a sample set, and re-enable it once its precision is defensible. A bot that's right 3/3 earns attention. One that's right 2/3 earns scepticism toward everything. The exam treats this as a legitimate first-class answer.

4.2 — When the rule exists but bends on hard cases

The criteria are in place and good. The bot gets the obvious cases right, but is inconsistent near the boundary. process_refund catches a broad exception and logs without re-raising: arguably a bug, arguably deliberate degradation. It's flagged on some runs and not others. The rule is defined and applied inconsistently where judgement is needed. Use few-shot examples.

Few-shot means 2–4 fully worked examples in the prompt: input, reasoning, output. It is the most effective technique for consistent, actionable output when prose keeps producing variation, because an example demonstrates the reasoning behind an ambiguous call, and the model can generalise that reasoning to novel cases. You aren't enumerating every pattern. You're showing how to weigh competing considerations.

Show the reasoning, not just the verdict.

• "diff X → no finding" teaches one answer.
• "diff X → broad catch swallows the exception without logging the customer-facing error path, no upstream handler visible, so a bug, not deliberate degradation → BUG, medium" teaches the test to run next time.

Examples without reasoning generalise worse, and this is the axis the exam probes.

Python

FEW_SHOT_EXAMPLES = """
Example 1 — ambiguous exception handling
Diff: process_refund catches `PaymentGatewayError` broadly, logs the message
at INFO level, and returns a generic success response to the caller.
Reasoning: swallowing a payment failure and reporting success to the caller
means a failed refund looks successful to the customer-facing code. This is
not a deliberate degrade-gracefully pattern (there's no fallback path here,
just silence), and the customer-facing consequence is a real correctness
issue, not a style choice.
Output: {"location": "process_refund:47", "issue": "Payment failure is
caught and logged but reported to the caller as success", "severity": "high",
"suggested_fix": "Return an error result when the gateway call fails instead
of a generic success payload"}

Example 2 — a broad catch that IS acceptable
Diff: lookup_order catches `Exception` around a metrics-emission call, logs
at DEBUG, and continues; the function's real return value is computed before
the try block and is unaffected either way.
Reasoning: the broad catch only protects a non-critical side effect
(metrics), and its failure cannot alter the function's actual return value.
This matches the existing pattern used around every other metrics call in
this file, so it's a local convention, not a bug.
Output: no finding.
"""

TypeScript

const FEW_SHOT_EXAMPLES = `
Example 1 -- ambiguous exception handling
Diff: processRefund catches PaymentGatewayError broadly, logs the message at
INFO level, and returns a generic success response to the caller.
Reasoning: swallowing a payment failure and reporting success to the caller
means a failed refund looks successful to the customer-facing code. This is
not a deliberate degrade-gracefully pattern (there's no fallback path here,
just silence), and the customer-facing consequence is a real correctness
issue, not a style choice.
Output: {"location": "processRefund:47", "issue": "Payment failure is caught
and logged but reported to the caller as success", "severity": "high",
"suggestedFix": "Return an error result when the gateway call fails instead
of a generic success payload"}

Example 2 -- a broad catch that IS acceptable
Diff: lookupOrder catches a generic error around a metrics-emission call,
logs at DEBUG, and continues; the function's real return value is computed
before the try block and is unaffected either way.
Reasoning: the broad catch only protects a non-critical side effect
(metrics), and its failure cannot alter the function's actual return value.
This matches the existing pattern used around every other metrics call in
this file, so it's a local convention, not a bug.
Output: no finding.
`;

Named uses of few-shot:

1. Ambiguous cases, shown with reasoning (above). The guide's examples: tool selection for ambiguous requests (below) and branch-level test coverage gaps (is an untested else branch worth flagging?).
2. Distinguishing acceptable patterns from real issues. The pair above deliberately shows one real issue and one lookalike that's fine. Positives alone teach surface matching ("broad catch = flag").
3. Output format. Location, issue, severity, fix, in order. The model copies a shown structure far more reliably than it follows a described one.
4. Reducing hallucination in extraction from informal measurements and varied document structures. extract_invoice will fabricate total_amount when the total appears only as "call it forty-five hundred, even" in a footnote, unless it has seen that handled. Give 2–3 worked extractions covering real variants (line-item table with a totals block, total only in prose, implied discount), including absent field → null, don't invent. The same applies to inline citations vs a bibliography, or methodology up front vs embedded in results.

Tool selection works the same way. "Where's my package" is clearly lookup_order. "Third time my order's been late, I want someone to look at my account" splits between lookup_order and escalate_to_human. An example reasoning that the repeated history and the word "someone" point to escalation, and that a lookup first only delays a human, teaches the weighing so it transfers to different wording.

Telling 4.1 and 4.2 apart under exam pressure

Does the boundary exist, or does it exist but wobble on hard cases?

• No one wrote down what counts: 4.1. Few-shot is wrong here, because there is no correct behaviour to demonstrate yet.
• Criteria are stated and agreed, trouble is on judgement calls: 4.2. Rewording the criteria won't help, because they were never the missing ingredient.

"Be conservative" fits neither column. It's neither a criterion nor an example, and it is offered as a distractor in scenarios that need one of them.

3.5 — Iterative refinement as a developer discipline

How you converge with Claude Code, turn by turn. Four techniques:

1. Concrete input/output examples when prose fails. "Normalise dates, handling the three legacy formats" gets read inconsistently. 2–3 exact input → output pairs pin it down. It's the same mechanism as few-shot, used to communicate a spec during development. Named case: data-migration edge cases. Instead of "handle nulls sensibly", give customer_id: null, order_total: 45.00 → route to the quarantine table, don't insert with a fabricated ID.
2. Test-driven iteration. Write tests first (expected behaviour, edge cases, performance requirements), then iterate by feeding Claude Code the test failures. A failing test is a precise, falsifiable statement of the gap, and every red test replaces a round of "no, I meant…".
3. The interview pattern. Have Claude Code ask you questions first, as a competent engineer would before coding. It is most valuable in unfamiliar domains, where it surfaces things you wouldn't think to specify: cache invalidation strategy for a caching layer, failure modes of a job queue you've never run. It's the clarifying question in plan mode, invoked deliberately and up front.
4. Single message vs sequential: do the problems interact?
   • Independent (a typo here, an unrelated off-by-one there): fix sequentially. Bundling adds nothing and makes each fix harder to verify.
   • Interacting (a caching bug plus an invalidation-ordering bug that only makes sense given the first): one detailed message. Fixed separately, the first fix can invalidate your description of the second or make it worse. Example: a migration's null-handling fix depends on where the transaction boundary lands.

Context awareness: a newer lever for long-running prompts

On Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5, Claude automatically receives <budget:token_budget> and <system_warning>Token usage: X/Y remaining</system_warning> as a conversation runs. So "as you approach your budget, stop exploring and write up what you have" now works without you computing the number.

It does not replace Chapter 14's case-facts block or manifest. Budget awareness paces work against a shrinking window. Those patterns decide which exact numbers and conclusions must survive compaction. A model that wraps up gracefully at 90% can still lose a dollar amount to summarisation.

Models older than Opus 4.7 don't get this automatically. They need an explicit task budget via a beta flag, stated in the prompt.

What the exam tests

• 4.1/4.2 as a diagnostic pair. The top trap is "be more conservative" or "raise the confidence threshold". Treat any general appeal to caution as wrong.
• Missing boundary (4.1): write criteria, define severity with concrete examples, or temporarily disable a collapsing category.
• Shaky on ambiguous cases (4.2): 2–4 few-shot examples showing reasoning, distinguishing acceptable from real issues, demonstrating output format.
• Extraction under 4.2: varied document structure and null-field fabrication are few-shot problems once the schema is correct. Schema problems are Chapter 13.
• 3.5: concrete I/O examples after prose has failed; test-first with failures as feedback; interview pattern for unfamiliar domains; single message vs sequential depends on interaction.

Exercises

1. Write a vague review prompt ("check for bugs and be careful about false positives"). Run it on a diff with one real bug and two borderline patterns. Rewrite it with explicit report/skip categories and confirm the borderline verdicts moved (the 4.1 effect).
2. Add two reasoning-bearing few-shot examples (one flag, one not). Test on a third, unseen borderline pattern and check that the model generalises the reasoning (the 4.2 effect).
3. Run an ambiguous transformation (real-world dates, or mapping requests to lookup_order/escalate_to_human) three times from prose alone, then three times with 2–3 I/O examples. Compare the variance.
4. Build interacting problems: a rate limiter with a counter race plus a window off-by-one, where fixing the boundary moves the race. Report both in one message, then sequentially in a fresh session. If the sequential first fix has to be partly undone, you've reproduced the 3.5 failure.

Chapter 13 — Structured output

When output goes to a program rather than a person, "write a careful prompt, ask for JSON, hope" isn't good enough. Running example: extract_invoice, which returns vendor, line items, taxes, and total for an accounts-payable pipeline.

It fails in two different ways, with different fixes:

• Syntax: malformed JSON (dangling comma, unescaped quote in "Smith & Sons"). Fixed by schema and tool use.
• Semantics: well-formed JSON that is wrong (line items sum to $412.50, total says $391.00). Fixed by validation.

Using a stricter schema for a semantic error, or validation for a syntax error, is a misdiagnosis. The second half of the chapter covers volume: the Message Batches API.

Tool use as the extraction mechanism

A tool_use block's input conforms to the tool's input_schema. For extraction, the "tool" does nothing. extract_invoice touches no database. It exists so its schema becomes the contract, and tool_use.input arrives as a parsed object.

This is the most reliable way to get schema-compliant output. Asking for JSON in prose means validating a string after the fact, and a dropped brace shows up at json.loads, far from the cause. With tool use there's no string to parse, so malformed JSON is structurally impossible, not just less likely.

Python

extract_invoice = {
    "name": "extract_invoice",
    "description": (
        "Extract structured data from an invoice document: vendor identity, "
        "line items, tax, and totals. Use this on any invoice, receipt, or "
        "billing statement provided as text or an attached document. Do not "
        "use this on purchase orders or shipping manifests, which have no "
        "total to reconcile."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "vendor_name": {"type": "string"},
            "invoice_number": {"type": ["string", "null"]},
            "invoice_date": {
                "type": ["string", "null"],
                "description": "ISO 8601 date, e.g. 2026-03-14. Null if unreadable.",
            },
            "line_items": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "description": {"type": "string"},
                        "quantity": {"type": "number"},
                        "unit_price": {"type": "number"},
                        "line_total": {"type": "number"},
                    },
                    "required": ["description", "quantity", "unit_price", "line_total"],
                },
            },
            "tax_amount": {"type": ["number", "null"]},
            "stated_total": {
                "type": "number",
                "description": "The total as printed on the invoice.",
            },
            "calculated_total": {
                "type": "number",
                "description": (
                    "Sum of line_total across line_items, plus tax_amount. "
                    "Compute this yourself; do not copy stated_total."
                ),
            },
            "conflict_detected": {
                "type": "boolean",
                "description": "True if calculated_total and stated_total differ by more than $0.01.",
            },
            "category": {
                "type": "string",
                "enum": ["goods", "services", "subscription", "other"],
            },
            "category_detail": {
                "type": ["string", "null"],
                "description": "Required when category is 'other'; free-text description.",
            },
        },
        "required": [
            "vendor_name", "line_items", "stated_total",
            "calculated_total", "conflict_detected", "category",
        ],
    },
}

TypeScript

const extractInvoice: Anthropic.Tool = {
  name: "extract_invoice",
  description: [
    "Extract structured data from an invoice document: vendor identity, line",
    "items, tax, and totals. Use this on any invoice, receipt, or billing",
    "statement provided as text or an attached document. Do not use this on",
    "purchase orders or shipping manifests, which have no total to reconcile.",
  ].join("\n"),
  input_schema: {
    type: "object",
    properties: {
      vendor_name: { type: "string" },
      invoice_number: { type: ["string", "null"] },
      invoice_date: {
        type: ["string", "null"],
        description: "ISO 8601 date, e.g. 2026-03-14. Null if unreadable.",
      },
      line_items: {
        type: "array",
        items: {
          type: "object",
          properties: {
            description: { type: "string" },
            quantity: { type: "number" },
            unit_price: { type: "number" },
            line_total: { type: "number" },
          },
          required: ["description", "quantity", "unit_price", "line_total"],
        },
      },
      tax_amount: { type: ["number", "null"] },
      stated_total: {
        type: "number",
        description: "The total as printed on the invoice.",
      },
      calculated_total: {
        type: "number",
        description:
          "Sum of line_total across line_items, plus tax_amount. Compute this yourself; do not copy stated_total.",
      },
      conflict_detected: {
        type: "boolean",
        description: "True if calculated_total and stated_total differ by more than $0.01.",
      },
      category: {
        type: "string",
        enum: ["goods", "services", "subscription", "other"],
      },
      category_detail: {
        type: ["string", "null"],
        description: "Required when category is 'other'; free-text description.",
      },
    },
    required: [
      "vendor_name", "line_items", "stated_total",
      "calculated_total", "conflict_detected", "category",
    ],
  },
};

Reading the result:

Python

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    tools=[extract_invoice],
    tool_choice={"type": "tool", "name": "extract_invoice"},
    messages=[{"role": "user", "content": invoice_text}],
)
invoice_block = next(b for b in response.content if b.type == "tool_use")
invoice_data = invoice_block.input  # already a parsed dict, schema-shaped

TypeScript

const response = await client.messages.create({
  model: "claude-sonnet-4-5",
  max_tokens: 1024,
  tools: [extractInvoice],
  tool_choice: { type: "tool", name: "extract_invoice" },
  messages: [{ role: "user", content: invoiceText }],
});
const invoiceBlock = response.content.find(
  (b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
);
const invoiceData = invoiceBlock!.input; // already parsed, schema-shaped

Two more mechanisms: output_config.format and strict tool use

• output_config.format with {"type": "json_schema", "schema": {...}} is dedicated structured output with no tools involved. Constrained decoding guarantees a match, and the JSON arrives in a plain text block. It is the more direct fit when there's no natural "tool", e.g. classifying a ticket into fixed fields.
• strict: true on a tool definition. tool_choice forcing guarantees that a call happens. Without strict, the arguments are reliable but not guaranteed. strict applies constrained decoding to the arguments, so extract_invoice should carry it.

The two are independent and combine: output_config.format shapes the answer, strict shapes tool calls, and one request can use both (e.g. lookup-then-report).

Python

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": invoice_text}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "vendor_name": {"type": "string"},
                    "stated_total": {"type": "number"},
                },
                "required": ["vendor_name", "stated_total"],
                "additionalProperties": False,
            },
        }
    },
)
invoice_data = json.loads(response.content[0].text)

TypeScript

const response = await client.messages.create({
  model: "claude-sonnet-4-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: invoiceText }],
  output_config: {
    format: {
      type: "json_schema",
      schema: {
        type: "object",
        properties: {
          vendor_name: { type: "string" },
          stated_total: { type: "number" },
        },
        required: ["vendor_name", "stated_total"],
        additionalProperties: false,
      },
    },
  },
});
const invoiceData = JSON.parse(response.content[0].text);

Supported JSON Schema subset (both mechanisms):

• No numeric bounds (minimum, maximum, multipleOf).
• No string length bounds.
• additionalProperties must be false.
• No recursive schemas.
• Only a limited regex dialect for pattern.

Nullable ["string", "null"] and enum-plus-detail (below) still work. Move range and length checks into your own code.

Grammar compilation cost: the first call with a schema pays compile latency. The grammar is cached for 24 h from last use. Changing the schema invalidates it, and also invalidates prompt caching. Keep the schema fixed and put per-request variation in the prompt text.

Choosing how the tool gets called

• "any": a mixed stream (invoices, POs, manifests). Claude must pick some extraction tool, classifying implicitly.
• Forced: force extract_invoice first when enrich_vendor_record needs vendor_name, or flag_for_review needs conflict_detected. Then return to "auto".

Schema design decisions that matter more than they look

• Required fields invite fabrication. Require invoice_number and a receipt that lacks one will sometimes get an invented value. Make it nullable (["string", "null"]). An honest null trips your validation, while a fabrication sails through.
• Enum with an escape hatch. category ∈ "goods", "services", "subscription", "other", plus category_detail, required when "other". You get clean aggregation on common cases and honesty on the rest.
• "unclear" for judgement enums (paid/unpaid/disputed), so genuine ambiguity has a legitimate answer instead of a forced wrong guess.
• Format normalisation goes in the prompt, not the schema. A string field accepts 03/14/2026, 14 Mar 2026, and 2026-03-14 alike. "Return dates as ISO 8601; if ambiguous, infer the most likely reading" does what no pattern can.

Syntax versus semantics — the load-bearing distinction

Schema compliance tells you the output parses, not that it's correct. The $412.50 vs $391.00 invoice is fully schema-valid. Structure says nothing about arithmetic, provenance, or internal consistency.

The exam's worked example: have the model do the arithmetic as a checkable field.

• stated_total: the number printed on the document.
• calculated_total: "sum of line_total across line_items, plus tax_amount. Compute this yourself; do not copy stated_total."

The model has to add up the lines instead of transcribing a headline, and your code diffs the two. A difference of more than a cent goes to a human before AP.

conflict_detected: a boolean the model sets for internal inconsistency (totals that disagree, two dates for one event, a name spelled two ways). It turns "the model might notice" into "the pipeline will notice".

Triage rule:

Validation, retry, and feedback loops

The tested skill is knowing which failures retry can fix:

• Retriable: the information is in the document and the first pass got it wrong (an omitted field, a dropped line item, a miscalculated total).
• Not retriable: the information isn't in the source (e.g. a PO in a system you never provided). Attempt five has no more access than attempt one, and an unguarded loop burns requests forever. Route to a human, or to a step that fetches the missing document.

Classify the failure before retrying. When you retry, send the original document + the failed extraction + the specific validation error, so the model fixes a named mistake.

Python

def retry_extraction(client, document_text, failed_extraction, validation_error):
    retry_prompt = (
        f"The previous extraction of this invoice failed validation:\n\n"
        f"{validation_error}\n\n"
        f"Previous extraction:\n{json.dumps(failed_extraction, indent=2)}\n\n"
        f"Original invoice:\n{document_text}\n\n"
        f"Re-extract, correcting the specific issue above. Recompute "
        f"calculated_total from line_items rather than reusing the prior value."
    )
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        tools=[extract_invoice],
        tool_choice={"type": "tool", "name": "extract_invoice"},
        messages=[{"role": "user", "content": retry_prompt}],
    )
    return next(b for b in response.content if b.type == "tool_use").input

TypeScript

async function retryExtraction(
  client: Anthropic,
  documentText: string,
  failedExtraction: unknown,
  validationError: string,
): Promise<unknown> {
  const retryPrompt = [
    `The previous extraction of this invoice failed validation:`,
    ``,
    validationError,
    ``,
    `Previous extraction:`,
    JSON.stringify(failedExtraction, null, 2),
    ``,
    `Original invoice:`,
    documentText,
    ``,
    `Re-extract, correcting the specific issue above. Recompute`,
    `calculated_total from line_items rather than reusing the prior value.`,
  ].join("\n");

  const response = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    tools: [extractInvoice],
    tool_choice: { type: "tool", name: "extract_invoice" },
    messages: [{ role: "user", content: retryPrompt }],
  });
  const block = response.content.find(
    (b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
  );
  return block!.input;
}

A bare retry of the unchanged prompt usually reproduces the same output. The error message is what does the work.

detected_pattern: for code-scanning agents, a field naming the construct that triggered each finding. When developers dismiss findings, group dismissals by detected_pattern to see exactly which construct over-triggers. Same idea as calculated_total: make the model's reasoning a queryable field.

Designing for batch volume

Message Batches API (task 4.5):

• 50% cheaper, flat, not volume-tiered.
• Processed within up to 24 hours, with no SLA for anything faster. Design for the slow case.
• Not for multi-turn tool calling (see below).
• Every request carries a custom_id.

The deciding question is latency tolerance, not cost: is anyone blocked waiting?

• Fits: overnight tech-debt report, weekly compliance audit, nightly test generation, nightly invoice reconciliation. The consumer checks a dashboard tomorrow.
• Doesn't fit: a pre-merge check. A developer can't wait an unknown 20 minutes to 24 hours.

The SLA arithmetic

Commitment: a 30-hour SLA, batching on a fixed schedule.

• Worst case = wait until the next submission + up to 24 h of processing.
• 30 − 24 = 6 h of budget for the wait.
• Every 6 h gives a 30 h worst case, exactly at the ceiling with no margin.
• Every 4 h gives 28 h worst case, a 2 h buffer. That is the answer.

Default rule: subtract the window from the SLA, then choose an interval comfortably inside the remainder, not at the boundary.

What a batch request looks like, and what does not fit inside one

A batch is a list of ordinary Messages requests, each with your custom_id. Submit, poll, and read results, which come back in any order, so match on custom_id, never position.

Python

from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

batch_requests = [
    Request(
        custom_id=f"invoice-{invoice['id']}",
        params=MessageCreateParamsNonStreaming(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            tools=[extract_invoice],
            tool_choice={"type": "tool", "name": "extract_invoice"},
            messages=[{"role": "user", "content": invoice["text"]}],
        ),
    )
    for invoice in todays_invoices
]

batch = client.messages.batches.create(requests=batch_requests)

# Later, after polling batch.processing_status until "ended":
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        block = next(b for b in result.result.message.content if b.type == "tool_use")
        store_invoice(custom_id=result.custom_id, data=block.input)
    else:
        queue_for_resubmission(custom_id=result.custom_id, failure=result.result)

TypeScript

const batchRequests = todaysInvoices.map((invoice) => ({
  custom_id: `invoice-${invoice.id}`,
  params: {
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    tools: [extractInvoice],
    tool_choice: { type: "tool" as const, name: "extract_invoice" },
    messages: [{ role: "user" as const, content: invoice.text }],
  },
}));

const batch = await client.messages.batches.create({ requests: batchRequests });

// Later, after polling batch.processing_status until "ended":
for await (const result of client.messages.batches.results(batch.id)) {
  if (result.result.type === "succeeded") {
    const block = result.result.message.content.find(
      (b): b is Anthropic.ToolUseBlock => b.type === "tool_use",
    );
    storeInvoice(result.custom_id, block!.input);
  } else {
    queueForResubmission(result.custom_id, result.result);
  }
}

A batch request does not support multi-turn tool calling. Each entry is one call. It can return tool_use, but nothing executes the tool and feeds the result back. This breaks designs quietly rather than throwing an error.

• Fits: invoice extraction, because extract_invoice is never executed. The tool block is just the vehicle for structured output.
• Doesn't fit: look up a vendor mid-extraction, decide whether to flag, then extract. That loop needs multi-turn execution. Run it synchronously, or split it into a batch-friendly first pass plus a later step.

Handling failures without resubmitting everything

• Results come back errored, canceled, or expired for some fraction. Resubmit only the failed custom_ids, never the whole batch.
• Fix, don't just retry: if failures share a cause (oversized scans exceeding context), chunk those documents before resending.
• Refine on a sample first: run 50–100 invoices synchronously and iterate until first-pass success is high, then batch the full volume. Discovering a date-format gap after a day-long batch costs the day plus the redo.

What the exam tests

• 4.3: guaranteed-parseable output needs tool_use + JSON schema, which eliminates syntax errors as a category. tool_choice: "any" when the doc type is unknown, forced when it's known or must precede enrichment.
• 4.4: syntax vs semantics. Malformed is a schema problem. Well-formed but inconsistent is a validation problem. Be able to reproduce calculated_total vs stated_total + conflict_detected from memory.
• 4.4 retry classification: format and arithmetic errors are retriable. Information absent from the source is not, so route to a human.
• 4.5 arithmetic: SLA minus 24 h window, then an interval comfortably inside the remainder (30 h gives every 4 h).
• 4.5: "call a tool, see the result, decide next" can't run in a batch request. Resubmit only failed custom_ids.

Exercises

1. Build extract_invoice. Run five invoices with inconsistent totals and confirm conflict_detected flags all five. Then simulate raw-text JSON corruption and note that tool use eliminates it.
2. Write a retry harness sending document + failed extraction + specific error. Test an arithmetic slip (fixable) and a field truly absent from the document (not fixable, whatever the attempt count).
3. Redo the SLA arithmetic for a 20-hour SLA with a 24 h window. Can any cadence satisfy it? Explain why not once the SLA is shorter than the window.
4. Batch ten invoices including one malformed document. Poll, iterate by custom_id, and resubmit only the failure.

Chapter 14 — Context engineering

Two hours into a session, answers soften ("typically follows the pattern used elsewhere") and after /compact a decision from 20 minutes ago has to be repeated. The cause is the same in both cases: the messages array is finite, everything the agent knows lives in it, and neither growth nor compression is free.

This chapter covers two problems that look alike. Mixing up their fixes is the mistake to avoid.

A case-facts block doesn't help a codebase crawl, and a manifest doesn't help a refund amount survive summarisation.

5.1 — Preserving facts inside one long conversation

Three failures compound:

1. Progressive summarisation destroys specifics. Paraphrase keeps the gist and loses numbers. "Customer disputed a charge and we discussed a refund" is a faithful summary of "disputed a $284.17 charge on SO-88213, promised a refund by the 15th, second contact". The amount, order ID, and date are gone. That is what summarisation does, not a bug.
2. Lost in the middle. Models attend reliably to the start and end of long input and worse to the middle. A fact stated once mid-conversation is at higher risk of being ignored. This is structural, not something an upgrade patches.
3. Tool results cost tokens by size, not relevance. A realistic lookup_order returns 40+ fields (SKUs, routing codes, tax flags, three address blocks). A return needs five: order id, item, amount, ship date, status. Nothing removes an appended result, so three lookups bury the signal under a hundred-plus irrelevant fields, reprocessed every turn.

Constraint: the API is stateless, so you must pass the complete history every request. Dropping earlier turns to save tokens doesn't leave the model with a rough understanding. It structurally cannot know what it doesn't have.

On Claude 4.5+, running out of room mid-generation gives stop_reason: "model_context_window_exceeded" (older models reject the request outright). Handle it separately from max_tokens. The effect looks similar, but the cause is different.

The case-facts block

The fix is architectural: extract transactional facts (amounts, dates, order numbers, statuses, customer-stated expectations) into a structured block that you include in full in every prompt, outside summarised history. It is the per-conversation equivalent of CLAUDE.md sitting in system rather than messages.

Python

from dataclasses import dataclass, field
from typing import Optional

@dataclass
class CaseFacts:
    """Durable, non-summarizable facts for one support case."""
    customer_id: Optional[str] = None
    tier: Optional[str] = None
    issues: list[dict] = field(default_factory=list)
    # each issue: {order_id, amount_cents, status, promised_date, stated_by_customer}

    def render(self) -> str:
        lines = [f"CASE FACTS (verified, do not paraphrase):",
                  f"customer_id={self.customer_id} tier={self.tier}"]
        for i, issue in enumerate(self.issues, start=1):
            lines.append(
                f"  issue {i}: order={issue['order_id']} "
                f"amount=${issue['amount_cents']/100:.2f} "
                f"status={issue['status']} "
                f"promised_by={issue.get('promised_date', 'n/a')}"
            )
        return "\n".join(lines)


def build_request(case_facts: CaseFacts, messages: list, user_turn: str) -> dict:
    # The facts block rides in `system`, alongside the persistent instructions,
    # so /compact and any summarization pass over `messages` never touch it.
    return {
        "model": "claude-sonnet-4-5",
        "system": f"{SUPPORT_SYSTEM_PROMPT}\n\n{case_facts.render()}",
        "messages": messages + [{"role": "user", "content": user_turn}],
    }

TypeScript

interface Issue {
  orderId: string;
  amountCents: number;
  status: string;
  promisedDate?: string;
}

interface CaseFacts {
  customerId?: string;
  tier?: string;
  issues: Issue[];
}

function renderCaseFacts(facts: CaseFacts): string {
  const lines = [
    "CASE FACTS (verified, do not paraphrase):",
    `customer_id=${facts.customerId} tier=${facts.tier}`,
  ];
  facts.issues.forEach((issue, i) => {
    lines.push(
      `  issue ${i + 1}: order=${issue.orderId} ` +
        `amount=$${(issue.amountCents / 100).toFixed(2)} ` +
        `status=${issue.status} promised_by=${issue.promisedDate ?? "n/a"}`,
    );
  });
  return lines.join("\n");
}

function buildRequest(facts: CaseFacts, messages: Anthropic.MessageParam[], userTurn: string) {
  return {
    model: "claude-sonnet-4-5",
    system: `${SUPPORT_SYSTEM_PROMPT}\n\n${renderCaseFacts(facts)}`,
    messages: [...messages, { role: "user" as const, content: userTurn }],
  };
}

Two design choices matter:

• Rebuilt each request from your application state, not produced by asking the model to summarise itself. The point is that it never passes through lossy compression.
• A separate layer per issue in multi-issue sessions. A billing dispute plus a later shipping complaint each keep their own order id, amount, and status. In one paragraph, the second issue's amount can quietly borrow the first's. Structured issue data lets the agent answer "status of the shipping issue?" without wading through the billing discussion.

Trimming tool output at the source

The case-facts block protects facts after extraction. Tool bloat happens earlier, when lookup_order returns. Trim before appending. The tool boundary is a design decision: the function the agent calls projects the full record down to what the task needs.

Python

RETURN_RELEVANT_FIELDS = {"order_id", "item", "amount_cents", "ship_date", "status"}

def lookup_order(order_id: str) -> dict:
    full_record = fulfillment_backend.get_order(order_id)  # 40+ fields
    return {k: v for k, v in full_record.items() if k in RETURN_RELEVANT_FIELDS}

TypeScript

const RETURN_RELEVANT_FIELDS = new Set([
  "orderId", "item", "amountCents", "shipDate", "status",
]);

async function lookupOrder(orderId: string): Promise<Record<string, unknown>> {
  const fullRecord = await fulfillmentBackend.getOrder(orderId); // 40+ fields
  return Object.fromEntries(
    Object.entries(fullRecord).filter(([key]) => RETURN_RELEVANT_FIELDS.has(key)),
  );
}

Prevention beats cleanup. Once fields are in several tool_results, only a destructive rewrite removes them, and that rewrite is the summarisation that damages numbers.

When you don't own the tool (a third-party MCP server), use the context editing API (beta context-management-2025-06-27) as a fallback:

• context_management: {"edits": [{"type": "clear_tool_uses_20250919", ...}]} clears old tool results by rule.
• Knobs: trigger (when), keep (recent results to leave), clear_at_least (minimum to reclaim), exclude_tools (never clear), clear_tool_inputs (also strip the call arguments).
• clear_thinking_20251015 does the same for old thinking blocks.

Trimming at the source is still better when you own the tool.

Position matters even after you have the facts right

When bundling several tool results plus a question, put the key findings summary first, then supporting detail under explicit section headers, so the middle has anchors to navigate back to. It costs nothing, and it matters most when results are short and you're tempted to concatenate them in call order.

KEY FINDINGS
- Order SO-88213: shipped 19 days ago, within 30-day return window.
- Customer has one prior return on this account (SO-77410, approved).
- Policy: standard return, no manager approval required.

## Order Detail
...

## Account History
...

## Applicable Policy Section
...

Requiring metadata and structure from subagents

A subagent returning verbose prose with no provenance leaves the consumer unable to tell which claim came from where, when, or how confident, and it wastes context re-deriving structure. Put a contract on the output shape:

• Every result carries dates, source locations, and methodological context.
• When the consumer's budget is limited (normal in multi-agent pipelines), return key facts, citations, and relevance scores instead of verbose content and reasoning chains.

Python

@dataclass
class SubagentFinding:
    claim: str
    source: str          # file path, ticket id, or document name
    as_of_date: str       # when this fact was true / last verified
    relevance: float      # 0-1, how directly this bears on the question asked
    # deliberately no "reasoning" field — the coordinator needs the conclusion,
    # not the chain that produced it

TypeScript

interface SubagentFinding {
  claim: string;
  source: string;      // file path, ticket id, or document name
  asOfDate: string;     // when this fact was true / last verified
  relevance: number;    // 0-1, how directly this bears on the question asked
  // deliberately no `reasoning` field — the coordinator needs the conclusion,
  // not the chain that produced it
}

The reasoning chain was necessary for the subagent and is nearly worthless to the coordinator. It costs tokens in proportion to thoroughness, which is backwards for a limited budget. Five paragraphs compress to one line that can sit beside four other subagents' findings.

5.4 — Context across a large codebase exploration

Here the task never fit in one window, so it's split across phases and subagents from the start. Named symptom: an hour in, asked about retries, the agent says "this typically follows a standard retry pattern with exponential backoff" instead of naming RetryableOrderProcessor at services/orders/retry.py:142, which it read 40 tool calls ago. It is reconstructing a generic answer because the specific one has degraded out of reach. The fact may still be in context, but the signal-to-noise ratio has collapsed. This is context degradation, not a reasoning failure a better prompt fixes.

Subagent delegation for verbose exploration

For "find every file touching the refund flow" or "trace what calls process_refund", spawn a subagent. Its context absorbs the dead ends and the coordinator's doesn't. It returns a short structured list, ideally in the metadata shape above.

The coordinator stays at Chapter 5's altitude: decide what to investigate, dispatch, reconcile. Reading files directly "just this once" pollutes exactly the context delegation protects.

Scratchpad files

Delegation protects the coordinator, but a long subagent trace can degrade too. A scratchpad file records key findings on disk as they're found, and the agent re-reads it rather than re-deriving. Write "refund cancellation → RefundCanceller.revert at services/orders/cancel.py:88, called from services/webhooks/stripe.py:210" the moment it's found, and read it back when needed.

# scratchpad: refund-flow-trace.md

## Confirmed call chain
- entry: `services/webhooks/stripe.py:210` (webhook handler)
- calls: `RefundCanceller.revert` at `services/orders/cancel.py:88`
- calls: `LedgerAdjustment.apply` at `services/ledger/adjust.py:41`

## Open questions
- Does `LedgerAdjustment.apply` retry on ledger-service timeout? Not yet checked.

## Files ruled out
- `services/orders/refund.py` — handles issuance, not cancellation. Not relevant here.

It's a curated record of conclusions, not a log (the transcript is already the log). It is the lost-in-the-middle countermeasure applied to a whole session: an external, always-current place to look.

Memory tool ({"type": "memory_20250818", "name": "memory"}): the supported version. It is client-executed and gives a persistent /memories directory across sessions with view, create, str_replace, insert, delete, and rename. Anthropic's "multisession software development" pattern (a progress log plus a feature checklist, read back on resume) is this chapter's scratchpad-plus-manifest. Build it yourself when you need a bespoke format or location.

Phase summaries before the next round of subagents

In a multi-phase exploration (map refund code paths, then audit each for a bug pattern), summarise phase one and inject that summary into each phase-two subagent's initial prompt. Phase two starts knowing the terrain instead of re-exploring it.

Manifests for crash recovery

Sessions get interrupted: the process dies, or context fills and needs a fresh start. Each agent exports its state to a known location as it works (conclusions, open items, scratchpad references). On resume, the coordinator loads a manifest (an index of what state exists where) and injects the relevant parts into fresh agents' prompts.

{
  "exploration_id": "refund-flow-audit-2026-09-03",
  "phase": 2,
  "phase_1_summary": "scratchpad/refund-flow-trace.md",
  "subagents_completed": ["trace-webhook-path", "trace-ledger-path"],
  "subagents_pending": ["audit-retry-behavior", "audit-idempotency"],
  "open_questions": [
    "Does LedgerAdjustment.apply retry on ledger-service timeout?"
  ]
}

A crash then costs minutes, not a rerun of phase one.

/compact inside an exploration, used deliberately

/compact is legitimate for reclaiming room filled with exploratory noise. The judgement is sequencing: compact after the phase summary is written and the scratchpad is current, so durable findings are already safe and only noise gets compressed. Compact before and you may lose the fact the summary was meant to capture.

Server-side compaction (beta compact-2026-01-12, context_management: {"edits": [{"type": "compact_20260112", "instructions": "..."}]}):

• instructions replaces the default summarisation prompt, e.g. "preserve all dollar amounts, order numbers, and dates verbatim". It complements the case-facts block and doesn't replace it, and it's only as good as your knowledge of what's at risk.
• pause_after_compaction stops after compacting so you can check what survived before the agent continues.

What the exam tests

• 5.1 stem: an amount, date, or order number goes vague after several turns. Diagnosis: progressive summarisation; the fix is a persistent case-facts block. Distractors: "summarise more carefully", "ask the model to double-check".
• 40-field tool result: trim at the source, before it enters context.
• Where the block lives: the guide says "included in each prompt, outside summarized history". Any answer keeping facts inside summarised history misses the point.
• 5.4 symptom: "typical patterns" instead of the specific class means context degradation, not something a better prompt fixes.
• Subagent for an isolated question, rather than the coordinator exploring directly.
• Manifest is for crash recovery. Scratchpad is for degradation within an ongoing session.
• /compact is legitimate. The tested judgement is compacting after findings are externalised.

Exercises

1. In a long Claude Code session, ask about something found 30+ tool calls ago (function name, path, config value). Note any hedged answer and its phrasing (look for "typically").
2. Design a case-facts block for one customer with two disputed charges and one shipping complaint. Decide what's persistent versus conversational, and write the block after all three are raised.
3. Take a tool returning a large record and write the trimmed projection for one task. Compare field counts and estimate the token difference over 20 turns.
4. Design a manifest for a three-phase audit (dependencies, dead code, coverage): what each phase writes, what the coordinator reads on resume, and what a phase-three subagent gets if the crash hit mid-phase-two.

Chapter 15 — Escalation, review, and provenance

Three problems at the point where a system produces an answer: who checks it (4.6), when the system should refuse to answer alone (5.2), and how humans review at scale and how evidence survives synthesis (5.5, 5.6). They share a temptation: let the model check itself, decide for itself when it's out of its depth, or let a summary drop the citations. They also share a villain: the model's opinion of itself. The last section resolves an apparent contradiction about confidence scores. If you remember one thing from this chapter, make it that.

Review architecture: why a second instance beats a second look

"Review the diff you just wrote" sometimes catches a typo but rarely questions a decision. The reason is structural. A model reviewing its own output in the same session keeps the reasoning context from generation. It isn't reading the code fresh. It is re-reading its own justification and stays biased toward it.

Extended thinking doesn't fix this. More reasoning helps generation, but the bias isn't too little effort. The reasoning already happened and now anchors the read. A human is likewise a worse reviewer of their own PR than a colleague who didn't watch it get written.

Fix: an independent instance whose context holds only the artifact (diff, extraction, plan) and none of the generator's reasoning. It catches subtle issues that both self-review instructions and extended thinking miss.

Python

async def generate_and_review(task_description: str) -> dict:
    generation = await claude.messages.create(
        model="claude-sonnet-4-5",
        messages=[{"role": "user", "content": task_description}],
    )
    code = extract_code(generation)

    # A fresh instance: no system prompt overlap with generation, no shared
    # conversation history, nothing but the artifact and a review brief.
    review = await claude.messages.create(
        model="claude-sonnet-4-5",
        messages=[{
            "role": "user",
            "content": (
                "Review the following code for correctness bugs, edge cases, "
                "and unstated assumptions. You have no knowledge of why any "
                "decision was made; judge only what is written.\n\n" + code
            ),
        }],
    )
    return {"code": code, "review": extract_text(review)}

TypeScript

async function generateAndReview(taskDescription: string) {
  const generation = await claude.messages.create({
    model: "claude-sonnet-4-5",
    messages: [{ role: "user", content: taskDescription }],
  });
  const code = extractCode(generation);

  const review = await claude.messages.create({
    model: "claude-sonnet-4-5",
    messages: [{
      role: "user",
      content:
        "Review the following code for correctness bugs, edge cases, and " +
        "unstated assumptions. You have no knowledge of why any decision " +
        "was made; judge only what is written.\n\n" + code,
    }],
  });
  return { code, review: extractText(review) };
}

Two more moves for large artifacts:

• Multi-pass review against attention dilution. One pass over 10 files spreads attention thin, and cross-file and single-file issues tangle into contradictory findings. Instead, do a per-file local pass for internal correctness plus a separate cross-file integration pass for data flow (does a return value match what callers assume, is a schema change honoured everywhere).
• Per-finding self-reported confidence as a routing signal. High confidence ("missing null check, throws on documented empty list") goes to the human first. Low ("might be a race depending on scheduling") goes to a follow-up pass or lower priority. It prioritises human attention. It does not decide correctness or block merges on its own. See the final section.

Escalation and ambiguity resolution: what actually earns a handoff

Escalating too often is expensive and useless. Escalating too rarely is dangerous. The exam's answer is a closed list of three valid triggers:

1. The customer explicitly asks for a human.
2. Policy has a gap the agent lacks authority to resolve.
3. The agent can't make meaningful progress after a real attempt.

Anything else ("felt hard", "seemed upset", "wasn't sure") misfires both ways: it escalates easy cases that looked complex, and grinds on hard cases where the model was confidently wrong.

Trigger 1 details:

• Explicit request: escalate immediately. No "let me just check one thing first". Investigating anyway treats the stated preference as an obstacle.
• Frustrated but no request: not a trigger. Acknowledge the frustration, offer to resolve if it's within capability, and escalate only if they reiterate. What counts is what they asked for, not what tone implied.

Trigger 2, policy gap. The policy covers price drops on your own site and says nothing about competitor price matching. "Policy doesn't forbid it" invents unapproved policy. "Not covered, so no" denies something the business might want. Both are wrong because the agent lacks authority either way. The silence is the trigger.

Trigger 3, no progress. It is measured by outcome (tools and reasonable alternatives tried and exhausted), not by a forecast before trying.

Two named invalid proxies:

• Sentiment. It doesn't correlate with complexity. An angry customer can have a trivial issue, and a calm one a case the agent shouldn't handle alone.
• Self-reported confidence. A model wrong about a hard case is often wrong about being wrong. The limitations that made the case hard also corrupt its self-assessment, so it says "confident" on exactly the cases it's likely to get wrong.

Fix: explicit escalation criteria in the system prompt, plus few-shot examples in both directions: a case that looks complex but resolves cleanly, and one that looks simple but hits a policy gap.

Python

ESCALATION_SYSTEM_PROMPT = """
Escalate to a human agent only when one of the following is true:
1. The customer explicitly asks to speak with a human agent.
2. The applicable policy is silent or ambiguous on the customer's specific
   request (not merely complex to apply).
3. You have attempted a resolution using the tools available and cannot
   make further progress.

Do not escalate based on how difficult the case seems, how frustrated the
customer sounds, or how confident you feel. Frustration without an explicit
request for a human should be met with acknowledgment and an offer to
resolve; escalate only if the customer repeats the request for a person.

Example — escalate:
  Customer: "I want a price match against a competitor's listing."
  Policy covers only same-site price drops. -> Escalate: policy gap.

Example — do not escalate:
  Customer (frustrated): "This is the third time I've had to explain this,
  just fix it!"
  Issue: a standard, in-policy refund. -> Acknowledge frustration, resolve
  the refund, do not escalate.
"""

TypeScript

const ESCALATION_SYSTEM_PROMPT = `
Escalate to a human agent only when one of the following is true:
1. The customer explicitly asks to speak with a human agent.
2. The applicable policy is silent or ambiguous on the customer's specific
   request (not merely complex to apply).
3. You have attempted a resolution using the tools available and cannot
   make further progress.

Do not escalate based on how difficult the case seems, how frustrated the
customer sounds, or how confident you feel. Frustration without an explicit
request for a human should be met with acknowledgment and an offer to
resolve; escalate only if the customer repeats the request for a person.

Example -- escalate:
  Customer: "I want a price match against a competitor's listing."
  Policy covers only same-site price drops. -> Escalate: policy gap.

Example -- do not escalate:
  Customer (frustrated): "This is the third time I've had to explain this,
  just fix it!"
  Issue: a standard, in-policy refund. -> Acknowledge frustration, resolve
  the refund, do not escalate.
`;

Multiple matches from get_customer: don't use a heuristic (most recent, closest name). It can silently act on the wrong customer's data, which is real harm near process_refund. Ask for an additional identifier (order number, billing zip) and disambiguate deterministically.

Human review workflows and confidence calibration

The aggregate-accuracy trap. "97% accurate" invites cutting human review. But 97% overall is consistent with 99.5% on the dominant invoice type and 60% on a rare type or a hard field (foreign-currency total, handwritten date, vendor-specific field). Automate on the aggregate and you remove oversight exactly where it's needed. Validate accuracy by document type and by field, and confirm it holds in every segment before reducing review on any.

Two ongoing mechanisms (not one-off checks):

• Stratified random sampling of high-confidence extractions, the ones that would otherwise go unseen. That's how you measure the true error rate on the bucket you aren't otherwise checking, and catch new error patterns early.
• Field-level confidence, calibrated on a labelled validation set. Per-field scores are checked against ground truth, and 0.9 should mean right about 90% of the time, not 65%. Once calibrated, route low-confidence fields and documents with contradictory sources to humans first, so limited reviewer time goes where errors are most likely.

Python

def route_for_review(extraction: dict, calibration: dict) -> str:
    """calibration maps confidence buckets to measured accuracy from a
    labeled validation set -- e.g. {0.9: 0.97, 0.8: 0.88, 0.7: 0.61}."""
    field_conf = extraction["fieldConfidence"]  # per-field raw score
    lowest = min(field_conf.values())
    measured_accuracy = calibration_lookup(calibration, lowest)

    if extraction.get("sourceContradictory"):
        return "human_review"          # contradictory source, regardless of score
    if measured_accuracy < 0.90:
        return "human_review"          # calibrated accuracy below threshold
    if is_in_stratified_sample(extraction["documentId"]):
        return "human_review"          # sampled even though it looks safe
    return "auto_accept"

TypeScript

function routeForReview(
  extraction: Extraction,
  calibration: Map<number, number>,
): "human_review" | "auto_accept" {
  const fieldConf = extraction.fieldConfidence;
  const lowest = Math.min(...Object.values(fieldConf));
  const measuredAccuracy = calibrationLookup(calibration, lowest);

  if (extraction.sourceContradictory) return "human_review";
  if (measuredAccuracy < 0.90) return "human_review";
  if (isInStratifiedSample(extraction.documentId)) return "human_review";
  return "auto_accept";
}

This confidence number is trustworthy where the escalation one wasn't, because it has been checked against reality. More in the final section.

Provenance in multi-source synthesis

Subagents gather from different sources and synthesis combines them. The failure is quiet: a right answer that can't say where it came from.

Why attribution gets lost: summarisation's job is to compress, and a citation looks like detail safe to drop, since the claim survives without it. After two or three passes, the report is full of confident, unattributed claims indistinguishable from verified ones.

Fix (structural): every subagent outputs claim-source mappings (claim + source URL or document name + supporting excerpt), and every downstream step, including synthesis, preserves and merges them. "Summarize these findings" invites loss. "Merge these findings; every claim must carry the source mapping it arrived with" makes a dropped citation a visible, checkable defect.

Python

class ClaimSource(TypedDict):
    claim: str
    sourceUrl: str
    excerpt: str
    publicationDate: str | None   # required when available -- see below

class SubagentFinding(TypedDict):
    topic: str
    claims: list[ClaimSource]

def synthesize(findings: list[SubagentFinding]) -> dict:
    # Merge, don't summarize away, the claim-source pairs.
    all_claims = [c for f in findings for c in f["claims"]]
    conflicts = detect_conflicting_values(all_claims)
    contested_claims = {c["claim"] for c in conflicts}
    return {
        "wellEstablished": [c for c in all_claims if c["claim"] not in contested_claims],
        "contested": conflicts,   # annotated, not resolved by picking one
        "allSources": all_claims,
    }

TypeScript

interface ClaimSource {
  claim: string;
  sourceUrl: string;
  excerpt: string;
  publicationDate?: string;
}

interface SubagentFinding {
  topic: string;
  claims: ClaimSource[];
}

function synthesize(findings: SubagentFinding[]) {
  const allClaims = findings.flatMap((f) => f.claims);
  const conflicts = detectConflictingValues(allClaims);
  const contestedClaims = new Set(conflicts.map((c) => c.claim));
  return {
    wellEstablished: allClaims.filter((c) => !contestedClaims.has(c.claim)),
    contested: conflicts,
    allSources: allClaims,
  };
}

Built-in citations, a different case: when Claude reads a source directly via a document content block with citations: {"enabled": true}, the API returns citation blocks automatically (cited text, document, location). Anthropic reports up to 15% better recall than prompt-based citation. Use it when Claude is reading the document. Keep ClaimSource for subagent handoffs, where what's passed is a prior finding with no document to cite into.

Conflicts: annotate, don't resolve. One source says 12% growth, another 9%. Both may be credible, measuring slightly different things or different times. Report both values with sources rather than picking the more reputable-sounding one. A labelled contested finding is useful. A confident one that quietly settled a disagreement hides a judgement call (compare Chapter 4's coverage gaps). Upstream document analysis should do the same: complete the analysis with both values marked as conflicting and pass them up, because the coordinator holds the full picture.

Temporal differences: figures that look contradictory may be measurements from different dates. Drop the dates and a normal trend reads as inconsistency. Require a publication or data-collection date on every claim (publicationDate) so synthesis can tell "sources disagree" from "the world changed". Structure the report with sections separating well-established from contested findings.

Render content types natively: financial figures as tables, news as prose, multi-part technical findings as lists. Keep the source's characterisation and methodology (survey, regulatory filing, estimate). It is part of the claim, not decoration.

The confidence contradiction, resolved

The chapter appears to contradict itself:

• Self-reported confidence is not a valid escalation trigger (5.2).
• Yet field-level confidence drives review routing (5.5), and review passes self-reporting confidence is good design (4.6).

The resolution is calibration.

• Uncalibrated self-report is the model's guess about itself, made by the system whose competence is in question. The case where it's most likely silently wrong is also where its self-assessment is most likely wrong, for the same reason. Two unreliable signals don't make a reliable one.
• Calibrated score: computed the same way, but checked against a labelled validation set and shown to track the true error rate (0.9 ≈ 90% correct on ground truth). It has stopped being an opinion and become a measurement, like any instrument verified before use.

Rule: never let a raw, unchecked self-report make an autonomous decision (to escalate, or that code is correct). Once calibrated against ground truth, a score is a legitimate routing input for human review.

What the exam tests

This chapter covers task statements 4.6, 5.2, 5.5 and 5.6.

• 4.6: independent instances beat self-review because retained reasoning context biases the generator, not because of effort. Extended thinking is no substitute. Multi-pass review is per-file plus cross-file integration.
• 5.2: the three triggers are a closed set. Sentiment and self-reported confidence are the plausible wrong answers. Explicit request (immediate) vs frustration (acknowledge, offer resolution). A silent policy means escalate the gap. Multiple matches means ask for another identifier.
• 5.5: aggregate accuracy masks segment failures. Validate per type and field, use stratified sampling of high-confidence items, and route on calibrated field-level confidence.
• 5.6: claim-source mappings survive synthesis. Conflicts are annotated, not resolved. Publication dates are required. Sections separate established from contested. Format matches content type.
• Above all: self-reported confidence is invalid as an autonomous trigger but valid as a calibrated routing input.

Exercises

1. Take code from one Claude Code session and have a fresh session with no shared history review it. Compare with the original session reviewing itself, and find one issue only the fresh instance caught.
2. Draft an escalation prompt for process_refund/escalate_to_human with few-shot examples for all three triggers plus negative examples (sentiment, confidence). Test an angry customer with a simple in-policy request and a calm one asking about something the policy never covers.
3. Build extract_invoice with per-field confidence and a 20-document labelled set. Compute accuracy per confidence bucket, decide whether the raw scores are calibrated, and choose a routing threshold.
4. Run a three-subagent research task with likely disagreement. Require claim-source mappings with dates, and have synthesis produce wellEstablished and contested sections. Rerun with "summarize the findings" and compare what survives.