Prompt Engineering and Structured Output

What this domain tests

20% — roughly 12 of 60 questions, tied with Claude Code Configuration & Workflows for second-heaviest behind Agentic Architecture. Expect the same memorization pressure that runs through the whole CCAR-F exam, concentrated here in parameter values and field names: the three tool_choice modes and their exact semantics, the Message Batches API's numbers, the field names in a self-correcting extraction schema.

The mix:

• Recall items: what tool_choice "any" guarantees versus forced selection, the batch API's cost and window figures, what custom_id is for, which error class strict schemas eliminate.
• Judgment items: when few-shot examples beat more instructions, when a retry is pointless, when to batch versus call synchronously, why an independent reviewer beats a self-review prompt.

Only two of the six published scenarios name this domain: Scenario 5 (Claude Code for Continuous Integration) — designing prompts that give actionable feedback and minimize false positives — and Scenario 6 (Structured Data Extraction) — JSON-schema validation, accuracy, graceful edge-case handling. That asymmetry is worth planning around: 20% of items but only two scenario homes means on a four-scenario draw your prompting questions often arrive wrapped in a code-review frame rather than an extraction frame. Practice reading task 4.1 and 4.6 material through Scenario 5's eyes.

Also worth naming: several prompting topics candidates arrive expecting — XML tag structuring, prefilled assistant turns, stop sequences, extended-thinking configuration — are not in CCAR-F Domain 4's task statements. The one place extended thinking appears here is as a wrong answer: the guide says independent review instances beat "self-review instructions or extended thinking" for catching subtle issues.

Theory outline

1. Explicit criteria to improve precision and reduce false positives (Task 4.1)

The core claim, and it is counterintuitive enough that the exam builds distractors on it: confidence language does not improve precision. Instructions like "be conservative" or "only report high-confidence findings" fail to improve precision compared with specific categorical criteria.

The contrast the guide draws:

The explicit version gives a decidable test. The vague version leaves the model to guess where the bar sits, and the guess drifts between runs.

Why false positives matter more than they look. The guide is direct about the second-order damage: a category with a high false-positive rate undermines developer confidence in the categories that are accurate. Reviewers who learn to skim past noise skim past the real bug too. The tactical response is unintuitive but explicitly endorsed — temporarily disable the high-false-positive category to restore trust while you improve its prompt, rather than leaving it on and hoping people filter it.

Three named skills:

• Write specific review criteria defining which issues to report (bugs, security) versus skip (minor style, local patterns) — instead of relying on confidence-based filtering.
• Temporarily disable high-false-positive categories while their prompts improve.
• Define explicit severity criteria with concrete code examples for each severity level to get consistent classification. Severity words alone ("critical," "minor") are as underspecified as "be conservative."

Exam angle: when a stem describes developers ignoring or distrusting review output, the answer defines categorical criteria or turns off the noisy category. Options that ask the model to be more careful, more conservative, or more confident are the trap.

2. Few-shot prompting for consistency and quality (Task 4.2)

The guide's framing: few-shot examples are the most effective technique for achieving consistently formatted, actionable output when detailed instructions alone produce inconsistent results. Instructions describe the target; examples demonstrate it, and demonstration transfers structure more reliably than description.

Four jobs examples do here:

Generalization is the point, and it's a favorite exam distinction. Well-chosen examples let the model generalize judgment to novel patterns rather than matching only the cases you enumerated. That is why the guide prescribes 2–4 targeted examples for ambiguous scenarios that show the reasoning for why one action was chosen over plausible alternatives — the reasoning is what generalizes. An example that shows only input and output teaches a mapping; an example that shows why teaches a rule.

Varied document structures get their own mention: inline citations versus bibliographies, methodology sections versus embedded details. When required fields come back empty or null, the fix is adding few-shot examples showing correct extraction from documents with those varied formats — not making the instruction sterner.

Exam angle: "detailed instructions but inconsistent results" is the phrase pattern that points at few-shot. So is "empty or null for fields that are present in the document." Distractors offer longer instructions, stricter wording, or more schema constraints.

3. Enforcing structured output with tool use and JSON schemas (Task 4.3)

Tool use with JSON schemas is the most reliable approach for guaranteed schema-compliant structured output, and it eliminates JSON syntax errors — the model fills a validated parameter structure instead of typing JSON into prose. Define an extraction tool whose input schema is your target shape, then read the structured data out of the tool_use response.

The three tool_choice modes — memorize the exact semantics:

The "any" versus forced distinction is the item to be crisp on: "any" guarantees structure; forced guarantees which structure.

The limit that carries the most exam weight: strict JSON schemas via tool use eliminate syntax errors but do not prevent semantic errors — line items that don't sum to the stated total, values placed in the wrong field. A schema validates shape, never meaning. That gap is exactly what task 4.4's validation layer exists to close.

Schema design decisions:

• Optional (nullable) fields when source documents may not contain the information. This is the anti-fabrication mechanism: a required field is pressure to invent a value, and the model will find something to put there.
• Enum plus escape hatches: an "other" value paired with a detail string for extensible categorization, and an "unclear" value for genuinely ambiguous cases. Without them, every unanticipated case gets crammed into the nearest wrong enum.
• Format normalization rules in the prompt alongside the strict schema when source formatting is inconsistent. The schema says the field is a date; the prompt says how to turn "3/4/26" and "March 4th" into one.

4. Validation, retry, and feedback loops for extraction quality (Task 4.4)

Retry-with-error-feedback is the named pattern: on failure, send a follow-up request containing the original document, the failed extraction, and the specific validation errors, so the model can self-correct against concrete signal. A bare "try again" repeats the same failure.

When retry cannot work — the judgment item:

Recognizing the third row is what separates a candidate who understands the loop from one who memorized it. Retrying an impossible extraction burns tokens and, worse, pressures the model toward fabrication.

Two error species, two owners:

Self-correcting schema design — the guide's concrete techniques, worth memorizing as field names:

• Extract calculated_total alongside stated_total so a mismatch is visible in the output itself rather than discovered downstream.
• Add a conflict_detected boolean for inconsistent source data, so the model can report the conflict instead of silently picking a side.
• Add a detected_pattern field to structured findings, recording which code construct triggered each finding. When developers dismiss findings, that field turns a pile of individual dismissals into a systematic analysis of which pattern is generating false positives — closing the loop back to task 4.1.

5. Efficient batch processing strategies (Task 4.5)

Message Batches API — the numbers the exam asks for:

Fit test. Batch is right for non-blocking, latency-tolerant workloads: overnight reports, weekly audits, nightly test generation. It is wrong for blocking workflows — a pre-merge check that gates a pull request cannot wait an unbounded number of hours. Match the API to the workflow's latency requirement, not to its size.

The no-multi-turn-tool-calling limit deserves its own beat, because it interacts with task 4.3: an extraction that needs a single tool call to shape its output is fine in batch; an agentic flow that calls a tool, reads the result, and calls another cannot run inside one batch request.

Two operational skills:

• Submission frequency from an SLA. Work backwards from the guarantee. With up to 24 hours of processing and a 30-hour SLA to honor, submitting on a 4-hour window keeps worst-case turnaround (queue wait plus processing) inside the promise. Expect arithmetic of exactly this shape.
• Failure handling. Resubmit only the failed documents, identified by custom_id, with appropriate modifications — chunking documents that exceeded context limits, for instance. Resubmitting the whole batch pays twice for work that already succeeded.

And one cost discipline: refine the prompt on a sample set before batch-processing large volumes. First-pass success rate is the lever that decides how many expensive resubmission rounds you run.

6. Multi-instance and multi-pass review architectures (Task 4.6)

The self-review limitation. A model retains its reasoning context from generation, which makes it less likely to question its own decisions in the same session. The code looked right when it wrote it, and the reasoning that made it look right is still in the window.

The fix is structural, not instructional. Independent review instances — without the prior reasoning context — are more effective at catching subtle issues than self-review instructions or extended thinking. Note what is being ranked here: not "extended thinking is bad," but "a second, uncontaminated instance beats thinking harder inside a contaminated one." A second Claude instance reviews the generated code with no knowledge of why it was written that way, so it evaluates what is there rather than what was intended.

Multi-pass review. Split large reviews into per-file local analysis passes plus cross-file integration passes, avoiding attention dilution and contradictory findings. Local passes catch local bugs; a dedicated integration pass catches the renamed function still called under its old name three files away. (Domain 1, task 1.6, tests the same split as a decomposition strategy — it is worth points in two domains.)

Calibrated routing. Run verification passes where the model self-reports confidence alongside each finding, then route by that confidence — high-confidence findings post automatically, low-confidence ones go to a human. Read this carefully against task 4.1: self-reported confidence is useful for routing findings you already have; it is not a substitute for explicit criteria when the goal is to reduce false positives in the first place. The exam rewards holding both ideas at once.

Worked problems

Problem 1. An automated code-review bot reports so many spurious comment-accuracy issues that developers now ignore its output entirely, including its security findings. What is the most effective response?

A. Add "be conservative and only report high-confidence findings" to the review prompt B. Define explicit criteria — flag comments only when the claimed behavior contradicts actual code behavior — and temporarily disable the comment-accuracy category while its prompt is improved C. Ask the model to self-report a confidence score and hide findings below 0.8 D. Increase the review model's thinking budget

Approach: Two things need fixing: the noisy category and the collateral damage to trust in accurate categories. The guide is explicit that general confidence instructions fail to improve precision compared with specific categorical criteria, and it explicitly endorses temporarily disabling a high-false-positive category to restore trust.

Why the others fail: A is the named ineffective instruction, verbatim. C confuses confidence-based routing (a task 4.6 technique for findings you already trust) with criteria design; a self-reported score does not make the underlying judgment sharper. D spends compute on a specification problem — the model isn't failing to think, it's succeeding at an underspecified goal.

Answer: B

Problem 2. An extraction pipeline uses a strict JSON schema through tool use. Output is always valid JSON, yet finance keeps finding invoices where the line items don't add up to the stated total. What addresses this?

A. Tighten the JSON schema with stricter type constraints B. Switch tool_choice from "auto" to "any" C. Extract calculated_total alongside stated_total and validate the discrepancy, retrying with the specific validation error when they diverge D. Add more few-shot examples of correctly formatted invoices

Approach: The stem carefully rules out syntax problems — the JSON is always valid. Strict schemas via tool use eliminate syntax errors but do not prevent semantic errors, and "line items that don't sum to total" is the guide's own example of one. The named remedy is a self-correcting validation flow with both totals present so the mismatch is detectable.

Why the others fail: A constrains types; no type system detects that a sum is wrong. B guarantees a tool gets called, which is already happening. D would help if the failure were formatting or an unfamiliar layout, but the extractions are well-formed and wrong in arithmetic.

Answer: C

Problem 3. A document-processing system handles several document types with different extraction schemas, and the type is not known before the call. Occasionally the model replies with prose instead of extracted data. What setting guarantees structured output?

A. tool_choice: "auto" B. tool_choice: "any" C. Forced selection: {"type": "tool", "name": "extract_invoice"} D. Adding "always respond with JSON" to the system prompt

Approach: Two constraints — a tool must be called, and the model must pick which schema fits. That is precisely tool_choice "any": the model must call a tool but can choose which one. The guide names this exact case: multiple extraction schemas, unknown document type.

Why the others fail: A permits the prose responses that are the complaint. C forces one schema, so a purchase order gets processed as an invoice. D is a prompt instruction where a parameter-level guarantee exists — probabilistic where deterministic was available.

Answer: B

Problem 4. An extraction job repeatedly fails to populate a required "supplier_tax_id" field. Investigation shows the tax ID appears only in a separate registration document that is not sent with the invoice. What should the team do?

A. Retry with the validation error appended until the model produces the field B. Force the extraction tool with tool_choice so the field must be filled C. Add few-shot examples showing tax IDs extracted from invoices D. Recognize that retries cannot supply information absent from the source, and either provide the registration document or make the field optional/nullable

Approach: The guide's limit on retry: retries are ineffective when the required information is simply absent from the source document, as opposed to format or structural errors. The stem hands you that diagnosis outright. Keeping the field required also violates the anti-fabrication rule — required fields the source can't support push the model to invent values.

Why the others fail: A retries an impossible task and invites a hallucinated tax ID that passes validation. B makes the tool call mandatory, which does nothing about a value that doesn't exist and increases fabrication pressure. C teaches extraction of something that isn't in the document.

Answer: D

Problem 5. A team wants Claude to review every pull request before merge, and separately to generate regression tests for the full repository each night. Which API arrangement fits?

A. Synchronous API for pre-merge review; Message Batches API for the nightly test generation B. Batch API for both — the 50% cost saving applies to both workloads C. Synchronous API for both — batch cannot handle code D. Batch API for pre-merge review with a 30-minute polling loop; synchronous for nightly generation

Approach: Sort by whether the workload blocks. Pre-merge review gates a human waiting on a merge, and the batch API offers no guaranteed latency SLA with a window of up to 24 hours — unusable for a gate. Nightly test generation is the guide's own example of a latency-tolerant workload, where the 50% saving is free money.

Why the others fail: B puts a blocking check behind an unbounded window. C forfeits the discount on work that has all night to finish. D inverts both assignments, and polling does not shorten the processing window.

Answer: A

Problem 6. In a CI pipeline, one Claude session generates a feature and is then instructed, in the same session, to review its own code critically. Subtle issues keep reaching production. What is the most effective architectural change?

A. Increase the extended-thinking budget for the review step B. Strengthen the self-review instruction with a detailed checklist C. Run the review in an independent instance that has no access to the generation session's reasoning context D. Have the same session generate the code twice and compare the versions

Approach: The guide states that a model retains reasoning context from generation and is therefore less likely to question its own decisions, and that independent review instances are more effective at catching subtle issues than self-review instructions or extended thinking. Both plausible-sounding fixes are pre-emptively ranked below the structural one.

Why the others fail: A and B are the two options the guide explicitly ranks lower — more thinking and better instructions inside a session that is already anchored to its own reasoning. D produces two outputs from the same anchored context, so shared blind spots persist in both.

Answer: C

Exam traps

• Confidence language as a precision fix. "Be conservative," "only report high-confidence findings" — named as ineffective compared with specific categorical criteria. Define what to report and what to skip.
• Ignoring the trust spillover. A noisy category degrades confidence in accurate categories. Temporarily disabling it while its prompt improves is the endorsed move, not a cop-out.
• "any" versus forced tool_choice. "any" = must call a tool, model picks. Forced {"type": "tool", "name": "..."} = must call that tool. Unknown document type with multiple schemas → "any". Mandatory ordering → forced.
• "auto" as a guarantee. Under "auto" the model may return text instead of calling a tool. If a stem complains about occasional prose responses, "auto" is the cause.
• Believing schemas prevent semantic errors. They eliminate JSON syntax errors only. Sums that don't add up and values in the wrong field survive validation and need their own checks.
• Required fields for information the source may lack. Make them optional/nullable. Required fields are fabrication pressure. Same family: no "other" + detail string and no "unclear" enum value means every unanticipated case lands in a wrong category.
• Retrying an impossible extraction. Retry fixes format and structural errors. When the information is absent from the source, no retry count helps — provide the source or relax the field.
• More instructions where examples belong. "Detailed instructions produce inconsistent results" and "required fields come back empty despite being present" both point to few-shot examples — 2–4 targeted ones that show the reasoning, so judgment generalizes to novel patterns instead of matching enumerated cases.
• Batch for blocking workflows. 50% savings, up to 24 hours, no latency SLA. Overnight and weekly work, yes; pre-merge gates, no. And no multi-turn tool calling inside a single batch request.
• Resubmitting whole batches. Resubmit only the failures, located by custom_id, with modifications (chunk the documents that blew the context limit).
• Self-review inside the generating session. Extended thinking and sterner self-review instructions are both explicitly ranked below an independent review instance without the generator's reasoning context.
• One giant review pass. Large multi-file reviews need per-file local passes plus a separate cross-file integration pass; a single pass dilutes attention and yields contradictory findings.