Exactprep CCAR-F Questions

The failure is caused by missing evidence, not insufficient reasoning over the supplied evidence. A static prompt containing only changed files cannot reliably identify callers, configuration dependencies, generated interfaces, or indirect relationships located elsewhere in the repository. Asking Claude to reason more deeply cannot recover code that it was never given.

Option A converts the review into a bounded agentic loop. Claude can search for symbol references, read relevant callers, inspect type definitions, and follow newly discovered dependencies before validating a potential defect. Anthropic describes the Claude Code agentic loop as gathering context, taking action, verifying results, and repeating based on tool feedback. A turn limit preserves predictable cost and execution time.

Option C improves coverage but imposes an arbitrary two-hop boundary and depends on the accuracy of a precomputed graph, which may omit dynamic imports, reflection, generated code, configuration references, or language-specific call relationships. Option D creates duplicated context and aggregation complexity while still limiting each reviewer to predetermined dependents. Option B changes the reasoning instructions but supplies no mechanism for checking unchanged files. Tool-enabled, just-in-time retrieval is therefore the most adaptable and reliable architecture for cross-file review.

================

Stratified random sampling provides both an ongoing quality-control mechanism and an unbiased measurement framework. By reviewing a fixed proportion of high-confidence outputs across meaningful strata—such as document type, field, source format, and business risk—the organization can estimate the residual error rate, compare performance across releases, and discover failure patterns that were not anticipated when existing rules were designed.

Anthropic recommends measurable success criteria and evaluations that mirror the real-world task distribution, including edge cases. Evaluation volume and repeatability are important because improvements must be demonstrated empirically rather than inferred from isolated examples. ( ) A weekly sampling program creates a stable benchmark and allows confidence intervals, trend analysis, regression detection, and error-taxonomy updates.

Option A detects only disagreements between stochastic extractions. Two attempts may produce the same plausible but incorrect answer, so agreement is not equivalent to factual correctness. Option B addresses known patterns but will miss new failure modes and may generate excessive false positives. Option C actually lowers the review threshold in the wrong direction: documents between 70% and 85% would be treated as automated rather than reviewed if the routing rule remains “below threshold,” and changing thresholds alone does not measure high-confidence error prevalence.

Official references/topics: Evaluation Design; Representative Test Distributions; Continuous Reliability Measurement; Human-in-the-Loop Sampling.

The prompt’s conservative reporting policy is directly causing the low recall. Claude may discover a legitimate race condition during analysis but suppress it because it cannot satisfy the instruction to report only issues about which it is certain. Option C separates two objectives that should not be conflated: broad defect discovery and strict acceptance filtering.

Anthropic’s current code-review prompting guidance explicitly recommends reporting every issue, including uncertain or lower-severity findings, assigning confidence and severity metadata, and allowing a separate verification stage to filter them. This maximizes recall while retaining control over developer-facing noise.

Option A leaves the suppression policy intact, so additional examples cannot guarantee that discovered problems will be reported. Option B improves recall but relies on historical category-level suppression, which can discard genuine findings that happen to belong to noisy categories. Option D can improve analysis quality but does not remove the instruction responsible for withholding findings. A dedicated finding stage followed by an independently configurable verification or thresholding stage produces measurable recall and precision controls without forcing one model call to optimize competing objectives.

================

Option C is the strongest choice among the listed approaches because it moves structured data generation from unconstrained response text into a schema-governed interface. Prompt-only JSON instructions, regular-expression extraction, and corrective retries are probabilistic recovery mechanisms: they may reduce failures, but none guarantees that the first response satisfies the contract. Anthropic now provides two schema-constrained mechanisms: JSON Structured Outputs through output_config.format and strict tool use through a tool definition with strict: true. Strict tool use uses grammar-constrained sampling so the tool input conforms to the declared JSON Schema, including required properties and data types. Therefore, the production implementation represented by option C should define the extraction tool’s input_schema and enable strict mode. The application reads the tool_use block as the extracted record; it does not need to scrape prose or search for braces. Option A remains vulnerable to preambles or malformed output. Option B can select a JSON-looking substring but cannot establish schema validity. Option D adds latency and cost and can still fail repeatedly. The schema-constrained tool contract is consequently the most reliable architecture available in these options.

================