All AIF-C01 Test Inside Amazon Web Services Questions

Retraining or pre-training is costly and unnecessary for just using company policies.

Fine-tuning adapts models but is inefficient for frequently changing company documents.

Retrieval-Augmented Generation (RAG) is the best approach — it retrieves relevant policy documents from a knowledge base and feeds them into the model context in real time, ensuring accurate and up-to-date responses.

???? Reference:

AWS Documentation – RAG with Amazon Bedrock

The verified answer is C. Faithfulness. The company is evaluating foundation models for a summarization task by using an LLM-as-a-judge evaluation job in Amazon Bedrock. AWS documentation states that with a model evaluation job that uses a judge model, Amazon Bedrock uses one LLM to score another model’s responses and provide an explanation of how it scored each prompt and response pair. AWS also states that Amazon Bedrock provides built-in metrics that can be selected for these judge-based evaluation jobs.

Among the answer choices, Faithfulness is the only valid built-in metric for an Amazon Bedrock LLM-as-a-judge model evaluation job. AWS API documentation lists the valid built-in metric names for LLM-as-a-judge evaluation jobs, including Correctness, Completeness, Faithfulness, Helpfulness, Coherence, Relevance, FollowingInstructions, ProfessionalStyleAndTone, and responsible AI metrics such as Harmfulness, Stereotyping, and Refusal. The same documentation identifies Summarization as a valid task type for model evaluation jobs. Therefore, Faithfulness is a valid built-in metric for evaluating whether generated summaries remain grounded in the source input rather than introducing unsupported claims.

Option A. Context relevance and option B. Context coverage are incorrect for this question because AWS lists those as metrics for knowledge base retrieval-only evaluation jobs, not general LLM-as-a-judge model evaluation of foundation model summarization output. These metrics evaluate retrieved context, not the quality of a summarization model’s generated response. Option D. RMSE is also incorrect because root mean square error is a regression metric. It is not a built-in LLM-as-a-judge metric for summarization in Amazon Bedrock.

Because the task is summarization and the evaluation job is LLM-as-a-judge, the correct built-in evaluation metric from the available choices is Faithfulness.

The problem involves a simple probability calculation that can be handled efficiently by straightforward mathematical rules and computations. Using machine learning techniques would introduce unnecessary complexity and operational overhead.

Option C (Correct): " Use code that will calculate probability by using simple rules and computations " : This is the correct answer because it directly solves the problem with minimal overhead, using basic probability rules.

Option A: " Use supervised learning to create a regression model " is incorrect as it overcomplicates the solution for a simple probability problem.

Option B: " Use reinforcement learning to train a model " is incorrect because reinforcement learning is not needed for a simple probability calculation.

Option D: " Use unsupervised learning to create a model " is incorrect as unsupervised learning is not applicable to this task.

AWS AI Practitioner References:

Choosing the Right Solution for AI Tasks: AWS recommends using the simplest and most efficient approach to solve a given problem, avoiding unnecessary machine learning techniques for straightforward tasks.

The correct answer is C because Amazon Bedrock Guardrails provides out-of-the-box configurable safety mechanisms to control the behavior of LLMs in generative AI applications. Guardrails can be configured with denylists, content filters, sensitive topics, and tone enforcement, all without retraining the model.

From AWS documentation:

" Amazon Bedrock Guardrails allows developers to define safety and responsible AI policies directly in the model inference layer, making it easy to prevent harmful, biased, or unsafe outputs with minimal configuration. "

Explanation of other options:

A. Bedrock playgrounds are interactive environments for testing prompts and models but do not provide production-grade safety enforcement.

B. SageMaker Clarify focuses on bias detection and explainability for supervised ML models — it does not directly apply guardrails to LLM outputs.

D. SageMaker JumpStart is for model fine-tuning and deployment, not for enforcing safety policies on LLM responses.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Documentation – Guardrails Overview

AWS Responsible AI Whitepaper

AWS Certified ML Specialty Study Guide – Safety in Generative AI