You need to recommend a solution to assess the responses generated by Agent1 when the...

A Retrieval Augmented Generation evaluator is required because Agent1 will retrieve product-sheet content and use it as grounding context when generating answers. Foundry RAG evaluators assess both retrieval quality and the quality of the final grounded response. This directly supports Contoso’s requirement that answers based on the product sheets be relevant, complete, and accurate.

The appropriate evaluation suite can include groundedness , which determines whether the answer is supported by the retrieved product information without fabrication; relevance , which evaluates whether the answer accurately and completely addresses the customer’s question; and response completeness , which measures whether critical expected information is missing. Retrieval evaluators can additionally determine whether the indexing and search pipeline returned context chunks relevant to the query.

A coherence evaluator assesses readability and logical organization but does not establish that claims are supported by storage1. A custom guardrail controls or blocks behavior rather than measuring RAG quality. Fine-tuning changes model behavior and does not evaluate whether retrieved documentation was selected and used correctly.

The AI-103 Study Guide includes evaluating models and applications for fabrications, relevance, quality, and safety , as well as monitoring grounding quality and retrieval relevance.