Iteration is an evidence-driven lifecycle phase. Option D combines three essential inputs: production telemetry, sampled output evaluation, and stakeholder feedback. Telemetry exposes latency, cost, errors, tool failures, retrieval behavior, and operational anomalies. Output sampling identifies quality, safety, grounding, or consistency failures that infrastructure metrics may not reveal. Stakeholder feedback establishes whether the system continues to satisfy real business and user needs.
The architect should prioritize the highest-impact validated problem, define the intended improvement, and test the proposed change against the existing evaluation framework and reference set. Anthropic emphasizes that evaluations make behavioral changes visible before they affect users and that their value compounds throughout an agent’s lifecycle. Demystifying Evals for AI Agents
Option A replaces targeted iteration with unnecessary redesign. Option B incorrectly treats deployment as the end of measurement. Option C removes the baseline needed to distinguish genuine improvement from regression or noise.
A mature iteration cycle preserves evaluation continuity, adds production failures as new test cases, validates changes offline, conducts controlled rollout, monitors results, and updates documentation and known limitations.
Study Guide references/topics: Production iteration; telemetry; sampled evaluation; stakeholder feedback; regression testing; evaluation continuity; change prioritization.
===============
