Fabrikam Inc.

The first action should be to evaluate the model output . Fabrikam already has evaluation data containing manually defined expected responses, and the technical requirements explicitly state that the organization must use its existing evaluation datasets based on real data with input-output pairs . Microsoft Foundry supports evaluating a model against an existing curated dataset to establish measurable baseline performance before optimization. Evaluation results can identify weaknesses in accuracy, relevance, groundedness, safety, or other quality dimensions and provide objective evidence for subsequent changes.

This sequencing is critical because Fabrikam also requires that advanced fine-tuning be applied only when prompt engineering is insufficient . Therefore, immediately fine-tuning the model would bypass the required baseline evaluation and optimization process. Fine-tuning should be driven by demonstrated performance gaps, not applied automatically.

Deploying directly to production is also inappropriate because leadership requires low operational risk and gradual rollout. Production traffic should not be used as the first evaluation mechanism when curated evaluation data already exists. Synthetic interaction data is useful when evaluation coverage is insufficient, but Microsoft specifically positions synthetic generation as an option when an adequate test dataset is unavailable. Fabrikam already has suitable evaluation data.

Study Guide Reference: Optimize generative AI systems and model performance — evaluate model performance, benchmark against curated datasets, iterate on prompts, and apply advanced fine-tuning when justified.