A company has fine-tuned an Amazon Bedrock foundation model (FM) to produce short document summaries.

AWS documentation identifies ROUGE as a standard evaluation metric for text summarization tasks . ROUGE measures the overlap of n-grams, sequences, or word pairs between a machine- generated summary and a human-written reference summary , making it ideal for automated evaluation.

In this scenario, the company needs a metric that directly compares generated summaries with human references. ROUGE scores quantify how much of the reference content is captured by the model output, which aligns with AWS recommendations for evaluating summarization quality.

The other metrics are not appropriate. F1 score is typically used for classification tasks. Perplexity measures language model confidence but does not compare outputs to reference summaries. Fréchet Inception Distance is used for image generation tasks, not text.

AWS consistently recommends ROUGE for summarization evaluation, making it the correct choice.