Your organization wants you to compare various, widely available ML models for Gen AI use...

Use Model Garden to deploy the candidate models to Vertex AI endpoints. Use the Gen AI Evaluation Service API to evaluate the performance of each deployed model on the internal benchmark datasets. Report the best models based on the experiments.

Stream raw data from open-source large language model leaderboards into a BigQuery dataset. Send the data to an internal Looker Studio dashboard. Evaluate the performance of each model by using open-source datasets that are similar to the internal benchmark datasets. Report the best models based on the dashboard metrics.

Review the model cards in Model Garden to evaluate each model ' s performance on open-source datasets that are similar to the internal benchmark datasets. Report the best models based on your analysis.

Download model weights from the respective provider website for each model. Write an inference script to deploy the candidate models to Vertex AI endpoints. Write an evaluation script to compare all deployed models on the internal benchmark datasets by using Vertex AI Experiments. Report the best models based on the experiments.