Many people are familiar with the outputs and apparent capabilities of large language models, with gut feelings about whether a model seems high performance or not. But few are aware of how we benchmark the performance capabilities of the different large language models. This talk explores the different measures and metrics we use to quantify the performance of large language models, as well as looking at the necessity of human in the loop and ethical concerns.