Measure more than one thing
Evaluation can consider task performance, instruction following, factuality and safety. The right measurements depend on the intended use and have limitations.
Keep evaluation separate
Training, tuning and final evaluation data play different roles. Repeatedly optimizing against a test set can make it less informative as an independent check. The displayed mean gives the two categories equal weight. A high average can hide a low score in one category.
INTERACTIVE EXPLORATION
Compare category scores
Change the score in category A. Category B stays at 80; both categories have equal weight.
Inspect trade-offs
An average can hide failures on particular categories. The experiment shows two hypothetical category scores and their unweighted average; it does not evaluate an actual model.
Check your understanding
Can the same overall average hide a decrease in one category?
Yes. Another category can improve enough to offset it.