Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance

CJ Willmott, K Matsuura - Climate research, 2005 - int-res.com
CJ Willmott, K Matsuura
Climate research, 2005int-res.com
The relative abilities of 2, dimensioned statistics—the root-mean-square error (RMSE) and
the mean absolute error (MAE)—to describe average model-performance error are
examined. The RMSE is of special interest because it is widely reported in the climatic and
environmental literature; nevertheless, it is an inappropriate and misinterpreted measure of
average error. RMSE is inappropriate because it is a function of 3 characteristics of a set of
errors, rather than of one (the average error). RMSE varies with the variability within the …
Abstract
The relative abilities of 2, dimensioned statistics—the root-mean-square error (RMSE) and the mean absolute error (MAE)—to describe average model-performance error are examined. The RMSE is of special interest because it is widely reported in the climatic and environmental literature; nevertheless, it is an inappropriate and misinterpreted measure of average error. RMSE is inappropriate because it is a function of 3 characteristics of a set of errors, rather than of one (the average error). RMSE varies with the variability within the distribution of error magnitudes and with the square root of the number of errors (n 1/2), as well as with the average-error magnitude (MAE). Our findings indicate that MAE is a more natural measure of average error, and (unlike RMSE) is unambiguous. Dimensioned evaluations and inter-comparisons of average model-performance error, therefore, should be based on MAE.
int-res.com