User Guide 026

15 Appendix E: Goodness of Fit Statistics

Brier Skill Score

The Brier Skill Score (BSS) is defined as

$BSS=1-{\frac {\langle (x_{m}-x_{c})^{2}\rangle }{\langle (x_{m}-x_{0})^{2}\rangle }}$

(A1)

where the angled brackets indicate averaging, subscripts m, c, and 0 indi-cate measured, calculated, and initial values, respectively. The BSS ranges between negative infinity and one. A BSS value of 1 indicates a perfect agreement between measured and calculated values. Scores equal to or less than 0 indicates that the mean observed value is as or more accurate than the calculated values. The following quantifications are used for de-scribing the BSS values: 0.8<BSS<1.0 = excellent, 0.6<BSS<0.8 = good, 0.3<BSS<0.6 = reasonable, 0<BSS<0.3 = poor, BSS<0 = bad.

Room Mean Squared Error

The Root Mean Squared Error (RMSE) is defined as

$RMSE={\sqrt {\langle (x_{c}-x_{m})^{2}\rangle }}$

(A2)

The RMSE has the same units as the measured data. Lower values of RMSE indicate a better match between measured and computed values.

Normalized Root Mean Squared Error

The Normalized Root Mean Squared Error (NRMSE) is

$NRMSE={\frac {RMSE}{range(x_{m})}}$

(A3)

The NRMSE is often expressed in units of percent. The measured data range range(x_m) can be estimated as max(x_m) - min(x_m) . Lower values of NRMSE indicate a better agreement between measured and computed values.

Mean Absolute Error

The Mean Absolute Error (MAE) is defined as

$MAE=\langle |x_{c}-x_{m}|\rangle$

(A4)

Normalized Mean Absolute Error

Similarly, the Normalized Mean Absolute Error (NMAE) is given by

$NMAE={\frac {MAE}{range(x_{m})}}$

(A5)

The NMAE is often expressed in units of percent. Smaller values of NMAE indicate a better agreement between measured and calculated values.

Correlation Coefficient

Correlation is a measure of the strength and direction of a linear relation-ship between two variables. The correlation coefficient R is defined as

$R={\frac {\langle x_{m}x_{c}\rangle -\langle x_{m}\rangle \langle x_{c}\rangle }{{\sqrt {\langle x_{m}^{2}\rangle -\langle x_{m}\rangle ^{2}}}{\sqrt {\langle x_{c}^{2}\rangle -\langle x_{2}\rangle ^{2}}}}}$

(A5)

A correlation of 1 indicates a perfect one-to-one linear relationship and -1 indicates a negative relationship. The square of the correlation coefficient describes how much of the variance between two variables is described by a linear fit. The interpretation of the correlation coefficient depends on the context and purposes. For the present work, the following qualifications are used: 0.7<R²<1 = strong, 0.4<R²<0.7 = medium, 0.2<R²<0.4 = small, and R²<0.2 = none.

Bias

The Bias is defined as

$Bias=\langle x_{c}-x_{m}\rangle$

(A6)

Positive values indicate over-prediction and negative values indicate under-prediction.