Statistics

Given the initial measured values $x_{0}$, final observed or measured values $x_{m}$ and final calculated values $x_{c}$, there are several goodness of fit statistics or skill scores which can be calculated. The definition for some of the more common ones are provided below.

Brier Skill Score

The Bier Skill Score (BSS) is given by

$BSS=1-{\frac {{\bigg \langle }{\big (}x_{m}-x_{c}{\big )}^{2}{\bigg \rangle }}{{\bigg \langle }{\big (}x_{m}-x_{0}{\big )}^{2}{\bigg \rangle }}}$

(1)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, $x_{0}$ is the initial measured values and the angled brackets indicate averaging. The BSS ranges between negative infinity and one. A BSS value of 1 indicates a perfect agreement between measured and calculated values. Scores equal to or less than 0 indicates that the mean value is as or more accurate than the calculated values.

Table 1. Brier Skill Score Quantifications

Range	Quantification
0.8<BSS<1.0	Excellent
0.6<BSS<0.8	Good
0.3<BSS<0.6	Reasonable
0<BSS<0.3	Poor
BSS<0	Bad

Example Matlab Code:

 BSS = 1 - mean((xc(:)-xm(:)).^2)/mean((xm(:)-x0(:)).^2);

Nash-Sutcliffe Coefficient

The Nash-Sutcliffe Coefficient (E) is commonly used to assess the predictive power of a model. It is defined as

$E=1-{\frac {{\bigg \langle }{\big (}x_{m}-x_{c}{\big )}^{2}{\bigg \rangle }}{{\bigg \langle }{\big (}x_{m}-\langle x_{m}\rangle {\big )}^{2}{\bigg \rangle }}}$

(2)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. The Nash-Sutcliffe efficiency coefficient ranges from negative infinity to one. An efficiency of 1 corresponds to a perfect match between measured and calculated values. An efficiencies equal 0 or less indicates that the mean observed value is as or more accurate than the calculated values.

Table 2. Nash-Sutcliffe Coefficient Quantifications

Range	Quantification
0.8<E<1.0	Excellent
0.6<E<0.8	Good
0.3<E<0.6	Reasonable
0<E<0.3	Poor
E<0	Bad

Example Matlab Code:

 E = 1 - mean((xc(:)-xm(:)).^2)/mean((xm(:)-mean(xm(:))).^2);

Root-Mean-Squared Error

The Root-Mean-Squared Error (RMSE) also referred to as Root-Mean-Squared Deviation (RMSD) is defined as

$RMSE={\sqrt {{\bigg \langle }{\big (}x_{m}-x_{c}{\big )}^{2}{\bigg \rangle }}}$

(3)

where where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. The RMSE has the same units as the measured and calculated data.

Example Matlab Code:

 RMSE = sqrt(mean((xc(:)-xm(:)).^2));

Normalized-Root-Mean-Squared Error

In order to make comparing different RMSE with different units or scales (lab vs field) several non-dimensional forms of the RMSE have been proposed in literature. Here the Normalized-Root-Mean-Squared Error (NRMSE) is defined as

$NRMSE={\frac {\sqrt {{\bigg \langle }{\big (}x_{m}-x_{c}{\big )}^{2}{\bigg \rangle }}}{\max {(x_{m})}-\min {(x_{m})}}}$

(4)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. The NRMSE is often expressed in units of percent. Smaller values indicate a better agreement between measured and calculated values.

Example Matlab Code:

 NRMSE = sqrt(mean((xc(:)-xm(:)).^2))/range(xm(:));

Mean-Absolute Error

The mean absolute error is given by

$MAE={\bigg \langle }{\big |}x_{m}-x_{c}{\big |}{\bigg \rangle }$

(5)

where where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging.

Example Matlab code:

 MAE = mean(abs(xc(:)-xm(:)));

Normalized-Mean-Absolute Error

The normalized-Mean-Absolute Error is defined as

$NMAE={\frac {{\bigg \langle }{\big |}x_{m}-x_{c}{\big |}{\bigg \rangle }}{\max {(x_{m})}-\min {(x_{m})}}}$

(6)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values.

Example Matlab code:

 NMAE = mean(abs(xc(:)-xm(:)))/range(xm(:));

Correlation coefficient is defined as

Correlation is a measure of the strength and direction of a linear relationship between two variables. The correlation coefficient $R$ is defined as

$R={\frac {\langle x_{m}x_{c}\rangle -\langle x_{m}\rangle \langle x_{c}\rangle }{{\sqrt {\langle x_{m}^{2}\rangle -\langle x_{m}\rangle ^{2}}}{\sqrt {\langle x_{c}^{2}\rangle -\langle x_{c}\rangle ^{2}}}}}$

(7)

where where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. A correlation of 1 indicates a perfect one-to-one linear relationship and -1 indicates a negative relationship. The square of the correlation coefficient describes how much of the variance between two variables is described by a linear fit.

Example Matlab code:

 R = corrcoef(yc,ym);

Bias

The bias is a measure of the over or under estimation and is defined as

$B=\langle x_{c}-x_{m}\rangle$

(8)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. The bias is a measure of the over or under prediction of a variable. Positive values indicate overprediction and negative values indicate underprediction.

Example Matlab code:

 B = mean(xc(:)-xm(:));

Normalized Bias

The normalized bias is a measure of the over or under estimation and is defined as

$NB={\frac {\langle xc-xm\rangle }{\max {(xm)}-\min {(xm)}}}$

(9)

where $x_{m}$ is the measured or observed values, $x_{c}$ is the calculated values, and the angled brackets indicate averaging. The normalized bias is a measure of the over or under prediction of a variable and is often expressed as a percentage. Positive values indicate overprediction and negative values indicate underprediction.

Example Matlab code:

 NB = mean(xc(:)-xm(:))/range(xm(:));

---

Documentation Portal