You are here: Home / Methods / AERONET-based scores

AERONET-based scores

Go to the AERONET-based monthly scores

Go to the AERONET-based seasonal scores

Go to the AERONET-based annual scores

A system to evaluate the performance of the model has been set. The system yields, on a monthly, seasonal and annual basis, evaluation scores computed from the comparison of the simulated dust optical depth (DOD) and the direct-sun AOD retrievals provided by the AErosol RObotic NETwork (AERONET).

The evaluation system is applied to instantaneous forecast values of DOD ranging from the initial day (D) at 15:00 UTC to the following day (D+1) at 12:00 UTC. It means that the lead times of forecasts to be evaluated range from 3 to 24 hours.

The scores are computed for each AERONET site, for 3 sub-regions (Sahel/Sahara, Middle East and Mediterranean) as well as globally considering all sites (stations list). It should be noted that scores for individual sites can be little significant for being calculated from a small number of data.

Observations with an Ångström exponent 440-870 higher than 0.6 are discarded in order to restrict the evaluation to situations in which mineral dust is the dominant aerosol type. However, other particles are always present (anthropogenic aerosol, products from biomass burning, etc.). Therefore, negative bias can be expected.

The common metrics that are used to quantify the mean departure between modelled (ci) and observed (oi) quantities are the mean bias error (BE), the root mean square error (RMSE), the correlation coefficient (r) and the fractional gross error (FGE). They are presented in the following table, where n denotes the number of data.

Statistic Parameter

Formula

Range

Perfect score

Mean Bias Error (BE)

 

−∞ to +∞

0

Root Mean Square Error (RMSE)

 

0 to +∞

0

Correlation coefficient (r)

 

-1 to 1

1

Fractional Gross Error (FGE)

 

0 to 2

0

−      The mean bias error (BE) captures the average deviations between two datasets. It has the units of the variable. Values near 0 are the best, negative values indicate underestimation and positive values indicate overestimation.

−      The root mean square error (RMSE) combines the spread of individual errors. It is strongly dominated by the largest values, due to the squaring operation. Especially in cases where prominent outliers occur, the usefulness of RMSE is questionable and the interpretation becomes more difficult.

−      The correlation coefficient (r) indicates the extent to which patterns in the model match those in the observations.

−      The fractional gross error (FGE) is a measure of model error, ranging between 0 and 2 and behaves symmetrically with respect to under- and overestimation, without over emphasizing outliers.