Track record
How well does it actually do?
Every forecast is scored against the analysis that later verifies it. Skill is measured against persistence — assuming tomorrow looks like today. 0 means no better than persistence; 1 would be perfect.
Skill by variable
Latitude-weighted RMSE against the verifying GDAS analysis, at +6h.
| Variable | What it is | RMSE | Persistence | Skill |
|---|---|---|---|---|
z500 | 500 hPa height — the steering flow | 34.36 m²/s² | 237.7 | +0.86 |
msl | Mean sea level pressure — storms and fronts | 95.93 Pa | 293.9 | +0.67 |
t500 | 0.479 | 1.385 | +0.65 | |
v10m | 10 m wind, north–south | 1.082 m/s | 2.828 | +0.62 |
t2m | Temperature at 2 m | 1.237 K | 2.895 | +0.57 |
tcwv | Total column water vapour | 1.353 kg/m² | 3.066 | +0.56 |
u10m | 10 m wind, east–west | 1.068 m/s | 2.332 | +0.54 |
t850 | 850 hPa temperature — airmass | 0.948 K | 1.764 | +0.46 |
q50 | 50 hPa humidity — the one it gets wrong | 1.06e-06 | 6.832e-08 | -14.52 |
What it does badly
Stratospheric humidity. The model is trained on ECMWF's ERA5 reanalysis but
initialised from NOAA's GDAS analysis, and the two disagree about how much
water vapour sits in the lower stratosphere — GDAS is about 81% drier at
50 hPa. The model pulls toward what it was trained on, so
q50 and q100 score worse than simply assuming no
change.
It is published here rather than quietly dropped because a forecast system that only reports its good numbers is not telling you anything. The error stays flat across a 48-hour run instead of growing, and the other 68 variables are unaffected — so the honest summary is that this model does not forecast stratospheric humidity, and does forecast everything else.
Method
Forecasts are compared with the GDAS analysis valid at the same time. Errors are root-mean-square, weighted by cos(latitude) so that grid cells near the poles do not count more than their area deserves. The baseline is persistence: the initial state carried forward unchanged. Skill is 1 − (forecast error ÷ persistence error).
Persistence is a weak baseline at long leads and a strong one at short leads, so skill against it rises with lead time. It is used because it needs no external data and cannot be gamed.