Built block by block
Every block implemented and unit-tested on its own: patch embedding, positional encoding, attention, transformer block and loss. A test checks that nn.MultiheadAttention is never used.
Scientific ML · forecasting
Projects Weather Transformer from Scratch
Forecasts four ERA5 weather fields six hours ahead with a vision transformer. Every block, attention included, is written and unit-tested by hand.

Tested on all 1,463 six-hour steps of 2020, a year held out from training, against two baselines: persistence, which assumes nothing changes in six hours, and anomaly persistence, which carries the current departure from the seasonal average forward. Scores are latitude-weighted and in physical units. On 10 m wind the model beats both. On 850 hPa temperature and 500 hPa geopotential it does not beat anomaly persistence: 1.24 K against 1.02 K, and 213 against 191 m²/s². Over India it is level with persistence on wind and behind it on temperature and geopotential.
| field | model RMSE | vs a no-change forecast |
|---|---|---|
| 850 hPa temperature | 1.24 K | 3.4% worse |
| 500 hPa geopotential | 213 m²/s² | 3.5% better |
| 10 m wind, east–west | 1.17 m/s | 21.8% better |
| 10 m wind, north–south | 1.18 m/s | 34.5% better |
| All four, pooled (standardised) | 0.185 | 28.7% better |
A vision transformer with 4.8 million parameters that forecasts four ERA5 fields six hours ahead, with attention written by hand instead of nn.MultiheadAttention. Its loss adds two penalties to the error: one for jagged fields and one for a wrong global mean.
Every block implemented and unit-tested on its own: patch embedding, positional encoding, attention, transformer block and loss. A test checks that nn.MultiheadAttention is never used.
129 unit teststest suite · Nov 2026
Contact / Rourkela, Odisha, India