hckrnws
These are available on Weatherbell[1] (which requires a subscription) now except for the HGEFS ensemble model which I'm guessing will probably be added later. AIGFS is on tropical tidbits which should be free for some stuff[5]. I believe some of the research on this is mentioned in these two[2][3] videos from NOAA weather partners site. They also talk about some of the other advances in weather model research.
One of the big benefits of both the single run (AIGFS) and ensemble (AIGEFS) models is the speed and (less) computation time required. Weather modeling is hard and these models should be used as complementary to deterministic models as they all have their own strengths and weaknesses. They run at the same 0.25 degree resolution as the ECMWF AIFS models which were introduced earlier this year and have been successful[4].
Edit: Spring 2025 forecasting experiment results is available here[6].
[1] https://www.weatherbell.com/
[2] https://www.youtube.com/watch?v=47HDk2BQMjU
[3] https://www.youtube.com/watch?v=DCQBgU0pPME
[4] https://www.ecmwf.int/en/forecasts/dataset/aifs-machine-lear...
[5] https://www.tropicaltidbits.com/analysis/models/
[6] https://repository.library.noaa.gov/view/noaa/71354/noaa_713...
Really exciting to see NOAA finally make some progress on this front, but the AIGFS suite likely won't outperform ECMWF's AIFS suite any time soon. The underlying architecture between AIFS and GraphCast/AIGFS is pretty similar (both GNNs), so there won't likely be a model-level improvement. And most of ECMWF's edge lies in its superior 4DVar data assimilation process. AIGFS is still being initialized on NOAA's hybrid 4DEnVar assimilation process as far as I understand it, which is still not as good as straight up 4DVar unfortunately.
I've seen the Microsoft Aurora team make a compelling argument that weather is an interesting contradiction of the AI-energy-waste narrative. Once deployed at scale, inference with these models is actually a sizable energy/compute improvement over classical simulation and forecasting methods. Of course it is energy intensive to train the model, but the usage itself is more energy efficient.
Obviously much simpler Neural Nets, but we did have some models in my domain whose role was to speed up design evaluation.
Eg you want to find a really good design. Designs are fairly easy to generate, but expensive to evaluate and score. Understand we can quickly generate millions of designs but evaluating one can take 100ms-1s. With simulations that are not easy to GPU parallelize. We ended up training models that try to predict said score. They don’t predict things perfectly, but you can be 99% sure that the actual score designs is within a certain distance of said score.
So if normally you want to get the 10 best design out of your 1 million, we can now first have the model predict the best 1000 and you can be reasonably certain your top 10 is a subset of these 1000. So you only need to run your simulation on these 1000.
It's definitely interesting that some neural nets can reduce compute requirements, but that's certainly not making a dent on the LLM part of the pie.
Sam Altman has made a lot of grandiose claims about how much power he's going to need to scale LLMs, but the evidence seems to suggest the amount of power required to train and operate LLMs is a lot more modest than he would have you believe. (DeepSeek reportedly being trained for just $5M, for example.)
I saw a claim that DeepSeek had piggybacked off of some aspect of training that ChatGPT had done, and so that cost needed to be included when evaluating DeepSeek.
This training part of LLMs is still mostly Greek to me, so if anyone could explain that claim as true or false and the reasons why, I’d appreciate it
And an LLM can be more energy efficient than a human -- and that's precisely when you should use it.
That's precisely when, (insert hand wavy motion), we should use any of this.
This jumped out at me as well - very interesting that it actually reduces necessary compute in this instance
The press statement is full of stuff like this:
"Area for future improvement: developers continue to improve the ensemble’s ability to create a range of forecast outcomes."
Someone else noted the models are fairly simple.
My question is "what happens if you scale up to attain the same levels of accuracy throughout? Will it still be as efficient?"
My reading is that these models work well in other regions but I reserve a certain skepticism because I think it's healthy in science, and also because I think those ultimately in charge have yet to prove reliable judges of anything scientific.
> My question is "what happens if you scale up to attain the same levels of accuracy throughout? Will it still be as efficient?"
I've done some work in this area, and the answer is probably 'more efficient, but not quite as spectacularly efficient.'
In a crude, back-of-the-envelope sense, AI-NWP models run about three orders of magnitude faster than notionally equivalent physics based NWP models. Those three orders of magnitude divide approximately evenly between three factors:
1. AI-NWP models produce much sparser outputs compared to physics-based models. That means fewer variables and levels, but also coarser timesteps. If a model needs to run 10x as often to produce an output every 30m rather than every 6h, that's an order of magnitude right there.
2. AI-NWP models are "GPU native," while physics-based models emphatically aren't. Hypothetically running physics-based models on GPUs would gain most of an order of magnitude back.
3. AI-NWP models have fantastic levels of numerical intensity compared to physics-based NWP models since the former are "matrix-matrix multiplications all the way down." Traditional NWP models perform relatively little work per grid point in comparison, which puts them on the wrong (badly memory-bandwidth limited) side of the roofline plots.
I'd expect a full-throated AI-NWP model to give up most of the gains from #1 (to have dense outputs), and dedicated work on physics-based NWP might close the gap on #2. However, that last point seems much more durable to me.
"it's more efficient if you ignore the part where it's not"
> "it's more efficient if you ignore the part where it's not"
Even when you include training, the payoff period is not that long. Operational NWP is enormously expensive because high-resolution models run under soft real-time deadlines; having today's forecast tomorrow won't do you any good.
The bigger problem is that traditional models have decades of legacy behind them, and getting them to work on GPUs is nontrivial. That means that in a real way, AI model training and inference comes at the expense of traditional-NWP systems, and weather centres globally are having to strike new balances without a lot of certainty.
It's more efficient anyway because the inference is what everyone will use for forecasting. Researchers will be using huge amounts of compute to develop better models, but that's also currently the case, and it isn't the majority of weather simulation use.
There's an interesting parallel to Formula One, where there are limits on the computational resources teams can use to design their cars, and where they can use an aerodynamic model that was previously trained to get pretty good outcomes with less compute use in the actual design phase.
I mean that’s cute, but surely you can add up the two parts (single training plus globally distributed inference) and understand that the net efficiency would be an improvement?
Comment was deleted :(
[dead]
Interestingly, while this model is based on a Google Deepmind AI weather model, it's based on a model from 2023 (GraphCast) rather than the WeatherNext 2 model which has grabbed headlines as of late. I'd imagine it takes a while to integrate and test everything, explaining the gap.
Google Research and Google DeepMind also build their models for Google's own TPU hardware. It's only natural for them, but weather centres can't buy TPUs and can't / don't want to be locked to Google's cloud offerings.
For Gencast ('WeatherNext Gen', I believe), the repository provides instructions and caveats (https://github.com/google-deepmind/graphcast/blob/main/docs/...) for inference on GPU, and it's generally slower and more memory intensive. I imagine that FGN/WeatherNext 2 would also have similar surprises.
Training is also harder. DeepMind has only open-sourced the inference code for its first two models, and getting a working, reasonably-performant training loop written is not trivial. NOAA hasn't retrained its weights from scratch, but the fine-tuning they did re: GFS inputs still requires the full training apparatus.
I've been assuming that, unlike graphcast, they have no intention to make weathernext 2 open source.
That seems to be the case from what I've heard.
I am dearly hoping that they are using the current "AI" craze to talk up the machine learning methods they have presumably been using for a decade at this point, and not that they have actually integrated an LLM into a weather model.
Graphcast (the model this is based on) has been validated in weather models for a while[1]. It uses transformers, much like LLMs. Transformers are really impressive at modeling a variety of things and have become very common throughout a lot of ML models, there's no reason to besmirch these methods as "integrating an LLM into a weather model"
A lot of shiny new "AI" features being shipped are language models being placed where they don't belong. It's reasonable to be skeptical here, not just because of the AI label, but especially for the troubled history of neural-network based ML methods for weather prediction.
Even before LLMs got big, a lot of machine learning research being published were models which underperformed SOTA (which was the case for weather modeling for a long time!) or models which are far far larger than they need to be (e.g. this [1] Nature paper using 'deep learning' for aftershock prediction being bested by this [2] Nature paper using one neuron.
Not all transformers are LLMs.
Yes, that is not in contention. Not all transformers are LLMs, not all neural networks are transformers, not all machine learning methods are neural networks, not all statistical methods are machine learning.
I'm not saying this is an LLM, margalabargala is not saying this is an LLM. They only said they hoped that they did not integrate an LLM into the weather model, which is a reasonable and informed concern to have.
Sigmar is correctly pointing out that they're using a transformer model, and that transformers are effective for modeling things other than language. (And, implicitly, that this _isn't_ adding a step where they ask ChatGPT to vibe check the forecast.)
It’s not an LLM, but it is genAI. It’s based on the same idea of predict-the-next-thing, but instead of predicting words it predicts the next state of the atmosphere from the current state.
It is in fact one of the least generalized forms of "AI" out there. A model focused solely on predicting weather.
"gen" stands for "generative". If you read the GenCast paper they call it a generative AI - IIRC it's an autoregressive GNN plus a diffusion model.
Which is surprising to me because I didn't think it would work for this; they're bad at estimating uncertainty for instance.
> Which is surprising to me because I didn't think it would work for this; they're bad at estimating uncertainty for instance.
FGN (the model that is 'WeatherNext 2'), FourCastNet 3 (NVIDIA's offering), and AIFS-CRPS (the model from ECMWF) have all moved to train on whole ensembles, using a cumulative ranked probability score (CRPS) loss function. Minimizing the CRPS minimizes the integrated square differences of the cumulative density function between the prediction and truth, so it's effectively teaching the model to have uncertainty proportional to its expected error.
GenCast is a more classic diffusion-based model trained on a mean-squared-error-type loss function, much like any of the image diffusion models. Nonetheless it performed well.
The GraphCast paper says "GraphCast is implemented using GNNs" without explaining that the acronym stands for Graph Neural Networks. It contrasts GNNs to the " convolutional neural network (CNN)" and "graph attention network." (GAN?) It doesn't really explain the difference between GAN and a GNN. I think LLMs are GANs. So no, it's not an LLM in a weather model, but it's very similar to an LLM in terms of how it is trained.
> I think LLMs are GANs.
They aren't, but both of them are transformer models.
nb GAN usually means something else (Generative Adversarial Network).
I used GAN to mean graph attention network in my comment, which is how the GraphCast paper defines transformers. https://arxiv.org/pdf/2212.12794
I was looking at this part in particular:
> And while Transformers [48] can also compute arbitrarily long-range computations, they do not scale well with very large inputs (e.g., the 1 million-plus grid points in GraphCast’s global inputs) because of the quadratic memory complexity induced by computing all-to-all interactions. Contemporary extensions of Transformers often sparsify possible interactions to reduce the complexity, which in effect makes them analogous to GNNs (e.g., graph attention networks [49]).
Which kind of makes a soup of the whole thing and suggests that LLMs/Graph Attention Networks are "extensions to transformers" and not exactly transformers themselves.
Oh yeah, GNN (graph neural network) is the common term, "graph attention network" is pretty confusing because a GAN is a totally different architecture.
(Well, not necessarily architecture. Training method?)
You're absolutely right! That was a category 5. Thanks for pointing that out.
Hopefully they weren’t all forced out this year. The NOAA had massive cuts.
NCAR is being dismantled as we speak.
I suspect the names of those perpetrating this kind of destruction will become synonymous with ignorance and intellectual cowardice.
Same. I hope this was written by hardened greybeards who have dedicated their lives to weather prediction and atmospheric modeling, and have "weathered" a few funding cycles.
inb4 it’s actually an intern maintaining a 3000+ line markdown file
I can see it now
The following snippet highlights the algorithm used to determine <thing>
```fortran
.....How well do these predict extremes/outliers? Given that I expect these are more "ML" type models, these are somewhat limited to interpolation, rather than extrapolation?
What does AI refer to here? Presumably weather models have been using all sorts of advanced machine learning for decades now, so what’s AI about this that wasn’t AI previously?
They're using a graph neural network. From the article - "The team leveraged Google DeepMind's GraphCast model as an initial foundation and fine-tuned the model using NOAA's own Global Data Assimilation System analyses".
> so what’s AI about this that wasn’t AI previously?
The weather models used today are physics-based numerical models. The machine learning models from DeepMind, ECMWF, Huawei and others are a big shift from the standard, numerical approach used for the last decades.
Do these ML models replace the numerical approach completely? A lot of numerical methods are iterative. If the ML model can produce a good initial guess, it might make convergence of an iterative process quite a bit quicker…
Thanks, I guess my assumption that ML was widely used in forecasting is wrong.
So are they essentially training a neural net on a bunch of weather data and getting a black box model that is expensive to train but comparatively cheap to run?
Are there any other benefits? Like is there a reason to believe it could be more accurate than a physics model with some error bars?
Machine learning _has_ been widely used in weather forecasting, but in a different way than these models. Going back to the 1970's, you never just take the output of a numerical weather model and call it a forecast. We know that limitations in the models' resolution and representation of physical processes lead to huge biases and missed details that cause the forecast to disagree with real world observations. So a standard technique has been to post-process model outputs, calibrating them for station observations where available. You don't need super complex ML to really dramatically improve the quality or skill of the forecast in this manner; typically multiple linear regressions with some degree of feature selection and other criteria will capture most of the variance, especially when you pool observation stations together.
> Are there any other benefits? Like is there a reason to believe it could be more accurate than a physics model with some error bars?
Surprisingly, the leading AI-NWP forecasts are more accurate than their traditional counterparts, even at large scales and long lead times (i.e. the 5-day forecast).
The reason for this is not at all obvious, to the point I'd call it an open question in the literature. Large-scale atmospheric dynamics are a well-studied domain, so physics-based models essentially have to be getting "the big stuff" right. It's reasonable to think that AI-NWP models are doing a better job at sub-grid parameterizations and local forcings because those are the 'gaps' in traditional NWP, but going from "improved modelling of turbulence over urban and forest areas" (as a hypothetical example) to "improvements in 10,000 km-scale atmospheric circulation 5 days later" isn't as certain.
This ambiguity resulted in some very funny drama on Bluesky: https://bsky.app/profile/nws.noaa.gov/post/3ma754dbtuj2t
Holy shit lmao. Like the wokest tumblr crowd focused into a laser for 2025. How do these guys get through life? Exhausting existence.
AI refers to whatever would have been called "Machine Learning" five years ago.
> Presumably weather models have been using all sorts of advanced machine learning for decades now
This isn't actually true, unless you're considering ML to be just linear regression, in which case we have been using "AI" for >100 years. "Advanced ML" with NN is what's being showcased here.
Is there a primer for reading these files?
https://www.nco.ncep.noaa.gov/pmb/products/gens/
https://www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_s...
No, not really. If you are just looking to work with the data you want to read about extracting from grib2 format. One of the faster ways off the ground is to use the Pywgrib2_s python package and iterate against the model files using python to extract the fields that are interesting. I have a container on docker hub that has pywgrib compiled with all its dependencies if you want to tinker.
pywgrib https://www.cpc.ncep.noaa.gov/products/people/lxu/cookbook/a... containerized https://hub.docker.com/repository/docker/jmarks213/container...
It's far, far simpler for users to simply use eccodes[1], particularly as implemented in xarray[2].
[1]: https://github.com/ecmwf/eccodes [2]: https://docs.xarray.dev/en/stable/index.html
These look like staging MVP releases with a full rollout planned for the future. They are only including a few parameters at every 6 hours which is barely interesting to anyone with their feet on the ground.
I wonder if the new models consider land use change and emissions from aggressive datacenter development and model training...
Apparently it seems to be impossible with these files and the best AI right now to answer the simple question, will it rain in midtown Manhattan tomorrow?
Take an umbrella if you're concerned.
What is possible is to know with near certainty the rough tonnage of water that will fall across a wide area grain region in an upcoming week.
Useful for the reliable production of grain (timing seeding, harvesting, spraying, etc) in the millions of tonnes.
how about working with Weather Underground to validate predicted weather at ground level? Here in Southern CO would be a perfect place to try this. Weather Underground has thousands of volunteer backyard weather stations, including mine.
I understand that aviation safety is certainly a primary concern for NWS/NOAA but ground level forecasts are also very important for public safety.
Whatever it is, it seems like it might be roughly competitive with ECMWF, the state of the art when it comes to global weather models: https://www.epic.noaa.gov/ai/eagle-verification/
A quick search didn't turn up anything about the model's skill or resolution, though I'm sure the data exists.
They run at 0.25 degree resolution (same as ECMWF AIFS models).
Neil Jacobs, Ph.D
This makes me skeptical that it isn’t just politicized Trumpian nonsense.
Protip: Any time you read "AI" in a news article, substitute the phrase "faster, more numerous, and confidently incorrect." I don't think we need "confidently incorrect" weather models. Who is asking for this?
These models actually outperform traditional methods on many fronts, including accuracy a lot of the time. They are technically generative AI models, but they're definitely not LLMs.
If it were LLM I’d agree, but that’s not the case here.
Crafted by Rajat
Source Code