Why AI Weather?
Let’s start with something familiar
I bet that by now, you’ve got familiar with at least one AI chat. They took Internet - and our reality - by storm. Ever wondered how this magic looks underneath? Or maybe you haven’t, because the power of the tool left you intimidated?
While an exact explanation of the matter is far beyond the scope of this first post, let me share the general idea - harnessing the power of human language in the form of Large Language Models, which you interface with through ChatGPT and similar tools, was possible because the scientists and engineers found an effective way to represent human language in a form of matrices of numbers.
Think of it - every word that the chat tells to us is backed by large matrices of numbers underneath. They get multiplied with one another just like you might have learned in algebra class. It sounds silly easy, yet at the same time, it brings us a revolution on the scale of the invention of Internet.
What is the weather like?
How does this translate to weather forecasting? Numerical Weather Prediction is a well established field that arose from understanding that air particles in the atmosphere move according to physical laws and that this movement can be predicted by applying the corresponding equations.
Now let’s reverse the problem. Given the state of the atmosphere as a form of data collected from multi-source observations, can we infer the rules of the physics from this representation and apply these rules to predict the future weather?
Taking one (or many, depending how you look at it) step back: how were the laws of physics inferred in the first place? A scientist in the lab (or sitting under a tree) conducted multiple experiments and measured the outcomes, and, based on these observations, they inferred the equation. The same way we are taught physics when we are teenagers: performing experiments, taking note of the output, and finding patterns.
The advent of AI-based weather
Now let’s connect the dots. Can we find a representation of the relationships in the observed weather and use the power of methods similar to those that brought language models to life to represent the physics as matrices of numbers, just as language has been successfully represented? This is what AI- (or ML-) based weather is about.
Why I’m here sharing this with you?
I’ve been finding connections between engineering and Earth Science throughout most of my professional career. Over 15 years in (mainly) software engineering, I’ve spent 10 years in meteorology and 7 years in machine learning. Yes, that includes combining the all three, first about ten years ago when I started my own research on adding ML-inferred parametrizations to a weather model, and then again recently as I split my time between working with global (or “large scale”) AI-based (or ML-based, or data-driven) weather models and running a software development team in a private meteorological company.
I’ve been conceptualizing the big ML weather models before the first contemporary ones emerged, and now I am eager to stand on the shoulders of these early experimenters and advance this science. I am launching this space to share my knowledge and experience to help meteorologists and ML engineers speak a common language, as well as to talk about my own work on the topic.
Everyone interested in this emerging field of meteorology - which is also a new frontier in machine learning - is welcome to follow (and subscribe to) this space!