Lena Volzhina: "How to combine physical models, machine learning and production performance"
How to combine physical models, machine learning and production performance
We all use weather forecasts to choose the right clothes, to wash the car in time or to plan a vacation. But how many people know how forecasts are calculated under the hood?
Weather forecasting is a difficult task: the data from weather stations located worldwide, weather radar and even satellites are used. To process these data, clusters with hundreds of thousands of cores are required to calculate the physical models of the atmosphere. And yet their predictions are not perfect.
But there are good news: some model errors can be corrected if we analyze the background of their occurrence. To automate this process, we at Yandex.Weather use machine learning.
In this talk, I will tell you how the architecture of our service works, how we organized the processing of terabytes of numeric data per day, and what hacks allow us to keep the forecasts as relevant as possible.
Lena Volzhina
Russia. Saint Petersburg
Software Engineer
Yandex.Weather
Developer at Yandex.Weather. Lena conducts experiments to improve weather forecasting with machine learning. Also she supports calculation of production models.