Probabilistic Machine Learning Solution for Dynamic Reserve Setting

Using artifical intelligence and machine learning this project will look to set reserve levels dynamically, day ahead. ​

When considering additional interconnectors and increasing weather driven effects on the uncertainty of visible demand (impacted by unmetered embedded generation) and metered generation, we would anticipate further increases in reserve holdings without innovative models and approaches to the challenges presented by a changing system. This innovation project attempts to prototype an advanced ML model as an ambitious methodology to capture the various sources of uncertainty, to be created in parallel with the BAU development of new models.

These algorithms would forecast reserve requirements by learning from historical behaviour and input drivers to find correlations with the uncertainty of system conditions. 

Benefits

Not used.

ENA smarter networks portal

Outcomes

The Dynamic Reserve Setting has achieved the following:

•  Built an ETL pipelines for cleaning data from ESO systems to be used in reserves forecasting 
• Built a database to store the cleaned data and forecast predictions on ESO systems 
• Delivered a software suite which can run and re-train several probabilistic ML model for reserves levels 
• Demonstrated that Machine Learning can be used to make better predictions of reserves when compared to the current approach 
• Successfully used SHAP values to provide explanations for each prediction made by the
• Machine Learning model and these explanations were well received by the control room engineers 
• Created a dashboard to display the models outputs to be used by control engineers to inform their decision making about reserves in real time. 
   

Lessons Learnt

A key lesson learnt was that liaising with control room staff to gather requirements and feedback is difficult.  Control staff work on a shift pattern whereas the rest of the members of the project team worked office hours.  This was exacerbated by the fact that control room staff are often too busy to join meetings when they are working in the control room.  For this reason, it was determined that a single point of contact with control staff should be established, this took the form of a control engineer whom had been seconded to an office based team.  This member of staff worked one shift a week across all shifts and so was able to gather feedback from control staff and deliver feedback to the project. 
 
A further lesson learnt was how the control staff interact with the data that is provided to them.  How the data is presented on a dashboard is directly related to how useful it is in a decision-making scenario.  There are many different opinions amongst control staff so many options to customise the data display were given to make the dashboard have as much utility as possible.  Many lessons have been learnt about the factors that effect reserve levels; tests were performed with various different data sources and the results measured.  Some tests were successful such as using perfect knowledge of system imbalance increasing both risk performance and number of MW saved and some were unsuccessful producing no benefit i.e. using European weather data.