Operating a future energy system with high levels of renewables will require significantly more flexible, zero carbon capacity than currently available. Large amounts of short-duration flexibility will be needed to match supply and demand within the same day. During periods of high or low renewable generation, greater amounts of within-day flexibility (WDF) will be needed.
There is no widely agreed method for quantifying the need for WDF. Unless this method is developed, inconsistent and flawed methods may be used, leading to inefficiencies.
The project will seek to develop a rigorous, repeatable, transparent method for quantifying the need for WDF. The method will include identifying relevant data sources and how to process them, assumptions, treatments of averages and extremes, calculations and interpretation of results.
Benefits
A rigorous method for quantification of the need for WDF will allow more accurate and consistent foundational assumptions across many ESO processes including setting market requirements for existing balancing services, developing new balancing services and markets, forecasting energy, flows and constraints, planning network development, planning control room capability.
Making the method transparent and public will allow other energy system participants with an interest in flexibility to challenge and improve the method, and to incorporate it into their own processes.
Learnings
Outcomes
The project addressed the challenge of replacing fossil fuel-based flexibility with sustainable alternatives by establishing a robust methodology to calculate within-day flexibility (WDF) needs. This methodology, based on probabilistic modelling and detailed time series analysis, enabled a comprehensive understanding of both typical and extreme values of WDF requirements under various future scenarios, ensuring the grid's reliability amidst increasing renewable integration and shifting demand patterns.
The project's rigorous approach to defining and measuring WDF needs incorporates unmodified generation and demand behaviours, distinguishing between inflexible and flexible resources. By excluding controllable behaviours from the WDF calculation, the methodology accurately captures the true flexibility requirements of the grid. The inclusion of extensive historical data and sensitivity analyses further strengthens the model's reliability, allowing it to accommodate future uncertainties in technology uptake and behaviour changes. This detailed analysis highlights wind generation as a primary driver of WDF needs.
Overall, the project equips the ESO and network licensees with a sophisticated tool for assessing and planning for within-day flexibility needs, which is critical for maintaining grid stability in a decarbonized energy system. The use of Python ensures the methodology's scalability and adaptability, providing a foundation for ongoing improvements as input data evolves. Importantly, this outcome represents a significant advancement over previous methods by focusing exclusively on understanding WDF need without conflating it with flexibility solutions. This approach maintains the methodology's integrity and ensures its applicability to future energy scenarios.
Lessons Learnt
When the model was fully developed and run internally by ESO colleagues, the code run time was on the order of several hours. A code optimization has been requested, which caused a 10% cost uplift, but allowed us to have a code 40 times faster. Unfortunately, delivering an optimized code wasn’t among the initial requirements for this project.
Over the course of the project and once the interim report was issued, the project team realized that the challenge of dividing the flexible component from the inflexible component of certain type of demand (in particular electric vehicle and heat pump demand) was underestimated. Currently, the method uses a conservative approach, assuming all EV and HP demand as inflexible, due to the lack of large public datasets able to show profiles of engaged and not engaged consumers, as well as predictions on how consumers will behave in the future.
