The current lack of centralised clarity in dispatch reasoning makes it difficult to identify and evaluate possible process improvements to mitigate perceived skips. This project will explore the current state of dispatch transparency and define innovative new routes to increased dispatch transparency, including developing a new definition and methodology as well as a proof-of-concept tool. This will give engineers greater opportunity to mitigate potential future skips and enable NESO to understand the wider system conditions that contribute to the occurrence of perceived skips. This will be achieved by consulting with stakeholders and specialists, auditing external and internal data sources, and exploring statistical and AI methods that will prove useful in terms of increasing the range and scope of dispatch transparency tools available to NESO.
Benefits
This project will build upon on-going work to provide improved dispatch advice and facilitate convergence between Electricity National Control Centre (ENCC) engineers and their support tools. This project should provide improved situational awareness for ENCC, giving potential balancing costs savings through identification/implementation of more economically beneficial options to obtain balancing services. It will also aim to enhance the clarity and authority with which NESO communicates on the subject of skip rates and dispatch reasoning.
For external stakeholders, an improved dispatch transparency methodology will provide clearer lines of communication between the market and the control room, giving deeper business intelligence to zero carbon operators enabling them to improve their participation in the balancing mechanism.
Learnings
Outcomes
Phase 1 delivered a comprehensive evaluation of dispatch transparency approaches, identifying Gradient-Boosted Trees and foundation modelling as the most promising methods. A specification for a PoC tool was produced.
Phase 2 delivered a production minded machine learning capability and user interface to support improved understanding of dispatch decisions across technology types and zones.
The project delivered four key outcomes:
> Established capability: A proof-of-concept Decision Intelligence tool capable of modelling dispatch outcomes and highlighting key contributing system factors at aggregate technology and zonal levels.
> New insight: Improved understanding of where dispatch behaviour is predictable and where it is inherently complex, including clear limitations at BMU-level granularity and in data-sparse contexts.
> Informed decision-making: Evidence-based assessment of which analytical approaches are viable for dispatch transparency, supporting NESO in avoiding further investment in less suitable methods and focusing future work on high-value areas.
> Strategic direction: Demonstrated that while technical feasibility exists in controlled context, current modelling capability is not yet sufficient to reliably explain dispatch decisions in a way that supports operational decision-making.
An XGBoost based modelling approach was used to forecast dispatch volumes, with SHAP explanations applied to identify the key factors influencing those outcomes.
The outputs provide a reusable analytical framework and evaluation baseline that can inform future dispatch transparency initiatives, and potential targeted tooling where predictability is demonstrably sufficient.
Lessons Learnt
AI model selection: LLMs were unsuitable for time series dispatch modelling. Early-stage experimentation with immature methods can reduce efficiency. Prioritise established methods (gradient-boosted trees) for time series problems and validate methodological suitability early through targeted prototyping.
Insight interpretability: SHAPexplanations produced stable but sometimes circular insights. Correlation outputs do not support operational decision-making. Future projects should prioritise causal influence approaches or hybrid techniques to increase decision usefulness. Feature naming: Unclear labelled features significantly reduce usability for non-technical users. Ensure Human-readable feature definitions and naming conventions are embedded as a design requirement from project inception, particularly for tools intended for operational users.
Data quality: Additional data sources were identified as helpful to this analysis, and changes to NESO data collection practices may be warranted to improve future model inputs. Future innovation projects should include early-stage data readiness assessments and where necessary, define changes to data collection processes as part of scope.
