Learnings

Outcomes

The outcomes of the project to date are:

• The national solar generation forecast developed in the course of this project is 2.8 times better than the existing ESO PV forecast (for forecasts up to two hours ahead). The best national PV forecasts developed to date have a mean absolute error (MAE) of 233, compared to the ESO's existing national solar PV forecasts which have an MAE of 650 MW.
• A prototype of the PV Nowcasting service has been developed based on the needs of key stakeholders and end users of the forecasts. A demonstration session has been held in June 2022 and a feedback session is scheduled for July 2022.

Lessons Learnt

In the course of W1 and WP2, the project identified the following lessons:

Never underestimate the importance of cleaning up and checking data in advance
Several approaches to loading data were tried, from on-the-fly to pre-preparing, and instituted automatic and visual tests of the data to ensure the project was always lining up the various data sources correctly.

Having infrastructure as code allows the main production service to run uninterrupted
Having code to easily instantiate infrastructure is very useful to the efficient management of environments to ensure the project could bring the algorithm into productive use. The Terraform software tool was used which makes spinning up (and down) environments very easy and repeatable. Being able to spin up new environments allowed the project to test new features in development environments while allowing the main production to keep on running uninterrupted. 

Using Microservices to “start simple and iterate” accelerates development
Using a microservice architecture allowed the project to upgrade individual components as we see benefit in improving them, independently of changing other components’ behaviour. This has been very useful when building out the prototype service, as it has allowed the project team to start with a simple architecture - even a trivial forecast model - and iteratively improve the functionality in the components. For example, first starting out with one PV provider of data has allowed the project to get a prototype working, and in WP3 we will expand to onboard an additional PV provider.

Data processing may take longer than expected
While it was initially planned to extend our dataset back to 2016 for all data sources during WP2, it turned out that data processing takes much longer than expected. This does not have a direct impact on project deliverables but is something to consider in further ML research.

Data validation is important
For both ML training and inference, using clear and simple data validation builds trust in the data. This helps build a reliable production system and keeps software bugs at a minimum.

Engaging specialist UX/UI skills is important
By acknowledging that UX and UI design is a specialised area and incorporating those skills, a UI has been developed which will be easier to use and convey information effectively. This will be validated over WP3 through working with the end users. 

Building our own hardware demonstrates value for money but may pose other challenges for a small team
Two computers have been built during the project with a total of six GPUs and it is estimated that using on-premises hardware instead of the cloud for data-heavy & GPU heavy machine learning R&D can significantly reduce the direct costs. However, the time it would require for a small team to put together all the components is significant (approx. 25 days for one person in total). While the total costs would still be lower, appropriate resource planning should be considered if planning hardware upgrades in the future.