Learnings

Outcomes

The outcomes of the project are as follows; 

• A better understanding has been gained of the data within our PowerOn system, including the degree to which multiple alarms are created in the background of PowerOn which users are not aware of.
• We have developed greater awareness of where Personal Data is unexpectedly contained within datasets and how this data can be cleansed in order to allow it to be shared.
• We have gained an understanding of the rate of alarms and events experienced by the DSO support systems, their categories and their relative prevalence.
• A trial system has been developed and tested which can be modified for use with another licence area within NGED or a DNO that is not part of the NGED group.
• A view of what visualisations are helpful to a user trying to understand the correlations between events and alarms from different systems has been gained from the user reviews of the system.
• The learning from NEAT to date has been fed into the workshops held by Energy Systems Catapult in February 2022 in relation to Local Area Energy Planning. 

Lessons Learnt

Data Quality and Availability

• Data Quality issues - Some of the data quality issues did not reflect issues with the underlying data sources but were introduced during the process of data extraction.  This included the incorrect use of field delimiters which resulted in some data being reported in the wrong columns after a certain record in the data extract.
• Prohibitive data volumes - There was a very large alarm flood in November 2020 with millions of records added in a single day.  This prevented the normal system for reporting from working as it could not manage the volume of alarms. These were eventually identified as relating to a particular device, a Panacea PMAR which when connected via a DNP3.0 comms link to the host RTU, will exhibit an abnormal behaviour as the device is unable to recognise the acknowledgement of the alarm being sent back from the host RTU to the device.
• 12h and 24h time formats - These were often mixed within the same report. Sometimes the use of 12h time without an am/pm indicator made it impossible to tell the time of the event.
• TSDS vs HISTAN data - While some initial analog value data was available from TSDS, it was decided to switch to the HISTAN system for the regular data extract of that data as it was more suited to the regular data extraction.
• GDPR compliance - While none of the fields titles for the extracted data suggested personal information, there were items of personal data entered within comment text fields which needed to be located using standard text items as search terms.  While not strictly necessary, additional anonymization was undertaken to remove the names of field staff who were despatched to faults or control staff that confirmed alarms.
• ANM data availability - The data available for the Exploratory Data Analysis and the System Trial was less than expected reflecting both the delay in commissioning the Devon ANM, and the low levels of customer constraint during the winter months.
• SVO data availability - The switch off of the SVO system during the project made it impossible to trial that element of the system in real time.   It was, however, possible to adapt the system to handle historic data and replay it in real time.  It was not known originally whether the changes in the rest of the network data in PowerOn would enable this but it appears that a sufficient majority of assets were unchanged to enable the data processing to work.
• Network Model Update – built in validation - Wellington electricity will be using electric office to create PowerOn patches, this could provide a model for future development by NGED as a means to reduce the issues around network model update that can cause problems with the state estimation modules of ANM and SVO.

Specification & Design    

• Filtering alarms - The data includes large amounts of irrelevant items such as bund pump alarms and radio alarms, excluding these allowed for more focussed analysis
• Alarm data content - Some items do not have an electrical location but still might contain useful information. There is a difficulty associating these with other alarms
• PowerOn Patches - It is not possible to easily determine the assets associated with a patch in PowerOn and hence whether a particular network amendment would be a likely cause for subsequent failures on the ANM or SVO systems.

Deployment & Test

• VPN - Unexpected VPN incompatibility issues resulted in delays while an alternative end point on Harmonic’s side was set up.
• Blind deployment - There was significant latency in some steps of the deployment while approaches were found and staff were unavailable to do some of the steps that Harmonic/PSC don’t have access to do.
• CyberArk - Using a Firefox browser was thought to be the reason behind a user getting stuck in a loop when trying to log in to CyberArk.   WPD only has Chrome and Edge as supported browsers.  This did not turn out to be the problem but the link provided was different to that used by WPD IR, it was missing "/V10"  in the middle of the link.
• OpenShift - OpenShift was deployed during the development phase of the project and was supposed to be ready for the deployment, but a number of unexpected issues delayed the availability of the platform and therefore the deployment of NEAT.
• Security processes - Multi factor authorisation failed to work the first time for one user because their phone was set to a time that was more than 6 minutes off from the correct time.
• Security processes - Using the WebEx to change passwords, the normal process to change initial passwords for security, was very complex with remote third parties.  Having a separate zoom meeting to be able to talk to each other during the process was very helpful.
• Testing data volume - Due to the low number of events due to the time of year and well-behaved systems, there was little current data to test with. It was not possible to import historic data because it was not kept in the expected format.
• Nested IT Issues  - Until the VPN issue was fixed we could not detect the CyberArk issue, until the CyberArk issue was fixed we could not detect the OpenShift issue. There is little opportunity for parallel work to resolve issues due to the serial way in which they are detected.

Trial   

• Lack of a representative user - The majority of the trial investigation was carried out by the project manager as there was no-one in the role of the system user that could be called upon. 
• Event investigation - Without other references it was difficult to evaluate whether the information provided was sufficient to evaluate determine the root cause of the event. 
• Word Cloud - The word cloud did not work well for events that were triggered by multiple instances of the same alarm as there was no additional text items to include. 
• 11/12 minutes past the hour - Incorrect date format in the export process caused alarms, switching events, and issues to have dates all 11 or 12 minutes past the hour until identified in NEAT. Data had not been used in this level of detail before.
• Issue drill down  - None of the issues that were examined could easily be related to work that had been identified as happening via the CROWN data. It is expected that this would be easier with a geographical visualisation of the CROWN events.
• Trial Timing - The delays to the project resulted in the trials being held during the winter, when the chances of ANM operation to curtail generation were low.   Given that it was possible to replay the SVO data it could have been possible to replay ANM data from the summer, however , it is believed that additional effort would be better spent testing a different licence area.
• Event categories - PowerOn alarms were the most prevalent with ANM alarms being second and SVO third.
• Event types - The Volume alarms e.g. a high number of stage 2 alarms being received in a 24 hour period, were the most frequently
• Event locations - Three locations were responsible for nearly 75% of all the events identified suggesting that targeted remedial work would be very effective in bringing down event volumes
• Overall alarm volumes - Overall alarm volumes were lower than expected suggesting that even with the additional focus on ANM systems following Access SCR changes, that this would not warrant data imports at a higher frequency than once a day.

Project process

• Assumed levels of support from National Grid IT - The learning from previous projects had been applied in that there were 1) No interfaces to National Grid systems, 2) Very early engagement with the IT staff, including the cyber team, to flag up future requirements for support. It was expected that would be sufficient to allow for smooth installation, but the introduction of new systems, impact of covid, and the change in IT priorities during the corporate merger were highly unusual.  This demonstrates the value in having projects that can extend their timescales and confirms the need for a moderate level of financial contingency for projects.
• Lack of a representative user - This was problematic throughout the various stages of the project and suggests that the project may have been undertaken too early in the process of determining the strategy for the DSO support systems.
• Working across time boundaries - There were several instances where time was lost due to the time difference between the UK and New Zealand.