Like many businesses SPEN is collecting far more data now than it ever has before. The current data historians used within the company are long established (20+ years) and costly. In recent years, data storage has become cheaper and many more options are available now than there were before. To this end, SPEN undertook a quick proof of concept in 2021 to established if a generic time series database such as InfluxDB will be able to fulfill the requirements of the company. It was predicted that InfluxDB will be approximately 7 times cheaper than PI historian over the ED2 period.
This project digs deeper to test if InfluxDB can integrate appropriately with SPEN systems. The InfluxDB Phase 2 PoC will host real time data from LV power quality monitors in parallel with the PI Data Historian. There is also a budget to establish how PowerOn would cope with a change in data historian as the current interfaces are PI Historian specific.
Benefits
- Significant reduction in operating costs in relation to current data historian solutions
- Ultimately Integration of digital and analogue data onto a single platform promoting more efficient data collation, management of the power network and meeting of SPEN regulatory requirements
- Migration to a contemporary cloud-based data historian platform, provides the ability to take advantage of modern interface techniques in line with SPEN Analytics and Historian architecture strategy, along with a significant drop in cost of ownership over traditional on-site solutions.
Learnings
Outcomes
The project has successfully shown that a generic time series database (such as InfluxDB) can be used as a Data Historian for SPEN. However, there are two different product streams for InfluxDB which would need different amounts of engineering support.
- On-Prem Enterprise – The esoteric nature of the InfluxDB database architecture, means that there would be significant overhead in terms of ‘upskilling’ personnel to deliver an efficient internal SPEN support function, which argues against going with an On-Prem solution.
- Cloud-Based SaaS (Software as a Service) – This proof of concept was based around the SaaS based solution, unlike with the On-Prem approach SPEN would not have to upskill in order to understand the specifics of the database architecture and design as these are managed by InfluxData as part of the SaaS solution. The help desk service is very efficient and was extremely responsive to support cases raised by the project team.
Functionally, InfluxDB is different from PI Historian (as would be expected) which does not lend well to a like-for-like comparison. However, each of the individual use cases prepared by SPEN and evaluated by Capula show that it would be possible to proceed with this kind of solution going forward, especially since the InfluxDB Cloud-Based SaaS implementation passes SPEN’s cyber security requirements.
InfluxDB is collecting data from the roll out of the LV monitor programme. The current licence is live until 31st of March 2024 using innovation funding as part of an extended trial, which will further validate this platform as a suitable Business as Usual (BaU) solution for SPEN. It will transition into a BaU state after this date, and will remain running until a formal tendering process is undertaken to replace the far larger data historians currently in place. InfluxDB will continue to be used alongside the existing historians until the tendering process is completed, the contracts are awarded and the new solution is fully implemented and tested. It is expected this process will take a couple of years.
There are a few drawbacks about InfluxDB that may require further investigation or to be specifically addressed in a future implementation. Firstly, it uses downsampling to better store the data but this could be detrimental to SPENs requirements as the granularity is lost over time, for example it may reduce the time stamp resolution in a way that looses some of the required data.
Another drawback is that InfluxDB is optimised for storage/insertion of data rather than updates and deletion. It is not a traditional CRUD database. This results in a laborious process for renaming a tag should this be required (the data would require to be downloaded, edited offline and reloaded). Whilst updates to and deletion of data can be performed, the process is fundamentally more complex than in PI. With PI a tag can simply be renamed. With InfluxDB 'tags’ are a definitive element of the database which cannot be altered/deleted so new data on a new tag is added, the old data is deleted but the old tag will remain in the database indefinitely.
Lastly, InfluxDB lacks any form of asset management framework which is an integral part of PI. This is not an issue from SPENS perspective as we do not use this feature, it is “built-in” and paid for as part of the PI Historian package.
The rough estimates for the financial side of InfluxDB is that including engineering support and rollout fees it would be around a third of the cost of PI historian. These figures are very rough as they include setting up and creating a bespoke influx implementation and were based on our current PI estimates for a 200k tag licence. Over ED2 SPENs data storage requirements will see a large increase and the tendering process will capture the requirements to the end of the regulatory period.
Overall, a very successful Innovation Project.
Lessons Learnt
This project has been successful but was slower than planned. In future, the project timelines will take into account SPEN’s extensive governance processes and allow extra time for plans to be approved and purchase orders to be raised etc.
It has also shown us that newer technology options can be evaluated using innovation funding should this be necessary in future.
Note: The following sections are only required for those projects which have been completed since 1st April 2013, or since the previous Project Progress information was reported.
