[Rosemary Peh]

Mahalo piha Jason!  It looks like you are fully on a path to explore an AWS structured platform.  Just an add on question…

 

Since you use PI – are you consolidating your PI data with the meter and other data onto the AWS platform?  If so, I take that you must be using OSI’s APIs?  If so, how has that worked out for you?  Easy, hard, complicated?

 

Let’s stay connected, so that as you progress, we can then compare notes on the TCO.

Let me know.

Mahalo!

Rosemary Peh
Hawaiian Electric Company
Operations Excellence Director
(808) 543-4790 | rosemary.peh@hawaiianelectric.com
P Please consider the environment before printing this email

 

______________________________________________

CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. Any unauthorized review, use, copying, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender immediately by reply e-mail and destroy the original message and all copies.

——————————————-
Original Message:
Sent: 1/7/2021 11:00:00 AM
From: Jason Pegg
Subject: RE: Time Series Data

Aloha!

We’re currently in the middle of an evaluation of new tools to handle streaming data from multiple IoT sources.  As background, we also utilize PI for generation and transmission (we’re a vertically-integrated IOU) and our smart meter vendor for basic smart meter data collection.  That said, we’ve been dipping our toe into the AWS waters for outages where some simple AWS tools (Lambda functions, S3 for storage, Neptune as a graph database of network nodes based on a CIM model) are used to handle PONs and PRNs.  We looked at Redshift and Snowflake for data lake capabilities and landed on Redshift since the price point was lower and the capabilities met our needs.  (We were in the middle of our Snowflake/Redshift evaluation a year ago when AWS released a slew of new features for Redshift and reduced its pricing.  Redshift’s basic functionality then met our needs and we went in that direction.)

Going forward, we’re actively evaluating how we handle timeseries data.  We’re evaluating an open source approach (writing a connector to Kafka to then feed a Timescale DB on-prem with an option to host in the cloud) as well as an AWS approach (using IoT Greengrass on-prem and in the cloud, connected to Amazon Kinesis and Amazon Timestream).  From there, we would provide data mining capabilities either through direct connection to the streaming database (Timescale or Timestream) via our regular reporting tools (Tableau, QuickSight, and now possibly Grafana).

You were curious if anyone had put together a business case analysis of the different approaches, and while ours is incomplete, I thought I’d at least share where we’re at.  We are blessed with many great options in front of us.  We are also cursed with many great options in front of us.  :-)  If you’re interested, I’ll keep you updated on our progress and decision process.

Have a great day!

——————————
Jason Pegg
Domain Architect
Avista Corp.
Spokane WA
509.495.4731
——————————
——————————————-
Original Message:
Sent: 12-15-2020 15:48
From: Rosemary Peh
Subject: Time Series Data

Aloha e UAI Network!

Another question for you as Hawaiian Electric is grappling with how time series data should be consolidated in order to facilitate enterprise analysis.  We currently use OSI PI to archive and consolidate our time series data from disparate EMS and substation systems.  We noticed that when meter data management systems were implemented, OSI PI was not necessarily the data consolidation tool of choice and that many chose to consolidate the meter time series data into other data analytics tools like open source no sql databases.  However, because we are vested in OSI PI technology and capabilities (and we are not looking to move away from its current use on the generation and substation side), we were thinking that it would make sense to convert to enterprise licensing and to consolidate all time series data to OSI PI – which includes EMS, ADMS, MDMS, weather, power quality and other time series data.  The price tag to do so is not “cheap” though.  And “cheap” is relative – we are not a large utility and our roll out of meters is not big bang but slow over time.  PI pricing is by “tag” which is essentially by data source column (e.g., each meter has 8 registers that collect separate pieces of info so it will need 8 tags per meter).  The amount of data is limited to the hardware capacity – of which, if one builds flexibility in its scalability then data volume is then scalable to that design.

However, given today’s advancement in database capabilities (e.g., high scalability, lower cost per use models) and data access technologies, is moving such data that could be housed on services that are based on open source technologies prudent?  Hence, my question…

Has anyone done any business case analysis comparing the use of traditional data systems (like OSI PI) to the current offerings from data analytics infrastructure and application service providers (like SnowFlake and Starburst?  And if so, does it find that the utilization of these newer technologies a better long term solution – considering the Total Cost of Ownership?

Additionally, what is your utility using to consolidate time series data?

Appreciate your input.  Mahalo nui (Thank you) in advance!

——————————
Rosemary Peh
Operations Excellence Director
Hawaiian Electric Company
rosemary.peh@hawaiianelectric.com
1-808-543-4790
——————————