[Gabe Prado]

Great points @Ben and @Ankush. Chiming in here a bit late but here’s my perspective & observations.

The semantic layer/data fabric is a product of architecture design and software tools and can be represented visually by a knowledge graph or kept track of in code. The data fabric essentially abstracts individual data sets/sources with various formats and standards into rules and relationships which are more intuitive to humans and can be more easily governed and adapted. The knowledge graph approach enables a more powerful, intuitive query: ie “show me all transformers (asset) manufactured by ABB (properties) with an active alert (Events) that have not been inspected in the last three months (Work History).

As you mentioned, Ben, there is some setup involved in creating this initial knowledge graph but automation is making this much easier. Teams of experts could perform the mappings themselves today, but it’s impractical due to the scale of data and human limitations (implying that these experts have perfect purview into every data relationship that must be mapped). Tool-based automation gets us 80% of the way there to cleaner, transformed data sets, reducing the burden on the SME to 20%. 

Here’s a really good example of one tool in a data fabric that automates the high overhead part of matching imperfect identifiers by extracting relationship rules from the string of characters in the identifier. An expert can then review and approve the rules and then apply them to new data sets as necessary. Once those mappings have been codified, they don’t need to be done each time you need the data, and therefore you create a new node in your knowledge graph which can be queried for whatever you need. Then it becomes an exercise in adding new data to your growing data model(s) as needed to solve sequential use cases. Other tools here include document parsing, image classification, etc. 

Final point- There’s a set of tools involved in accelerating the creation of the data model/knowledge graph, but you still need to pay attention to data quality, governance, lineage, etc. Collectively, this canon of software tools form the practice of Industrial DataOperations, which is focused on making the end-to-end data journey from source system to application API as efficient, consumable, and scalable as possible. This subject matter didn’t exist yet in a structured format, so we actually wrote the book on it at Cognite. Happy to send a copy your way if interested!

——————————
Gabe Prado
Director of Product Marketing
Cognite
——————————
——————————————-
Original Message:
Sent: 09-03-2021 23:43
From: Ankush Agarwal
Subject: Bi-Weekly Feature — What is a Data Fabric and How Does it Complement my Data Warehouse?

I love that you answered your own question Ben 😊. Semantic layers make the data “truly” useful. Data will always need some level of clean-up for each specific use case, but this layer brings it pretty close to “prime-time”. We’ve onboarded a data engineer on our team recently and he has become an invaluable member of our team, because he is creating this new layer. There’s still work to be done to expand this so that it can be scalable across multiple use cases, but what a change!

——————————
Ankush Agarwal
Director, Grid T&D Analytics
Exelon Corporation
——————————

Original Message:
Sent: 09-03-2021 10:29
From: Ben Ettlinger
Subject: Bi-Weekly Feature — What is a Data Fabric and How Does it Complement my Data Warehouse?

I’ll answer my own question based on investigation into some Gartner research, after the previous post. The semantic layer can be a knowledge graph. “The main advantage of using a knowledge graph for this type data integration scenario is that it not only solves the actual data integration task, but it’s also bridging the gap between disparate corporate datasets and the actual business requirements. It allows business users to model, explore and discover how datasets are connected using a human-readable conceptual knowledge graph model where there is little to no difference between the conceptual and the physical, machine-readable data models.” (Demystifying the Data Fabric- Gartner). At first glance, it would appear to be a huge effort to set up a knowledge graph, requiring both IT and Business subject matter experts. And even before reaching that phase a dedicated effort in data cleaning, data quality, and data governance would be required. Otherwise the data fabric would have a lot of warped weft.

——————————
Ben Ettlinger
Emerging Technologies
Enterprise Architecture & Engineering Group
New York Power Authority
White Plains, NY
914 681 6496

Original Message:
Sent: 09-03-2021 08:51
From: Ben Ettlinger
Subject: Bi-Weekly Feature — What is a Data Fabric and How Does it Complement my Data Warehouse?

Thanks for this article. But it still leaves me trying to understand what that semantic layer is all about. Is it a set of indices pointing to data attributes defined with meta data? When the rubber hits the road what is being requested on the front end of the semantic layer? data fields?, meta data? What connects the data requestor to the semantic layer, a data fabric application? What’s the difference in having a semantic layer or technologies used by products like Databricks, Snowflake or Redshift?

——————————
Ben Ettlinger
Emerging Technologies
Enterprise Architecture & Engineering Group
New York Power Authority
White Plains, NY
914 681 6496

Original Message:
Sent: 09-01-2021 15:01
From: Erin Hardick
Subject: Bi-Weekly Feature — What is a Data Fabric and How Does it Complement my Data Warehouse?

Hello members,

This week’s feature shows how data fabric is critical to connecting a traditional data warehouse (DWH) to advanced analytics at scale. “What is a Data Fabric and How Does it Complement my Data Warehouse?”, written by @Gabe, defines data fabric and the two main pillars of data fabric: context and discovery. ​

“Data context is the sum of meaningful use, case supportive relationships within and across different data types and data artifacts. It is the result of data relationship mining and curation in a so-called contextualization pipeline.
Data discovery is about making data effortlessly available to the right user in the right format. This always has been the goal of data and information architects. Discovery in B2C technology is instantaneous, autonomous and continuously self-learning.”

Read the full article and let us know what you think! What is a Data Fabric and How Does it Complement my Data Warehouse? – Utility Analytics Institute

Utility Analytics Institute		remove preview
What is a Data Fabric and How Does it Complement my Data Warehouse? – Utility Analytics Institute Connecting DWH to advanced analytics at scale requires a data fabric, not just data availability. For industrial companies, the path to ultimate value from data liberation requires three crucial steps. Many organizations have already achieved step one: liberating data from siloed source systems and aggregating it in a traditional data warehouse (DWH). View this on Utility Analytics Institute >

Happy reading!

——————————
Erin Hardick
Senior Research Analyst
Utility Analytics Institute (UAI)
CO
——————————

[Kevin Praet (Adm)]

Hello UAI Utility Members,

I hope everyone had a nice weekend! I just wanted to send one final reminder for our upcoming session on Asset Health Analytics Use Case Identification & Prioritization (part of our Asset Health Analytics Community) taking place Wednesday, September 22nd at 2:00 PM CT and invite you to be part of the discussion. Due to the relveancy of this topic, we extend this invite to ANY utility member of UAI and encourage you to spread the word internally. For more info on this session, including how to get registered, please see below. Feel free to email me at kpraet@utilityanalytics.com if you have any questions or need help registering.

Session Description: 

Join the Asset Health Analytics Community for part two of a two-part series on use case identification and prioritization on Wednesday, September 22, 2021 at 2:00 PM CT. Part two of this series includes an interactive, open discussion to learn what asset health analytics use cases members are working on, identifying valuable use cases you are pursuing or are wanting to pursue, and discuss how you prioritize use cases, i.e. What is your organization using to prioritize use cases? What does that process look like? How do you monetize the projects as they come in (earn revenue from or add value to the business)?

We will ask attendees to take a poll before the session to identify your use cases by category and type. We will use the valuable information captured during the poll to drive the conversation during this session. After the Community Conversation, we will share them with the members of the Asset Health Analytics Community via an Asset Health Analytics Use Case Repository.

Register for this session: https://eb.endeavorb2b.com/uai-community-signup
Take our pre-session poll: https://www.menti.com/iw15h4u63x/0

Again please reach out to me if you have any questions.

Thanks and I look forward to connecting Wednesday!

——————————
Kevin Praet
Membership Coordinator
Utility Analytics Institute (UAI)
Boulder CO
315-440-3033
——————————
——————————————-
Original Message:
Sent: 09-15-2021 10:00
From: Kevin Praet
Subject: Utility Members Invited to Upcoming Asset Health Analytics Community Conversation: Part 2 Asset Health Analytics Use Case Identification & Prioritization

Hello UAI Utility Members,

This month we have a very exciting Asset Health Analytics Community Conversation planned and want to invite you to be part of the discussion. On Wednesday, September 22nd at 2:00 PM CT, we will be having part 2 of our discussion around Asset Health Analytics Use Case Identification & Prioritization and attendees will participate in the discussion through an interactive poll and open discussion. Due to the popularity of part 1, we’re very excited to continue to the discussion and invite each of our utility members to take part!

*If you’re not already an Asset Health Analytics Community Member, please feel free to register for this community on our“request to join form”OR reach out to me directly to let me know your interested and I’ll be happy to get you registered. As a reminder utility members recieve unlimited access to our 8 current communities.

Please also feel free to get started on the poll by using the link below!
https://www.menti.com/iw15h4u63x/0

Session Description: 

Join the Asset Health Analytics Community for part two of a two-part series on use case identification and prioritization on Wednesday, September 22, 2021 at 2:00 PM CT. Part two of this series includes an interactive, open discussion to learn what asset health analytics use cases members are working on, identifying valuable use cases you are pursuing or are wanting to pursue, and discuss how you prioritize use cases, i.e. What is your organization using to prioritize use cases? What does that process look like? How do you monetize the projects as they come in (earn revenue from or add value to the business)?

We will ask attendees to take a poll before the session to identify your use cases by category and type. We will use the valuable information captured during the poll to drive the conversation during this session. After the Community Conversation, we will share them with the members of the Asset Health Analytics Community via an Asset Health Analytics Use Case Repository.

__________________________________________________________________________________________________________________________________________

Please spread the word and again let us know if we can help get anyone registered.

Thanks and we hope to see you online next week!

——————————
Kevin Praet
Membership Coordinator
Utility Analytics Institute (UAI)
Boulder CO
315-440-3033
——————————