Ask Your Network
Public Group
Public Group
Active 2 months ago
Have a question? Want to bounce ideas off your peers? Need to do a sanity check for a project? Reach... View more
Public Group
Group Description
Have a question? Want to bounce ideas off your peers? Need to do a sanity check for a project? Reach out via Ask Your Network on UAI Connect to engage all UAI members and collaborate with your peers. Get answers from the brightest minds in the industry. Experience a new level of peer networking, share your knowledge and in return learn and grow with one another.
A strategic question about data stored in Excel files
-
A strategic question about data stored in Excel files
Posted by Michael Fourman on January 27, 2023 at 10:24 amGood morning,
We are several months into our analytics journey and we have a decision to make on how to best handle data stored in Excel files. Here is the situation, we have several use cases now (and I suspect many more that will arise) where staff creates there own data and keeps it in Excel files they save on their personal network drive. We’d like to get this data into Azure while allowing the user to continue to update the files when they need to. Some will be updated daily, others not as often. Our current thinking is to have the staff store and work from a common network drive which makes ingestion easy but introduces some security concerns. We could have them password protect their Excel files to help with this but I’m not sure how that impacts the pipeline. Could we save the files in Azure and have the users update it there? Are there other solutions I’ve not described here? If you’ve already solved this, I’d love to hear more about your solution. Thanks!
Mike Fourman
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
——————————Cara Gilad replied 1 year, 9 months ago 6 Members · 8 Replies -
8 Replies
-
Hi Mike,
If it has to be Excel (for certain workflows Power Apps-based solution could work), it could be stored in SharePoint (Online or on-premise) to enhance user access control, maintain file versioning and perhaps implement a QA workflow in SharePoint prior to ingesting the data into its final destination. The data “sink” (in ADF-speak) could be a data lake in Azure, a SQL db or a Synapse data warehouse among other things. Synapse or Azure Data Factory (ADF) could be used to ingest, transform and enrich the data as needed. There may be more appropriate options based on your IT architecture, use of third-party products, data governance standards, etc.
Thanks,
——————————
Oleg Kachirski
Manager, Global Advisory
Black & Veatch
407-970-1034
——————————
——————————————-
Original Message:
Sent: 01-27-2023 10:23
From: Michael Fourman
Subject: A strategic question about data stored in Excel filesGood morning,
We are several months into our analytics journey and we have a decision to make on how to best handle data stored in Excel files. Here is the situation, we have several use cases now (and I suspect many more that will arise) where staff creates there own data and keeps it in Excel files they save on their personal network drive. We’d like to get this data into Azure while allowing the user to continue to update the files when they need to. Some will be updated daily, others not as often. Our current thinking is to have the staff store and work from a common network drive which makes ingestion easy but introduces some security concerns. We could have them password protect their Excel files to help with this but I’m not sure how that impacts the pipeline. Could we save the files in Azure and have the users update it there? Are there other solutions I’ve not described here? If you’ve already solved this, I’d love to hear more about your solution. Thanks!
Mike Fourman
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
—————————— -
Thanks, Bert. I have forwarded this to our Safety group for consideration.
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
——————————
——————————————-
Original Message:
Sent: 01-19-2023 10:30
From: Bert Hargesheimer
Subject: Attention Safety and Analytics Leaders!Safety and Analytics Leaders!
We are looking to build a benchmark group for Safety Metrics within UAI. We are interested in looking at the traditional metrics, but also at Serious Injuries and Fatalities (SIF). First we’d like to discuss the SIF definition (Edison has one that is broadly used) and then begin benchmarking to gauge performance. Please check in with our survey and join us during the February Safety Analytics Community meeting for the SIF discussion and in March we’ll begin looking at PSIF (Potential Serious Injuries and Fatalities).
Bert Hargesheimer
Safety Analytics Working Group Co-Lead
CPS Energy——————————
Bert Hargesheimer
VP Operational Support Services
CPS Energy
San Antonio TX
2103534323
—————————— -
Hi Doni. GTC is still early in its analytics journey. We’ve drafted a data governance document which is currently being reviewed. I’m not sure we address data classification by name but we do allude to it specifically when we talk about security. I hope someone is willing to share because I think it would not only help you, but us as well as others. Thanks for posting this!
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
——————————
——————————————-
Original Message:
Sent: 01-25-2023 09:46
From: Doni Gustafson
Subject: Data Classification and GuidelinesA few requests/questions:
- Policy: Would anyone be willing to share their Data Classification policy, which could include things like the companywide responsibilities for classifying data (e.g., confidential, sensitive, etc.) and a basic classification framework?
- Guidelines: I would also appreciate seeing your Data Classification Guidelines or Data Handling Standards that provide guidance on how to classify data and defines handling requirements based on data classifications.
- Manual or Automated: Does your organization classify data manually or automatically? If manually, how are you holding employees accountable and what are your lessons learned? If automatically, can you share what tool you are using as well as lessons learned?
- Contact: If you are willing to have a deeper conversation about the questions above, can you please let me know your contact information?
Thank you all for your time and feedback!
——————————
Donielle Gustafson
Sr Program Manager of Enterprise Data & Analytics
Black Hills Energy
1-605-800-1799
—————————— -
Hello UAI Members!
Thank you to everyone that has taken the time to participate in the Safety Analytics Benchmarking survey! So far, we have 8 members that have taken the survey. We would love to see at least 30 members take it.
We are looking to build a benchmark group for Safety Metrics within UAI. We are interested in looking at the traditional metrics, but also at Serious Injuries and Fatalities (SIF). First we’d like to discuss the SIF definition (Edison has one that is broadly used) and then begin benchmarking to gauge performance. Please check in with our survey!
If you are not in safety within your utility, please take @Michael‘s lead and forward benchmarking survey to your safety group for consideration. Thank you so much, Michael, for forwarding the Benchmarking survey to your safety group. This is an excellent first step to getting involved in this Benchmarking project. I am hopeful that more UAI Utility members forward this on to their safety groups.
Also, make sure you join us for the February 28, 2023 UAI Safety Analytics Community Conversation, where @Bert from CPS Energy will facilitate a discussion on SIF as follows and review results from the Benchmarking survey. If you are not part of the UAI Safety Analytics Community, you can request to join by completing the following form: https://uaievents.wufoo.com/forms/request-to-join-uai-communities.
February 28, 2023 UAI Safety Analytics Community Conversation:
Session Title: Part 1: Serious Injuries and Fatalities (SIF) Benchmarking and Community Discussion (Part 2 will be on PSIF)
Session Description: CPS Energy has been exploring safety metrics that help to create a better picture of safety performance in an effort to better inform leaders in where time spent making improvements will be most valuable. There are many ways to look at injury severity; DART, OSHA Severity Rate among others. Serious Injuries and Fatalities (SIF) is an emerging trend with some varying definitions. We’ve done a survey to get information from utilities regarding what metrics they’re using and we’ll share the results and then have a short discussion about interest in longer term benchmarking safety metrics with this group.If you have any questions about the benchmarking project, please reach out to me, @Leslie, at lcook@utilityanalytics.com or simply reply to this discussion thread in UAI Connect.
Thanks!
Leslie
——————————
Leslie Cook
Membership & Digital Engagement Manager
Utility Analytics Institute (UAI)
719-203-8650, lcook@utilityanalytics.com
——————————
——————————————-
Original Message:
Sent: 01-30-2023 14:20
From: Michael Fourman
Subject: Attention Safety and Analytics Leaders!Thanks, Bert. I have forwarded this to our Safety group for consideration.
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
——————————Original Message:
Sent: 01-19-2023 10:30
From: Bert Hargesheimer
Subject: Attention Safety and Analytics Leaders!Safety and Analytics Leaders!
We are looking to build a benchmark group for Safety Metrics within UAI. We are interested in looking at the traditional metrics, but also at Serious Injuries and Fatalities (SIF). First we’d like to discuss the SIF definition (Edison has one that is broadly used) and then begin benchmarking to gauge performance. Please check in with our survey and join us during the February Safety Analytics Community meeting for the SIF discussion and in March we’ll begin looking at PSIF (Potential Serious Injuries and Fatalities).
Bert Hargesheimer
Safety Analytics Working Group Co-Lead
CPS Energy——————————
Bert Hargesheimer
VP Operational Support Services
CPS Energy
San Antonio TX
2103534323
—————————— -
Hello Ty-Rell and welcome to UAI!
I spent time in Alexandria, Louisiana and love the food, culture and people! Still have many friends there. Looking forward to connecting with you at the UA Summit.
nadia
——————————
Nadia Powell
Director Enterprise Advanced Analytics
El Paso Electric
——————————
——————————————-
Original Message:
Sent: 01-11-2023 10:46
From: Ty-Rell Walton
Subject: Hello Everyone!!Hello everyone
My name is Ty-Rell Walton. I currently work for CenterPoint Energy at the Louisiana office. I’ve been with the company for a few months but, have been working in Data Analysis for three years now. I am a southern boy from the great Cajun State of Louisiana where we love our Gumbo, Boudin, and Mardi Gras! I am a 2019 graduated at Louisiana State University (Go Tigers) in Computer Science. I love to travel, spend time with family, friends, camp, watch my favorite show on Netflix (NCIS) and bowl among other things. I am very adventurous and always looking to try new activities. I am excited to join the UAI Community and look forward to learning, collaborating, and meeting new colleagues in this data analysis world. As we say back home, laissez les bon temps rouler (Let the Good time roll)!
——————————
Ty-Rell Walton
Data Analyst
CenterPoint Energy
—————————— -
Hi Nolan,
Is there a recording available for this seminar?
nadia
——————————
Nadia Powell
Director Enterprise Advanced Analytics
El Paso Electric
——————————
——————————————-
Original Message:
Sent: 11-08-2022 15:53
From: Nolan Steiner
Subject: Live Webinar: Powering a Data Catalogue – Avista’s data culture journey (using data catalogue)Live Webinar: Wednesday November 16, 2022
8am Pacific time; 11am Eastern time
Please join us for an overview of Avista’s experience adopting a data catalogue (to promote ‘analyst productivity’)Avista Utilities is an energy company founded in 1889 based in Spokane WA. Avista provides electricity to 400,000+ customers and natural gas to 370,000+ customers in WA, OR, ID and MT.
Sign up for this event using the attached link.
Powering a Catalog: Avista’s Data Culture Journey | Alation Resources——————————
Nolan Steiner
Manager, Data Science
Avista Corporation
Spokane WA
ph: 509-495-4479
—————————— -
Hi Nadia: Yes…per your request, attached is the link to the webinar recording we did recently for Alation (our journey on our Data Catalogue), for those interested. It’s about an hour total.
Thanks for asking!
——————————
Nolan Steiner
Data Science
Avista Corporation
Spokane WA
5094954479
——————————
——————————————-
Original Message:
Sent: 02-01-2023 10:44
From: Nadia Powell
Subject: Live Webinar: Powering a Data Catalogue – Avista’s data culture journey (using data catalogue)Hi Nolan,
Is there a recording available for this seminar?
nadia
——————————
Nadia Powell
Director Enterprise Advanced Analytics
El Paso Electric
——————————Original Message:
Sent: 11-08-2022 15:53
From: Nolan Steiner
Subject: Live Webinar: Powering a Data Catalogue – Avista’s data culture journey (using data catalogue)Live Webinar: Wednesday November 16, 2022
8am Pacific time; 11am Eastern time
Please join us for an overview of Avista’s experience adopting a data catalogue (to promote ‘analyst productivity’)Avista Utilities is an energy company founded in 1889 based in Spokane WA. Avista provides electricity to 400,000+ customers and natural gas to 370,000+ customers in WA, OR, ID and MT.
Sign up for this event using the attached link.
Powering a Catalog: Avista’s Data Culture Journey | Alation Resources——————————
Nolan Steiner
Manager, Data Science
Avista Corporation
Spokane WA
ph: 509-495-4479
—————————— -
We also use the Sharepoint-hosted solution for Excel files, but I do not have experience with the password protection. You may want to consider changing the way the data is captured from staff – I agree with the PowerApps suggestion. If all updates are appended new rows, without editing old rows, you could use a web-based survey that feeds a sharepoint List, which is a straightforward way to capture data entry from multiple people without them being able to see or edit each other’s responses. But this only works in limited cases.
——————————
Cara Gilad
Data Scientist
Exelon
——————————
——————————————-
Original Message:
Sent: 01-27-2023 10:23
From: Michael Fourman
Subject: A strategic question about data stored in Excel filesGood morning,
We are several months into our analytics journey and we have a decision to make on how to best handle data stored in Excel files. Here is the situation, we have several use cases now (and I suspect many more that will arise) where staff creates there own data and keeps it in Excel files they save on their personal network drive. We’d like to get this data into Azure while allowing the user to continue to update the files when they need to. Some will be updated daily, others not as often. Our current thinking is to have the staff store and work from a common network drive which makes ingestion easy but introduces some security concerns. We could have them password protect their Excel files to help with this but I’m not sure how that impacts the pipeline. Could we save the files in Azure and have the users update it there? Are there other solutions I’ve not described here? If you’ve already solved this, I’d love to hear more about your solution. Thanks!
Mike Fourman
——————————
Michael Fourman
Director – Engineering Services
Georgia Transmission
——————————
Log in to reply.