Alexis
Hi everyone. Welcome to another episode of Talk Tech with Data Dave. My name is Alexis, and I am here today with none other than our very own Data Dave. Hi Dave, how are you today?
Data Dave
I’m very well, Alexis. How are you?
Alexis
I’m good.
Data Dave
Glad to be here.
Alexis
Yeah. Yeah. And it is so morning here. I am happy for another day. The sun is finally shining here on this spring morning, and I’m very excited because we have a listener question.
Data Dave
Excellent.
Alexis
Specifically, a follow-up to our Data Lake vs. Data Warehouses kind of series that we did early on this year. So very, very excited about this.
The question came from one of our listeners via your LinkedIn, Dave, and I’ve said it before, I’ll say it again. Names are so important to me, and I don’t want to get this incorrect. So, listener out there, I’m going to link your name in the post when we put it out there, but I don’t want to say it incorrectly on the pod, so I’m not going to say it here. But thanks for the question, really appreciate it.
So Dave, here’s the question the listener said.
“Really enjoyed your perspective on data lakes versus data warehouses. Especially the idea that many teams don’t have a data problem, they have a data understanding problem.” I remember you said that, Dave; that was very clever. “Curious to hear your thoughts on where AI initiatives break down from a data architecture standpoint today.”
Data Dave
That’s an interesting question. That’s a very loaded question from an AI perspective. The biggest breakdown, I break that up a little bit. The biggest breakdown I see from a data perspective in AI initiatives is a level of expectation.
Alexis
Okay.
Data Dave
People expect magic, and they can’t necessarily get it.
And I’ll give you an example.
We worked with a client a little while ago. They were a building materials manufacturer, and they wanted to improve their path to market, improve their marketing, improve how they should do that. And what they were saying is the people who drive our product adoption in the market are the architects who design buildings.
Alexis
Oh yeah, okay, that makes sense.
Data Dave
Yeah.
So, they came to us and said, “Okay, we would like you to do analysis into our data set of which architects are recommending our paints. Can we reach out to influencers, given that architects are our influencers. Right. They’re influencing the design of the buildings, et cetera.”
Alexis
That’s super smart.
Data Dave
Okay, Great idea. The data they gave us was all their commercial data. So, we knew who the customer was, we knew who the supplier was if they distributed, we knew how they got the materials. We knew a lot about the supply chain and the material, but nowhere did it mention the architect.
Alexis
No.
Data Dave
Right. So, you could infer a lot. You could infer the person who was installing it, the primary contractor, all kinds of things. Right? On the physical creation of the building with these materials. But nowhere did it say who designed the building.
Alexis
So all the AI in the world…
Data Dave
Exactly. All the AI, all the technology in the world is never going to answer that question from that data.
Alexis
Oh, wow. Yeah.
Data Dave
There’s this expectation set that magic is going to happen. And they couldn’t point us at publicly available data that we could augment that data set with. That would say, “Okay, this project done here was driven by this,” or anything like that. Right. That does not exist.
We had to sort of burst that bubble a little bit with sort of, “The answer to your question does not live in the data that you provided.”
Alexis
Wow.
Data Dave
It’s like buying a book on the Roman Empire and expecting to read about Aztecs.
Alexis
Yeah.
Data Dave
It’s simply not there. Right. You’re looking in the wrong place.
Alexis
Yes.
Data Dave
It’s a very interesting question. It’s a very interesting data set. The data set can tell you a lot, but it can’t answer that particular question.
Alexis
Yeah. And I could see why and how a lot of companies today might hear words like generative AI and agentic AI and expect magic, and it’s just not there. Like, that’s just not how it works.
Data Dave
Yeah. It’s just not there.
Alexis
Yeah.
Data Dave
You could read those documents. There’s a lot you can do, but there’s a lot you can’t do if the data’s simply not there.
Alexis
Right.
Data Dave
And that, I think, is probably the biggest misconception, as well as the hallucination, which is… if I analyze this, you might not get the answer you want because your data was not quite clean enough for this purpose.
So, you might get misleading answers as well. You can’t just trust what you find in your data because it might give you misleading answers.
We go back to the data warehousing/data lake discussion. You’ve got a large cadre of data. You might ask a question of that large cadre of data that goes back 20, 30 years, but the answers might not be relevant anymore.
I did another project with a computer manufacturer who gave me 20 years of sales data and wanted to predict behavior patterns and future customers from the 20 years. And it’s kind of, well, actually, the relevance drops off after about five years. The guy who bought a computer five years ago might have moved, might have done something else, might have changed jobs, might have done whatever. He may or may not still be relevant. So, giving me the 20 years isn’t very useful.
Alexis
I was just in a webinar yesterday, and they were talking about, I think it was AWS, and it was early on in the AI kind of boom. And AWS was using AI to analyze job applicants. They were just kind of using the history of job applicants and the history of people that they hired, and AI analyzed applications. The AI was basically showing the historical bias that the humans had always put into it. The AI was showing that that had always been there, and it was adapting it because that was what the history was showing. So, it was there. So to your point, those are the things that I think you’re going to see whenever you’re putting in 20 years of data.
Data Dave
Whenever you’re taking all of your history and putting it together, it’s just going to extrapolate the trends that exist. Yeah, it’s not going to reinvent unless you tell it to reinvent, unless you guide it to do something else. So to answer the question a little further, it takes time to get results, even if the answer does exist in the data set, but there’s an expectation that you’ll find it right away.
There’s an expectation that it will be correct. But when you start talking about predictive and that sort of thing, you’re going to exaggerate and extrapolate trends that exist in your data that you may or may not like. So, you’re going to have to tune your answer to eliminate the pieces that you don’t like, because your data is going to describe what was not what should have been.
Alexis
Right.
Data Dave
Or what you think might have been, which is another way to read the results. You have to read them with that lens in order to then use the results in an appropriate way.
Alexis
So there’s another part to this question that I want to ask you about, Dave. The person who asked the question said, “Can I hear your thoughts on where AI initiatives break down from a data architecture standpoint today?” And data architecture is the part of that question that I want to ask you about, because I really liked your answer there. But I want to kind of lean into the data architecture question. Understanding that data architecture is really “the way we manage data. What would you say is your answer from that standpoint?
Data Dave
The answer from that standpoint is probably more “finding the particular answer.”
So, we’ve talked in theory about the data science side of it, right? Which is, does the answer exist? Can I use this data for the direction that I’m trying to go? Can I get something meaningful from it?
The next question becomes, if I can get something meaningful from it, can I do it quickly enough, repeatably enough, and in a structured manner enough? And can I trust it?
So this is where your data architecture starts to come in, which is, yes, we can find the result. Okay. Yes, we can find the architects. Are we always collecting architects? Have we got the architects indexed within our data sets?
Alexis
Have we got the artifacts? Architects?
Data Dave
Architects. So, I’m going back to the building materials, right? Building manufacturing, right? So, I’m going back to my example.
Alexis
Okay, got you.
Data Dave
And saying, okay, we’ve decided we do have architects, right? So we do have architects. Can we trust that data? Is it indexed? How frequently do we collect it? Do we always collect it? What is the data governance on it not being there? How many false nulls do we have in it that says “Mickey Mouse” is the architect, right?
So even if we have the data, how good is the data for the purpose that we’re putting it to?
And have we engineered the data in a way that these lines of influence or these paths through the data can be navigated efficiently and effectively? So, even if we’ve got a field for architect, so there was an influencer on this building, and it was this guy, is it Mickey Mouse, right? Is it null? Is it Mickey Mouse? How do we put it together? Can we navigate it? How do we collect that data? Do we need to put governance on it, et cetera, et cetera, to make sure this data that is, often in an AI world, ancillary data rather than main business purpose data.
So, we’re looking at sales data or delivery data, and then we’re looking for influencers. Well, there was no governance on influencers at the time that data was collected. You can’t retrofit it and say, “Who was the architect on that building that was put up five years ago?” Well, you probably can, but that’s probably a lot of work when you have all that data.
So, have you got the right structures in your data to answer that question? And so, if you start saying, okay, I’m going to do the data science that says, with the first part of the question, can this data answer this question? And if so, how and how would I use it? That’s the sort of the data science exploration part of it. The next part of it is, if the answer is yes, how do I engineer the operationalization of getting that answer and using that answer?
So how do I structure my data? Is my data clean enough? How do you follow these lines through that data? And how do I do it all day, every day, and put it in front of the person that needs it all day, every day?
Do you think that helps answer the question?
Alexis
I think so. And what I took away from that is to temper your expectations for a moment.
Data Dave
Right.
Alexis
And then take your time looking for your answers.
Data Dave
Exactly. There’s a lot of work in this that people don’t realize. You can ask the question, but do you get the answer? And these are pretty in depth questions. I’m not decrying the, “I asked ChatGPT or whatever AI engine you want to use to do some analysis on my Roman Empire looking for Aztecs.” Be cautious of the results and understand how you’re going to use the results as an individual. You can temper that.
Alexis
Yeah.
Data Dave
If you feed that into a business process or a business structure or into a point to make automated decisions, that’s a different level of responsibility. Correct?
Alexis
Yes. Perfect. I love that answer, Dave.
Well, listener, I hope that we hit the answer to your question here. If we didn’t, please let me know. Send us an email. Send Dave another message on LinkedIn. Send me a message on LinkedIn, let me know where we missed the mark, and we will happily address this question again, but I think we get it today.
Other listeners out there, please send us more questions. You can always email us at talktech@d3clarity.com or you can reach out to Dave or myself on LinkedIn.
And Dave, it has been great podcasting with you today.
Data Dave
Thank you. Always a pleasure, Alexis.
Alexis
Thanks everyone out there. Till next time, thank you.
Data Dave
Thanks.
