Alexis
Hi everyone. Welcome to Talk Tech with Data Dave. I am your host, Alexis, and I am here today with Data Dave. We are here to talk about a kind of follow-up question to last week’s podcast. So, let me start by introducing Dave. Dave, how are you today?
Data Dave
I’m very well, Alexis. I’m feeling good today, so thank you, thank you for that introduction. How are you?
Alexis
I’m doing great. I’m excited about this follow up question because I know we’ve kind of talked about it on the podcast before, but I think that this is going to give us a wonderful opportunity to really deep dive into this topic and kind of throw it on top of what we talked about last week when it came to why data projects are really so hard. And then offer up a little bit of D3Clarity advice on how to make them, you know a little bit easier.
Data Dave
Excellent.
Alexis
So, in the data world, there is often this feeling that data projects cannot be run using the Agile methodology.
For those of you who don’t know what Agile is, quick and dirty explanation. Using small sprints to push the project along, always following up, building minimal viable products as you go, and getting constant feedback from your stakeholders. That was an awful definition on my part, but I think that if you just go Google it, you’ll probably find a great answer. Or Dave, maybe one day we’ll do an actual podcast about it, but that’s not the point.
A lot of people argue that Agile is not a good way to do data projects, but Dave, I’ve heard you argue it a hundred times that Agile is the only way to do data projects because of that consistent feedback, because of that stakeholder impact. And so, I was hoping that kind of building on why data projects are so hard, you could explain here why you think Agile is the preferred methodology for data projects.
Data Dave
Let me use my shed analogy again, and my grain.
Alexis
Yes please. That’s what I was hoping you’d do. The same one from last episode.
Data Dave
If I can take a field full of grain, harvest it, and get it to my baker and the baker says, “Oh, this is great grain!” That’s one step. I can get that grain good. I could look at that and say, “Here’s the providence of the flow of all this grain that took it from the field to the baker who made a fabulous loaf of bread, and everybody’s happy.”
Alexis
And in the agile world, the that would be one sprint or one iteration.
Data Dave
Now, do I need a shed to do that?
No, not necessarily, but I might need a shed If I’ve got 50, 100,000 acres of grain, right?
So, how do I scale that up so I can take one bag of grain, get it to a baker, it’s high quality, it makes good bread. I know the providence. I picked it out of the field. I carried it all away, and I got it there.
Alexis
And again, in your analogy, if we’re talking agile, your baker, your baker’s your stakeholder, your baker’s the one who is saying, “Yes, the grain is good, that one bag made good bread. The data or whatever you’re working on, your project so far on your one iteration is good.” Right?
Data Dave
So, now it’s a question of: how do I scale that? So, if that is one iteration, that’s the full flow of that data all the way through from source to destination.
It is good.
Okay, where does it fail when I scale this up to other data, other parts of the business, other parts of the organization, how does it get corrupt? Why isn’t all my data this good?
Let me measure the next bag of grain. Is that good enough? If it isn’t, why not? Was it harvested badly? Was it stored badly? Was it just bad grain in the first place? Or not for purpose? Was it rice instead of wheat? So, now I’m making a rice pudding instead of a loaf of bread? I don’t know, is it describing the right stuff?
That’s what I’m saying. Is the data good enough for the purpose that we’re putting it to? And so, when we start talking about data products and agility, you start saying, “Well, if I can flow one piece of data all the way through and watch it all the way through, then I can do 2, 3, 4, 50, 100. Or can I?”
What is causing the breakdown? Rather than jumping to the conclusion that says:
If I put a brand spanking new shed here, all my data is suddenly going to be fabulous, right? All my grain, all my produce, is going to be perfect because I’ve put a big shed in the process.
Does that make sense?
Alexis
Yes, that’s a perfect way of describing why we argue that agile makes so much sense for data projects.
Data Dave
And we get into the weeds. We get into the details. We look at one piece of data and say, “Does that piece of data satisfy its need. How do I make this one piece of data satisfy the need that I’m putting it to?”
And then we do 2 and 3 and 4 and 5 and 6. It’s not sort of, “I’m going to buy a new tractor so I can get tons of it from the field better.” Well, we hire the tractor to do a job. Yes, we do. But we’ve got to use it properly as well.
Alexis
Yes.
Data Dave
And we’ve got to know what we’re using it for and what job we’re hiring it for.
Again, I’m really butchering this analogy.
Alexis
But the idea is solid. It was quite helpful for me to put it in that picture because pictures obviously help. And when we talk about food, it always helps for me too.
Data Dave
There you go.
Alexis
I hear you. And tomorrow’s bread-baking day for me. So, talking about bread actually was very, very helpful.
Data Dave
Okay.
Alexis
I think that that was an easy and a good way for us to kind of put it out there. That that’s our quick piece of advice on how to make a data project just a little bit easier. A pitch to go Agile versus something a little bit more difficult.
Data Dave
Right. And it’s a pitch to look at the big picture. Right? Which is— how am I collecting my data and how am I using my data? So, we use grain and shed. The shed didn’t actually know anything about the field. It doesn’t know anything about the baker. All it knows is that it’s going to store some grain for a period of time.
Alexis
Right.
Data Dave
But that’s what its job is. But that says nothing about the quality of the data, the quality of the grain that’s in it.
So, if we’re going to grow grain and deliver it to a baker for the purpose of making bread, we’ve got to know that.
Alexis
Yes.
Data Dave
And we’ve got to know that the grain that we’re growing in this field, we would like it to make bread, not animal feed.
That talks about the quality of the grain, and that tells us then what the requirements are for the tractor and the trailer and the shed and the various other things that are in the way. But none of those components along the transportation of that grain actually talk about that purpose. We are growing this grain in order to make bread.
So, we look at it from that perspective and say, “If we are making bread, then this needs to be the flow of this grain to get to bread level quality.”
Alexis
So if your data is being used for something specific, we need to make sure that your data is at a specific quality or it is fit for that purpose, which is a theme that you and I end up at all the time.
Data Dave
We talk about all the time.
Alexis
Yes. The idea that data is fit for purpose and that putting a shiny object, putting a new shed or a new tractor on your data isn’t magically going to fix it.
Data Dave
No, it doesn’t magically fix it. You so correctly said. And so knowing that purpose, knowing that flow, knowing how it’s going to be consumed is important. And knowing that big picture.
Alexis
I love that you said knowing that big picture, that is one of the principles of Agile. Like one of the core principles is knowing that big picture. Knowing that North Star. That’s the word we use. I know you guys have heard us say it on the podcast before, from the start, and then constantly building and iterating towards that North Star.
I’m sorry, Dave, I interrupted you so that I could make that point. You were getting there. You said knowing that big picture from the start.
Data Dave
And this is why data projects are challenging.
Alexis
Well, look at that. We brought it around full picture to the last episode.
Data Dave
We often do projects without knowing the whole big picture, and we often do them without knowing the source and the destination and the purpose of the data. And so, you’re building a shed without knowing what’s coming in and without knowing what the quality is that’s going out.
Alexis
Yeah, Dave, that was kind of a perfect way of both, explaining why we stand behind Agile and looping that back together with the last question we answered: Why are data projects so hard?
I think it’s great that we were able to answer those two questions kind of hand in hand and share that information at the same time. And you did a perfect job right there bringing them both together.
So, thank you so much for being with me here today and answering that question for our listeners out there.
If you have a question for Data Dave, please send us an email at talktech@d3clarity.com or you can connect with Dave or myself right on LinkedIn. We love to answer your questions on the podcast.
Dave, thank you again for being with me here today. And we’ll catch everyone in our next podcast. Thanks.
Data Dave
Thank you, Alexis. Always a pleasure.
