DataOps is seen as a key approach for supporting insight-driven culture at organizations seeking to extract more value from their data. DataOps is somewhat open to interpretation. It has been said that DataOps is not something you buy, it's something you do, or that it's a combination of agile development and DevOps lean manufacturing, but whatever the description, automation is an integral part of the process.
Recently, Chris Bergh, CEO and founder of DataKitchen, a DataOps vendor that provides an end-to-end DataOps software platform as well as advisory services focused on DataOps transformation, talked to BDQ about what DataOps is, how the methodology has evolved, and how customers can use it to get more value from their data.
BDQ: With the strong emphasis on using data more effectively, is DataOps becoming more widely understood?
Chris Bergh: More people are learning about it and realizing that it's important, but it's certainly not a standard yet. It isn't about a new technology; it's not about "Let's buy a faster database" or "Let's buy that cool new visualization tool." They're starting to realize that, after all this investment in new databases, big data and streaming data, and IoT, AI, ML, and all the acronyms out there, they've got all the technology that they need. It's more about the people and the process that matters—the people who use that technology—and the journey to be data-driven is getting to its natural point of maturity.
BDQ: How so?
CB: You really have to make the team, with all the tools and all the data, work together. I grew up in Wisconsin and when I was in high school, American Motors, which made the Pacer, went out of business. My dad worked for Wisconsin Bell and he drove a Toyota Corolla and people didn't like that because he was in a union and it was an American car versus a Japanese one, but my dad said it's cheaper, it's better, and it lasts longer. And so why did Toyota make better cars than American Motors? You have to work on the people, and on the process that people work in. The journey to be data-driven is less about data and more about the people and the teams who are creating value from the data.
BDQ: You have said that what you do is less important than how you do it. What does that mean?
CB: It reflects my own journey and starting to do data and analytics 15 years ago. I always look at it from the lens of a leader and a manager: How do you lead your people, what is the organizing principle to make it work, and how do you focus on the value that your customers are really receiving instead of getting into the trap of building something and expecting that a year later, wonderful things are going to happen? That has to be the shining light. In order to be data-driven, you have to be of service to the people who need the data, and help them and deliver value to them, and then work to improve upon it.
BDQ: Descriptions of DataOps methodology vary by organizations. What is your definition?
CB: I think it is a set of technical practices and cultural norms for how people work to get value out of data. It is a technology and in some ways it's technical practices or architectural practices that go hand-in-hand with how people work. There is a philosophy in DataOps that is about iterative development and delivering things short and focusing on your customer. And those come from agile processes and even from lean manufacturing ideas. And then there is a technical part to it because it's really hard to do these things when you have constellations of tools running in different systems. Those technical practices and the cultural norms are intertwined.
BDQ: How does someone start with DataOps and what does DataKitchen provide?
CB: Typically, it's a senior person in a large company who is saying, "Hey, we want to be data-driven, but it's not working." And they examine why after they have bought all the great tools, and they realize that they've got to work on the people and their process. And they stumble upon DataOps as a sort of philosophy, and then they bring us in. First, they want us to help them transform their team and lay out a set of steps for how they can go from zero to DataOps, and then they bring us in as a technology provider to help them create a central place where they can plug all the technology they have —their ETL tool, their data tool, all their people in all their different locations—and that can provide a central point of kind of coordination of all those steps that are involved in creating value, while not replacing what they currently have.
That's one of the reasons we wrote the DataOps Manifesto. It is the collected beliefs that we have from years of working problems. And, that manifesto jibes with people—they could either be a chief data officer make looking to change at his or her organization, or a small team that wants to start working in a different way.
BDQ: Does that make DataKitchen both a technology provider and consultancy?
CB: Yes, we offer advisory services and other assistance, as well as software. We realized that since this problem is both a technology problem and a people problem, you've got to work with both. We also have partners that are working to come up to speed and do this DataOps transformation work that we do, but right now we're doing it because people need the help.