Even with the most modern tooling, it’s likely that you’re generating waste in the production of data in your organisation. Waste manifests as business misalignment, slow response to opportunities, poor quality of outputs, and employee disengagement. Waste can be any activity that doesn’t deliver value, and where the cause may be hidden.
We’ll review the manufacturing roots of the 7 forms of waste known as Muda in Lean, and how they have been reinterpreted for knowledge work like software engineering. Identifying and managing these wastes is core to modern software delivery and all spheres of business operations. We’ll then consider the data organisation as a factory that produces data, a factory that is constantly reconfigured by engineering as business needs change. This will allow us to identify and characterise the impact of the wastes of data production that emerge in building and running a data organisation and data platform. Initiatives like the DataOps Manifesto and Cookbook also embed this Lean philosophy.
With wastes understood, we’ll identify potential interventions to improve alignment, responsiveness, quality and engagement in data engineering. We will also introduce the Improvement Kata approach that provides a framework that any team can use for continuous improvement. You’ll leave with a good understanding how to reduce waste in data production, in order to restore pristine pipelines.
