Data Makers Festival
Instead of deciding the best way to run data and AI teams, learning about antipatterns can be more productive. In this talk, I’ll go through typical negative scenarios that occur during the development of data science projects. I’ll provide effective methods borrowed from the Lean manufacturing methodology to alleviate these issues.
Despite significant investments in data science and engineering teams, adopting and deploying advanced analytics and AI in the industry is still far from common. What are the reasons for this? Even if you have a competent team focused on a valid use case, we still struggle to get results.
To set the stage, I’ll illustrate an aggregation of all that can go wrong in a data science and engineering project - what I describe in the Elements of Data Strategy as an “implementation maze”. This story shows how issues like lack of planning for integration, vague requirements, or unreproducible work can derail even the best teams. We can see those issues as “waste” that must be removed. To do that, I’ll go through the best methods I have learned as an engineering manager and CTO, such as templating, rapid prototyping, and focused documentation of results that will make our data science work lean and productive.