An AI project is usually born out of a perceived potential in a data and product of a company. All the buzz around AI and machine learning excites executives to invest in a data science teams. But a data science project is not the same as an IT project, a software development project or an engineering project. The key differentiation is in the word "science". In this article, I argue, industrial AI projects are more akin to drug discovery projects. And companies that have mature R&D mindset are the only ones that can successfully bring an idea to a solution on the mass commercialization level and reap large benefits.
Let's first create contrasting stories so we can see how NOT to conduct an AI project:
1 - The Waterfall Process
Think of building a bridge, a mechanical machine or an electronic hardware device. You know the requirements, the necessary building blocks and clear steps that need to be followed. Each step has a clear start and finish: 1- prepare the ground, 2- construct a foundation, 3- construct poles, etc…
I guess for anyone in information technology and familiar with agile software development, it is clear why this would not work. The reason also is the interactive nature of data and product in the AI solution. It is rare that all necessary data for the development of a full AI solution is ready. Think about a self-driving car project. No one has all data necessary to build a product. When a prototype is built, it is sent to the road to collect data. I argue the current self-driving cars are not really the product. They are just data collection devices. The idea is that the AI solution will evolve and improve with each iteration of this cycle.
2 - The Agile Process
The iterative nature of building data collection devices and prototypes and their continuous improvements reminds us of an agile process. The way the agile/lean methodology builds a product is nicely captured by the metaphor below by Henrik Keniberg.
If the car is a metaphor for a software product, it is easy to determine what is the MVP version of it. For going from point A to B a skateboard is the most minimal way to do so. We can build skateboard component easily and in the software world, the same codebase can be used to build up future versions on the same foundation. This will not work for an AI product.
Think about image classification for example. The figure below shows an error rate of best models developed in different years by different teams on the classification of images from ImageNet database. The models improve and eventually surpass human performance recently. This is not the same story as continuous improvements over the years.
A clear jump is mid-2012 with AlexNet. This is the deep learning model that Alex Krizhevsky developed at the University of Toronto's Geoffrey Hinton research group. Prior to his work, an approach like his would have been unimaginable. Building AlexNet has an inherent uncertainty. We could not have write a product requirement or timeline for it. The key advancements here are made through scientific exploration. And scientific methodology does not quite fit the agile process.
This example is in the research domain. But in the industrial environment, there are similar situations where an approach may appear totally out of the product development cycle. Sometimes for building a car, one needs to start at researching a various type of wheels and need to actually develop the step one wheel above (despite the displeasure of customers or product managers).
3 - The Independent R&D Lab Process
This brings us to another scenario; to have AI research in a separate lab track and then inject its findings into the production line when ready.
Many companies have R&D labs. This process suites many situations when there is a clear independent discovery to be made. For example, drug discovery. Production lines do not need to know anything about what is happening in the lab. When a drug passes many stages of discovery and trial. It is then "shipped" to large-scale production. AI solutions differ from this independent development as they need the data captured from the product during its development and their development changes the product and heavily impacts user experience.
Certain AI component of the product if fully modularized can be developed in the lab given appropriate time ahead of the productionization. This is especially true if AI replacing a current step of the process deployed as an expert system or a heuristic algorithm. The final product goal is clear, however, the current or older driver assistant features are more basic controllers or simpler algorithms. As products are deployed, they allow more comprehensive data to be collected and certain parts to be replaced by training better AI on this data. This should be noted that the product vision and user experience has already been taken care of and ready for full AI implementation.
The Best Process: An Integrated AI-First Approach
Lets look an example, self-driving car companies adopted a different strategy than lab or typical software development practice. In this approach, the intermediate solutions are not to deploy products but a series of carefully planned experimental setups. Graduating from Google X that can be regarded as an applied R&D lab, Waymo's approach to autonomous vehicles much more resembles that of an academic AI lab with field tests and simulations.
Similar to the scientific methodology, this approach starts with a question, a business or product idea. The cycle starts with a hypothesis on what possible solutions might be. To develop and validate the solution, one needs to first collect data, the critical ingredient of any AI solution. Data collection itself may require complex infrastructure.
With the given data, an ML/AI solution is developed. This process happens itself iteratively using research methodology used by data scientists. The solution often needs to be tested back in the real environment with appropriate metrics to evaluate its success and impact on the overall product. Proving that the metrics show a successful solution, the prototype is ready to be placed in the production environment.
The iterative approach here is independent of the product but is informed by its requirements. Often metrics used for evaluating the AI solutions are also used to monitor the product in the field. The step from prototype to production should not be a lengthy and complex process. The best practice is that the product already anticipates the arrival of the AI solution. It has built all necessary data pipelines and the input/output contracts between AI modules and the product is clear. If the data science team has had scalability in mind during the development, the productionization step is merely pressing a button to run the AI module in the scalable and distributed production environment.
Gmail Auto-Response: A Case Study
Let's make an example of the auto-response feature in Gmail. It was initially appeared in the product in the form of three buttons at the end of the email screen so one can choose one. This was not the product, I argue, but an experimental setup to determine the success of the current algorithms to generate automated responses. It is also a data collection procedure to generate a larger set of responses by humans to train the next version of the AI. This has an impact on how the actual product user experience has become. In the newer iterations, I see sentence suggestions as I type in Gmail. And this evolution yet to be completed.
Team Structure
The best way to structure a data science team is not to isolate them in a Lab but to position it as a subgroup that collaborates directly (and independently) with the engineering teams. Software teams usually adopt an agile methodology with various multi-disciplinary teams (e.g. front-end, back-end, mobile, design, etc.) handling each business or product line. In this structure, Data Science team team has clear mandates, metrics and research vision for the future of the product. New discoveries get implemented in the product rapidly while larger research topics continue to be conducted along with product development.