Talk - Carlos Kidman: Testing Machine Learning Models

As ML/AI systems are becoming more prevalent, the need for setting quality standards and testing practices has become crucial. Testing these models goes beyond validation metrics like accuracy, precision, and recall. Instead, quality attributes like model Behaviors, Usability, and Fairness need to be tested and measured using exploratory and automated strategies. In this talk, we'll cover some of the risks and biases that can happen throughout the MLOps pipeline, demonstrate a few techniques to test a model's behaviors and fairness, and apply them against some real-world scenarios and state-of-the-art models. By the end, you will have new ideas and techniques that you can use to test your own ML/AI systems and approach testing these quality attributes from a user's perspective.