Training & Monitoring AI - Drift Detection • Thomas Viehmann • GOTO 2022

This presentation was recorded at GOTO Copenhagen 2022. #GOTOcon #GOTOcph https://gotocph.com Thomas Viehmann - Co-Author of "Deep Learning with PyTorch" @PyTorch RESOURCES http://torchdrift.org http://driftdash.de https://twitter.com/thomasviehmann https://github.com/t-vi https://linkedin.com/in/tviehmann http://thomas.viehmann.net https://lernapparat.de ABSTRACT Do our deployed AIs work as expected? Before we deploy AI models, we extensively validate them to make sure they perform well. But how confident are we that we get comparable performance after deployment? How can we check whether the assurances we took from our validation are still applicable? To answer these questions and sleep well again, drift detection is a key aspect of monitoring deployed models. It checks whether the information flowing through our models - either probed at the input, output or somewhere in-between - is still consistent with the one it was trained and evaluated on. Thomas shows things that can go wrong when we neglect drift detection, looks at how drift detection works, and then demonstrates how effective drift detection can be implemented in practice. He draws on his experience with implementing the open-source TorchDrift.org, the turn-key solution DriftDash.de, as well as consulting with clients to implement drift detection in their processes. [...] TIMECODES 00:00 Intro 05:12 Agenda 05:53 Why detect drift? 12:51 Statistics of detecting drift 21:21 Practical drift detection 32:02 Conclusion 33:52 Outro Download slides and read the full abstract here: https://gotocph.com/2022/sessions/2169 RECOMMENDED BOOKS Thomas Viehmann, Eli Stevens & Luca Antiga • Deep Learning with PyTorch • https://amzn.to/41wLNgD Stefan Helzle • Low-Code Application Development with Appian • https://amzn.to/3rAC0Xq Phil Winder • Reinforcement Learning • https://amzn.to/3t1S1VZ Kelleher & Tierney • Data Science (The MIT Press Essential Knowledge series) • https://amzn.to/3AQmIRg Lakshmanan, Robinson & Munn • Machine Learning Design Patterns • https://amzn.to/2ZD7t0x https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.facebook.com/GOTOConferences #PyTorch #DriftDetection #MonitoringAI #AI #ML #DataScience #ArtificialIntelligence #DeepLearning #NeuralNetworks #MathInf #TorchDrift #DriftDash #ThomasViehmann #Programming #SoftwareEngineering #SoftwareDevelopment Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1