Voxxed Days Thessaloniki 2023 - Beyond Logs: Real-Time Anomaly Detection with Machine Learning

Speaker : Fawaz Ghali Logs and traces generated by applications are valuable sources of information that can help detect issues and improve performance. However, they are often treated separately from other data, even though they are no different from the data an application works with. In this talk, we will explore a different approach: treating logs and traces as part of a scalable cloud storage repository that can be analyzed with the same techniques used for big data. By keeping all the data together, we can apply machine learning models to detect situations of interest and alert us in real-time when unwanted behavior is occurring or brewing. This approach enables intelligent monitoring that goes beyond simple threshold-based alerts and can help identify complex issues that would otherwise go unnoticed. We will discuss how to harness existing technologies to implement this approach, providing attendees with practical tips and insights that they can apply to their own projects.