Real Time Data Warehousing: A Journey from Batch to Streaming with Faust by Manon Charvet

Faust is a Python library for building real-time data processing applications with stream-based architectures. Discover how we used it to transform one of our data processing workflows to integrate real-time events into the CERN Business Computing group's data warehouse. In this short talk, we will see how Faust was used to build an application capable of handling streaming events. We will explore Faust’s components such as pages and agents, and show the ease of creating distributed pipelines with the library. Finally, we will walk through the architecture, from the data source to the final storage database. #vdc25