Netflix Keystone - Real-time Stream Processing Cloud Service by Monal Daxini

The need for gleaning answers from data in real-time from unbounded data streams is moving from nicety to a necessity. Netflix is a data driven company and Netflix Keystone Stream Processing Service processes over a Trillion messages a day with at-least once processing semantics in the cloud to derive insights. To ease extracting intelligence from unbounded streams easily we are building a multi-tenant, self-serve, real-time event processing service infrastructure so that the user can focus on extracting value and not have to worry about building infrastructure. We will share our experience * Building the service in AWS by leveraging open source software like Kafka, Samza, Flink, Mesos, and Docker * How to evolve the service inflight without downtime * Lessons learnt along the way Monal Daxini leads the Stream Processing team at Netflix building a scalable and multi-tenant real-time event processing pipeline. He has worked on Netflix's Cassandra & Dynamite infrastructure, and was instrumental in developing the encoding compute infrastructure for Netflix content. He has 15 years of experience building distributed systems at organizations like Netflix, NFL.com, and Cisco.