Hans-Peter Grahsl - Taming IoT Data: Making Sense of Sensors with SQL Streaming

We are living in a data streaming era, yet until recently it has been particularly hard to leverage existing stream processing technologies. On the one hand, because dealing with data in motion has its inherent challenges. On the other hand, most frameworks and APIs which are allowing for stream processing are typically very hard to employ and/or operate – NOT so for KSQL, the newest kid in Apache Kafka’s ecosystem. Based on a simplified version of an IoT use case this session gives a gentle introduction into KSQL – Kafka’s SQL streaming engine for the masses. Join this fast-paced tour during which we are discussing a streaming IoT architecture. Concretely, we are going to: (1) ingest smart home energy data into Apache Kafka, (2) use KSQL for flexible, powerful and scalable SQL-only stream processing, (3) send raw data as well as pre-processed results to an operational NoSQL data store, (4) reactively serve data to clients in near real-time and (5) finally build informative live charts.