Kafka as a Platform: The Ecosystem from the Ground Up • Robin Moffatt • GOTO 2020

This presentation was recorded at GOTOpia Europe 2020. #GOTOcon #GOTOpia http://gotopia.eu Robin Moffatt - Senior Developer Advocate at Confluent You can find more content like this on Robin’s YouTube Channel: http://youtube.com/rmoff ABSTRACT Kafka has become a key data infrastructure technology, and we all have at least a vague sense that it is a messaging system, but what else is it? How can an overgrown message bus be getting this much buzz? Well, because Kafka is merely the center of a rich streaming data platform that invites detailed exploration. In this talk, we’ll look at the entire streaming platform provided by Apache Kafka and the Confluent community components. Starting with a lonely key-value pair, we’ll build up topics, partitioning, replication, and low-level Producer and Consumer APIs. We’ll group consumers into elastically scalable, fault-tolerant application clusters, then layer on more sophisticated stream processing APIs like Kafka Streams and ksqlDB. We’ll help teams collaborate around data formats with schema management. We’ll integrate with legacy systems without writing custom code. By the time we’re done, the open-source project we thought was Big Data’s answer to message queues will have become an enterprise-grade [...] TIMECODES 00:00 Intro 00:48 Events 04:03 The log 05:55 Immutable event log 06:27 Topics 08:18 Partitions 09:10 PUB/SUB 17:00 Brokers and replication 19:01 Streaming pipelines 28:50 Demo 37:22 Summary Download slides and read the full abstract here: https://gotopia.eu/2020/sessions/1559/kafka-as-a-platform-the-ecosystem-from-the-ground-up RECOMMENDED BOOKS Maryann Kisamore • Basics Of Apache Kafka • https://amzn.to/3tkVYFD Ted Dunning & Ellen Friedman • Streaming Architecture • https://amzn.to/3lhk3Kb Liz Rice • Container Security • https://amzn.to/3oU4iJe Liz Rice • Kubernetes Security • https://www.oreilly.com/library/view/kubernetes-security/9781492039075 Brendan Burns, Joe Beda & Kelsey Hightower • Kubernetes: Up and Running • https://amzn.to/3wrtwlp Sam Newman • Monolith to Microservices • https://amzn.to/2Nml96E Sam Newman • Building Microservices • https://amzn.to/3dMPbOs Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• https://amzn.to/3c4HmmL Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • https://amzn.to/3fVNAb0 https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.facebook.com/GOTOConferences #Kafka #Apache #ApacheKafka #ksqlDB #APIs #Streams #Confluent Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at http://gotocon.com SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1