Kafka in the Wild • Laura Schornack & Maureen Penzenik • GOTO 2021

This presentation was recorded at GOTOpia Chicago 2021. #GOTOcon #GOTOpia http://gotochgo.com Laura Schornack - Head Department Architect at Northern Trust Corporation Maureen Penzenik - Data Architect at Northern Trust Corporation ABSTRACT Cut the time of application delivery by reusing Kafka data structure between projects! Expecting boundaries and data definitions to remain consistent between source and consuming projects can be a constant source of surprise - a Kafka spiderweb. Duplicate datasets with mutations often bring with them monetary, opportunity and reputational costs, as well as errors, inconsistencies, tech debt, reactive approach to data problems, audit issues, data ownership confusion and data that is not fit for purpose. Solving this requires moving towards uniform datasets. And this moves requires an Enterprise Domain Driven Design approach, combined with Event Storming. Doing this architecture work upfront allows for real-time, ubiquitous, distributed data implementations, as opposed to a Kafka spiderweb. This talk and demo will show the design, along with a demo illustrating this approach [...] TIMECODES 00:00 Intro 01:29 Agenda 02:35 The problem 08:34 History of data segregation within apps 10:22 CQRS for operational systems 11:26 DDD for a single application – Generic example 13:14 DDD merged with event storming 13:58 Eventing diagram 15:56 Enterprise domain driven design 18:42 Operational vs Data/Analytics ecosystems 20:23 Monoliths vs Data mesh 24:45 Summary Download slides and read the full abstract here: https://gotochgo.com/2021/sessions/1761/kafka-in-the-wild RECOMMENDED BOOKS Maryann Kisamore • Basics Of Apache Kafka • https://amzn.to/3tkVYFD Ted Dunning & Ellen Friedman • Streaming Architecture • https://amzn.to/3lhk3Kb Liz Rice • Container Security • https://amzn.to/3oU4iJe Liz Rice • Kubernetes Security • https://www.oreilly.com/library/view/kubernetes-security/9781492039075 Brendan Burns, Joe Beda & Kelsey Hightower • Kubernetes: Up and Running • https://amzn.to/3wrtwlp Sam Newman • Monolith to Microservices • https://amzn.to/2Nml96E Sam Newman • Building Microservices • https://amzn.to/3dMPbOs Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• https://amzn.to/3c4HmmL Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • https://amzn.to/3fVNAb0 https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.facebook.com/GOTOConferences #ApacheKafka #Kafka #Apache #SoftwareArchitecture #DDD #EDDD #CQRS #DataSourcing #EventStorming #EventingDiagram #DomainDrivenDesign #DataMesh Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1