What's the Best Big Data Architecture for You? • Christoph Windheuser • GOTO 2024

This presentation was recorded at GOTO Amsterdam 2024. #GOTOcon #GOTOams https://gotoams.nl Christoph Windheuser - Building Applications Fast in High Quality, Based on Data & AI @christophwindheuser3252 RESOURCES https://twitter.com/Windheuser https://www.linkedin.com/in/windheuser http://christophwindheuser.com ABSTRACT To implement Big Data use cases like Analytics, Machine Learning or GenAI, having the right data architecture is essential. And data architectures have come a long way from the first data warehouses to modern cloud-based data and compute platforms. In this talk, we will look into the history, present and future of big data architectures. We will shed light on competing big data architecture patterns like Data Mesh, Data Lakehouse, Data Fabric and Data Vault. What are the differences, what are the main challenges to implement them in an organization? And what will the future of big data architectures look like? [...] TIMECODES 00:00 Intro 02:27 Agenda 03:48 Why big data & big data architecture (BDA)? 06:56 What is a BDA & its requirements? 10:19 History of data architectures 13:00 Different BDAs 13:22 Moder data stack 20:42 Data Lakehouse 27:51 Data Mesh 34:40 Future of BDA 42:15 Summary: 4 things to remember 43:15 Outro Download slides and read the full abstract here: https://gotoams.nl/2024/sessions/3149 RECOMMENDED BOOKS Katherine Kostereva • No-Code Playbook • https://amzn.to/475WsTo Forsgren, Humble & Kim • Accelerate • https://amzn.to/367RI5o David Farley • Modern Software Engineering • https://amzn.to/3GI468M https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #BigData #BigDataArchitecture #SoftwareArchitecture #GenAI #DataMesh #DataLakehouse #DataFabric #DataVault #SQL #ApacheSpark #DatAsAProduct #ChristophWindheuser CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks: https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1