How Scale Makes Distributed Systems Slower • Jonathan Magen • GOTO 2024

This presentation was recorded at GOTO Chicago 2024. #GOTOcon #GOTOchgo https://gotochgo.com Jonathan Magen - Computer Scientist with 1.5 Decades of Professional Experience @yonkeltron ORIGINAL TALK TITLE Swimming Through Molasses: How Scale Makes Distributed Systems Slower RESOURCES https://jawns.club/@yonkeltron https://twitter.com/yonkeltron https://www.linkedin.com/in/jonathan-magen-9848711b https://github.com/yonkeltron https://yonkeltron.com Links https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://arxiv.org/pdf/1901.01930 https://dsf.berkeley.edu/papers/datalog2011-dedalus.pdf https://percival.ink https://youtu.be/ztY1YRiaSiE ABSTRACT Have you ever noticed that some systems don't seem to perform better as you scale them? No matter how many machines you throw at the problem, it never seems to get materially faster. Sometimes it feels like swimming through molasses. Have you ever wondered why? This talk will explore such phenomena by discussing how coordination and synchronization protocols introduce costly overhead. We will examine experimental evidence from both academic settings and industrial sources. Together, we will learn about mental and conceptual models to help structure problems such to prevent runtime slowdowns. Come for the distributed systems content, stay for the pragmatic solutions. [...] TIMECODES 00:00 Intro 02:28 Scalability! But at what COST? 08:07 Scale-Originated Coordination Overhead (SOCO) 15:12 What have we learned so far? 15:34 Where do we go from here? 15:47 Keeping CALM: When Distributed Consistency is Easy 26:37 Dedalus: Datalog in Time & Space 30:38 From Datalog to Dataflow 36:51 Conclusion 38:06 Outro Download slides and read the full abstract here: https://gotochgo.com/2024/sessions/3438 RECOMMENDED BOOKS Unmesh Joshi • Patterns of Distributed Systems • https://amzn.to/430yJ5G Darnell, Harrison & Seldess • CockroachDB: The Definitive Guide • https://amzn.to/3nL8YCC Guy Harrison • Next Generation Databases • https://amzn.to/3IoK5WL Burns, Beda & Hightower • Kubernetes: Up & Running • https://amzn.to/3sueuuI Jez Humble & Dave Farley • Continuous Delivery • https://amzn.to/3ocIHwd https://bsky.app/profile/gotocon.com https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #DistributedSystems #Patterns #DeclarativeProgramming #ImperativeProgramming #Datalog #DDlog #Dataflow #CockroachDB #Kubernetes #ApacheKafka #YugabyteDB #MongoDB #Cassandra #Neo4j #CloudNative #Cloud #Microservices #Programming #Raft #Paxos #JonathanMagen CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks: https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1