Using Serverless & ARM64 for Real-Time Observability • Liz Fong-Jones • GOTO 2024

This presentation was recorded at GOTO Chicago 2024. #GOTOcon #GOTOchgo https://gotochgo.com Liz Fong-Jones - Field CTO at Honeycomb.io @lizthegrey RESOURCES https://bsky.app/profile/lizthegrey.com https://github.com/lizthegrey https://twitter.com/lizthegrey https://linkedin.com/in/efong https://www.lizthegrey.com Links https://github.com/egonelbre/gophers https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html ABSTRACT In this talk, you will learn how a sub-second latency query engine inspired by Facebook's Scuba was extended from running in RAM only, to querying the most recent data that could still fit on local SSD, to querying months of data at a time using cloud object storage and serverless functions. We'll describe the pitfalls of managing AWS Lambda at scale, including impatience, maximum concurrency, runtime and hardware architecture configuration/experimentation, and the price/performance of renting tens of thousands of parallel workers. [...] TIMECODES 00:00 Intro 00:27 Agenda 01:07 What is lambda for? 01:37 What is retriever for? 02:31 What is Honeycomb for? 03:09 Demo 04:08 Interactive investigation of production behavior 04:44 Emphasis: interactive 07:48 Dynamic aggregation of any fields across any time range 10:31 Buy compute in 1ms units 11:08 Considerations 18:55 It's not all about the money 19:59 Software contributes significantly 21:02 Wasting less power 37:23 Yes*, do this at home 41:06 Outro Download slides and read the full abstract here: https://gotochgo.com/2024/sessions/3358 RECOMMENDED BOOKS Charity Majors, Liz Fong-Jones & George Miranda • Observability Engineering • https://amzn.to/38scbma Kelly Shortridge & Aaron Rinehart • Security Chaos Engineering • https://www.verica.io/sce-book Nora Jones & Casey Rosenthal • Chaos Engineering • https://amzn.to/3hUmuAH Mikolaj Pawlikowski • Chaos Engineering • https://amzn.to/2SQ5Olf Russ Miles • Learning Chaos Engineering • https://amzn.to/3hCiUe8 https://bsky.app/profile/gotocon.com https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #Serverless #ARM64 #x64 #Observability #MOARstorage #DevOps #DeveloperProductivity #ContinuousDelivery #FeatureFlagging #ServiceLevelObjectives #SLOs #ChaosEngineering #SRE #GenerativeCulture #LizFongJones CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks: https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1