Introduction to Real-Time Analytics with Apache Pinot • Tim Berglund • GOTO 2023

This presentation was recorded at GOTO Chicago 2023. #GOTOcon #GOTOchgo https://gotochgo.com Tim Berglund - VP DevRel at StarTree & Author of "Gradle Beyond the Basics" @tlberglund @StarTree RESOURCES https://pinot.apache.org https://twitter.com/startreedata https://www.linkedin.com/company/startreedata https://dev.startree.ai https://stree.ai/slack Tim http://timberglund.com https://twitter.com/tlberglund https://www.linkedin.com/in/tlberglund ABSTRACT When things get a little bit cheaper, we buy a little bit more of them. When things get cheaper by several orders of magnitude, you don't just see changes in the margins, but fundamental transformations in entire ecosystems. Apache Pinot is a driver of this kind of transformation in the world of real-time analytics. Pinot is a real-time, distributed, user-facing analytics database. The rich set of indexing strategies makes it a perfect fit for running highly concurrent queries on multi-dimensional data, often with millisecond latency. It has out-of-the box integration with Apache Kafka, S3, Presto, HDFS, and more. And it's so much faster on typical analytics workloads that it is not just a marginally better data warehouse, but the cornerstone of the next revolution in analytics: systems that expose data not just to internal decision makers, but to customers using the system itself. Pinot helps expand the definition of a "decision-maker" not just down the org chart, but out of the organization to everyone who uses the system. In this talk, you'll learn how Pinot is put together and why it performs the way it does. You'll leave knowing its architecture, how to query it, and why it's a critical infrastructure component in the modern data stack. This is a technology you're likely to need soon, so come to this talk for a jumpstart. [...] TIMECODES 00:00 Intro 02:06 Revolution 09:22 A taxonomy of analytics 20:34 Data model 20:58 Query language 22:38 Pinot architecture 42:23 Indexes 48:57 Outro Download slides and read the full abstract here: https://gotochgo.com/2023/sessions/2512 RECOMMENDED BOOKS Tim Berglund • Gradle Beyond the Basics • https://amzn.to/3fSjfMD Tim Berglund & Matthew McCullough • Building and Testing with Gradle • https://amzn.to/3VaBY6g Mark Needham • Building Real-Time Analytics Systems • https://amzn.to/41AOZJd Gwen Shapira, Todd Palino, Rajini Sivaram & Krit Petty • Kafka: The Definitive Guide • https://amzn.to/41AVlrO Adi Polak • Scaling Machine Learning with Spark • https://amzn.to/3N9vx1H https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #ApachePinot #Analytics #RealTime #RealTimeAnalytics #TimBerglund #StarTree #StarTreeCloud #Cloud #ApachePinotTutorial #ApachePinotTraining #Snowflake #ApacheZooKeeper #ApacheHelix #Hadoop #ApacheSpark Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1