Big & Fast Data at PayPal • Sid Anand • YOW! 2018

This presentation was recorded at YOW! 2018. #GOTOcon #YOW https://yowcon.com Sid Anand - Chief Architect & Head of Engineering at Datazoom ABSTRACT PayPal processes more than $1B in payments every day, is an accepted method of payment in over 200 countries, and covers everything in the online payments ecosystem from issuing credit and debit cards, to supporting credit card & POS processing, facilitating checkout at online merchants, and making it easy for friends to split a dinner bill or resolve shared expenses. All of this convenience comes with the added security of some of the most advanced fraud prevention algorithms powered by huge repositories of data. Given the vast ecosystem that PayPal has been playing in over the past 20 years, PayPal has collected large amounts of payment transaction and contextual data relating to consumers, merchants, and products -- over 150PB of data to be precise. How does PayPal manage this massive amount of data while all the while extracting useful insights that can propel the business ever forward? Come to this talk to find out! Sid Anand currently serves as PayPal's Chief Data Engineer, focusing on ways to realize the value of data. Prior to joining PayPal, he held several positions including Agari's Data Architect, a Technical Lead in Search at LinkedIn, Netflix’s Cloud Data Architect, Etsy’s VP of Engineering, and several technical roles at eBay. Sid earned his BS and MS degrees in CS from Cornell University, where he focused on Distributed Systems. In his spare time, he is a maintainer/committer on Apache Airflow, a co-chair for QCon, and a frequent speaker at conferences. When not working, Sid spends time with his wife, Shalini, and their 2 kids. [...] TIMECODES 0:00 Introduction 0:57 About Me 5:03 Setting the Context 7:23 Building a Modern Day Web Site 13:58 Key Take-aways 15:30 PayPal's (Core) Architecture (Simplified) 22:28 Fast Data in Action 24:59 XA Transactions (a.k.a. 2-Phase Commits) 28:57 Fast Data Architecture 32:46 Fast Data Requirements 34:02 Fast Data Challenges 36:40 Code Deployments 38:56 Closing Thoughts RECOMMENDED BOOKS Joshua Kerievsky • Joy of Agility • https://amzn.to/3NQnGnX Joshua Kerievsky • Refactoring to Patterns • https://amzn.to/3WGxsgF Aino Vonge Corry • Retrospectives Antipatterns • https://amzn.to/3naFk84 Matthew Skelton & Manuel Pais • Team Topologies • http://amzn.to/3sVLyLQ Subramaniam & Hunt • Practices of an Agile Developer • https://amzn.to/2XjbWor https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.facebook.com/GOTOConferences #Data #BigData #FastData #PayPal #ContextualData #SoftwareDevelopment #SidAnand #Programming #YOWcon Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1