Concerto for Java & AI - Building Production-Ready LLM Apps • Thomas Vitale • YOW! 2024

This presentation was recorded at YOW! Australia 2024. #GOTOcon #YOW https://yowcon.com Thomas Vitale - Software Architect at Systematic & Author of "Cloud Native Spring in Action", CNCF Ambassador & Oracle ACE Pro @ThomasVitale RESOURCES https://bsky.app/profile/thomasvitale.com https://www.thomasvitale.com https://twitter.com/vitalethomas https://linkedin.com/in/vitalethomas https://github.com/ThomasVitale ABSTRACT Imagine you're a music composer struggling to find inspiration for a pivotal movie scene. Then, you remember you’re also a software engineer, and the solution becomes suddenly obvious. Join me in this session, where I'll demonstrate how I enhanced my music composition process by harnessing the power of Java and AI. This talk will discuss the core architectural patterns for introducing AI capabilities into an existing software system, exploring use cases like text classification, structured data extraction, semantic search, and agentic tools. The Java ecosystem is getting more and more capabilities for building AI applications. But are they ready for production? Are there any gaps? Throughout the session, I’ll build a "composer assistant" application using Spring AI to showcase how to make an LLM application production-ready. Is the developer experience affected when working locally with language models? How is observability different when it comes to tokens? Can we ensure resilience across the many integrations orchestrated by the AI? What strategies are available for deploying LLM applications? In a final twist, you’ll choose which movie scene to score, and I’ll compose and perform the music live for it, supported by AI. Will it meet the mark? There’s only one way to find out: join me in exploring the practical side of AI applications, where Java and Generative AI offer tangible solutions to real-world use cases. Aaaaand action! [...] TIMECODES 00:00 Intro 01:14 GenAI 02:18 The WHY factor 09:19 Demo 11:46 Text classification 12:00 Demo 21:01 Semantic search 22:43 Demo 26:45 Question answering with docs 27:53 Demo 33:44 Structured data extraction 34:02 Speech transcription 34:26 Demo 44:14 Final product 45:00 Outro Download slides and read the full abstract here: https://yowcon.com/brisbane-2024/sessions/3482 RECOMMENDED BOOKS Thomas Vitale • Cloud Native Spring in Action • https://amzn.to/3kLu1ns Mauricio Salatino • Platform Engineering on Kubernetes • https://amzn.to/3X14qZK Mark Heckler • Spring Boot: Up & Running • https://amzn.to/3c0NK19 Phil Winder • Reinforcement Learning • https://amzn.to/3t1S1VZ Kelleher & Tierney • Data Science (The MIT Press Essential Knowledge series) • https://amzn.to/3AQmIRg https://bsky.app/profile/gotocon.com https://twitter.com/GOTOcon https://www.linkedin.com/company/goto- https://www.instagram.com/goto_con https://www.facebook.com/GOTOConferences #Java #AI #ArtificialIntelligence #AIAssistant #LLM #ComposerAssistant #AIComposer #ThomasVitale #YOWcon CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks: https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConferences/?sub_confirmation=1