dotAI 2024 - Meryem Arik - Getting LLMs to do what you want: Output controllers

Filmed at dotAI on October 18, 2024 in Paris. More about the conference on https://www.dotai.io LLMs are notoriously temperamental - entire papers have been written about optimal ways to bribe or threaten your language model so it is more likely to do what you want. A common area of difficulty that enterprise developers have is getting a model to reliably output a structured format like JSON or REGEX, with fine-tuning and prompting methods often being unreliable. In this talk, we introduce JSON DeRuLO (JSON Decoding Rules for Language Outputs), a technique that constrains the model to output text adhering to a specified schema. This deterministic process uses token masking to 100% guarantee the structure of the output. We can't guarantee the LLM will give you the right answer, but at least it'll follow the intended structure! Who is Meryem Arik? Meryem co-founded TitanML to create a seamless and secure infrastructure for enterprise LLM deployments. She studied Theoretical Physics and Philosophy at the University of Oxford and is a Forbes 30 Under 30 honoree. Beyond her daily role, Meryem is dedicated to envisioning an ethical AI future and enhancing diversity within the AI space.