LLMs have driven the most significant change in my career in how we can build conversational systems. Where before we thought in terms of turns, grammars, intents, slots, and deterministic actions, now we must think in terms of prompts, fine tuning, and conversations.

Since 2023, I’ve been focused on delivering a great experience for Alexa powered by LLMs. I’ve written prompts, created exemplars for positive and negative behaviors, and defined evaluation criteria that can be used to help tune model behavior to produce the best results for the user.

Along the way, I’ve learned a ton about why LLMs behave the way they do, and how to build applications that make the best use of their strengths and effectively work around their weaknesses.