Building Uber's AI Assistant: How Genie Revolutionizes On-Call Support with Paarth Chothani from Uber

The Data Engineering Show

Journey inside Uber's innovative AI assistant "Genie" with Paarth Chotani, Staff Engineer at Uber, as he shares how they're revolutionizing on-call support using LLMs and vector search. From processing massive amounts of internal documentation to building scalable RAG pipelines, discover how Uber tackles the challenges of implementing AI assistants at scale. Get insights into the evolution from traditional chatbots to agent-based solutions, and learn practical lessons about staying current in the rapidly evolving AI landscape. Whether you're building AI-powered tools or scaling data infrastructure, this episode offers valuable perspectives on balancing innovation with real-world implementation.

• Building and scaling RAG pipelines at enterprise scale

• Evolution from traditional chatbots to AI agents

• Practical insights on data processing and vector search implementation

• Leveraging open-source technologies in production environments

• Navigating rapid technological changes in AI development

What You'll Learn:

• How Uber transformed its on-call support system by building an AI assistant that searches across internal documentation, wikis, and code
• Why combining multiple data sources with vector databases creates more accurate and contextual responses for enterprise support
• The evolution from basic RAG implementation to agent-based architecture for handling complex support scenarios
• How to scale AI processing pipelines using Apache Spark for large-scale data chunking and embedding generation
• Why customization and internal data sources are crucial for enterprise AI assistant effectiveness
• The future of AI assistants: moving from documentation lookup to automated problem resolution through multi-agent systems
• How to balance rapid AI innovation with setting realistic customer expectations in fast-moving tech environments
  

Paarth is a Staff Engineer at Uber, where he works on Michelangelo, Uber's machine learning platform. With over four years at Uber, he specializes in feature store development, online serving at scale, and GenAI implementations. He has been instrumental in developing Genie, an AI-powered on-call assistant that revolutionizes how Uber's engineering teams handle support requests and documentation access. In this episode, Paarth shares valuable insights on building and scaling RAG-based systems, vector search implementations, and the evolution of AI assistants from traditional chatbots to sophisticated agent-based solutions. His experience spanning both AWS chatbot development and current GenAI innovations at Uber offers listeners a unique perspective on the rapid advancement of AI-powered enterprise solutions.
If you enjoyed this episode, make sure to subscribe, rate, and review it on Apple Podcasts, Spotify, and YouTube Podcasts. Instructions on how to do this are here.

Quotes

"Think of Genie as your on-call assistant. Different infra teams have their Slack channels, and because these technologies are widely used, you have to wait a lot." - Paarth
"What we realized is for our engineers to really get help, data sources really should be internal only because we customize lot of these open source engines for making it work at Uber scale." - Paarth
"Instead of building a mega scale pipeline that just ingest all data sources and then keeps a central data source solution, we instead are giving users the flexibility to ingest what data sources they want." - Paarth
"We had to scale our you can say the whole infrared layer to chunk data faster to be able to create embedding set scale." - Paarth
"It almost felt like they're doing what EMR was doing. You have your Hadoop and big data technology, and we needed these pipelines to basically process all this data quickly." - Paarth
"We've even evolved from just giving you the right documentation to starting to evolve into a situation where we'll also start taking actions on your behalf." - Paarth
"That intuition that comes from building this kind of bot, I feel like that intuition came again as we were starting to see this technology come, and we're like, hey, this looks like where you can pretty much fit all these pieces together." - Paarth
"What we have seen with several use cases is agentic genie works well when designed well, when you've analyzed the problem of which type of subproblems the bot should resolve per channel, per use case." - Paarth
"I think having a problem in mind always helps that way, the energy is little bit focused and directed." - Paarth
"Whatever you're building is not enough because the expectation has already gone to the next level, so the pace is too fast right now." - Paarth

Resources

• Companies & Platforms:
• Uber - ML Platform & Engineering
• Firebolt - Cloud Data Warehouse (firebolt.io)
  

Tools & Technologies:

• Michelangelo - Uber's ML Platform
• Genie - Uber's On-Call Assistant Bot
• Cursor - Developer IDE
• OpenSearch - Vector Database
• LangGraph - Agent Framework
  

Notable Projects Mentioned:

• MetaMate (Meta)
• Query Copilot (Uber)
• Scale at AI (Meta Meetup)
  

Company Blogs:

• Uber Engineering Blog - Genie and Query Optimization articles
  

Primary Speakers:

• Paarth Chotani - Staff Engineer, Uber
• Benjamin - Firebolt
• Eldad - Firebolt

For Feedback & Discussions on Firebolt Core:

• Join Firebolt Discord Community
• Join Firebolt GitHub Discussions
• Firebolt Core Github Repository
• [email protected]

The Data Engineering Show is brought to you by firebolt.io and handcrafted by our friends over at: fame.so
Previous guests include: Joseph Machado of Linkedin, Metthew Weingarten of Disney, Joe Reis and Matt Housely, authors of The Fundamentals of Data Engineering, Zach Wilson of Eczachly Inc, Megan Lieu of Deepnote, Erik Heintare of Bolt, Lior Solomon of Vimeo, Krishna Naidu of Canva, Mike Cohen of Substack, Jens Larsson of Ark, Gunnar Tangring of Klarna, Yoav Shmaria of Similarweb and Xiaoxu Gao of Adyen.
Check out our three most downloaded episodes:

• Zach Wilson on What Makes a Great Data Engineer
• Joe Reis and Matt Housley on The Fundamentals of Data Engineering
• Bill Inmon, The Godfather of Data Warehousing