Trust at Scale: Security and Governance for Open Source Models // Hudson Buzby // #338

MLOps.community

Trust at Scale: Security and Governance for Open Source Models // MLOps Podcast #338 with Hudson Buzby, Solutions Architect at JFrog.

Appreciate JFrog for their support in bringing this blog to life.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

For better or for worse, machine learning has traditionally escaped the gaze of security and infrastructure teams, operating outside traditional DevOps practices and not always adhering to organizations' development or security standards. With the introduction of open source catalogs like HuggingFace and Ollama, a new standard has been established for locating, identifying, and deploying machine learning and AI models. But with this new standard comes a plethora of security, governance, and legal challenges that organizations need to address before they can comfortably allow developers to freely build and deploy ML/AI applications. In this conversation, we will discuss ways that enterprise-scale organizations are addressing these challenges to safely and securely build these development environments.

// Bio

Hudson Buzby is a solution engineer with an emphasis on MLOps, LLMOps, Big Data, and Distributed Systems, leveraging his expertise to help organizations optimize their machine learning operations and large language model deployments. His role involves providing technical solutions and guidance to enhance the efficiency and effectiveness of AI-driven projects.

// Related Links

https://www.youtube.com/channel/UCh2hNg76zo3d1qQqTWIQxDg

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Hudson on LinkedIn: /hudson-buzby/

Timestamps:[00:00] Value of Centralized Gateway[00:35] Point Break vs Big Lebowski[01:47] AI adoption failure stats[05:12] ML vs Generative AI[12:04] LLM adoption in enterprise[18:08] MLOps Community alternative[23:43] AI governance challenges[27:39] Organizational debt comparison[31:41] AI tool sprawl[35:59] MLOps to platform evolution[40:56] MLOps then vs now[49:48] Model trust and safety[52:19] AI model effectiveness[55:54] Product discovery process[58:38] Wrap up