567: The year of Small Models

Coder Radio

Llama 3 and Phi-3-mini are up and running on phones, Raspberry Pis, and we give them a go. Plus Google kills the vibe, and Meta opens up Horizon OS.

Sponsored By:

• Coder QA: Take $1 a month off your membership for a year, and contribute to our show directly! Promo Code: darthjarjar

Support Coder Radio

Links:

• 💥 Gets Sats Quick and Easy with Strike — Strike is a lightning-powered app that lets you quickly and cheaply grab sats in over 36 countries.
• 📻 Boost with Fountain.FM — Fountain 1.0 has a new UI, upgrades, and super simple Strike integration for easy Boosts.
• Building for our AI future — But ultimately we are a workplace and our policies and expectations are clear: this is a business, and not a place to act in a way that disrupts coworkers or makes them feel unsafe, to attempt to use the company as a personal platform, or to fight over disruptive issues or debate politics.
• Google, after protester firings, now looks just like every other company — Google's firing of 28 protesting employees Wednesday sends the clearest signal yet that the tech giant — whose founders pledged it was "not a conventional company" — has become just that.
• In the Plex: How Google Thinks, Works, and Shapes Our Lives: Levy, Steven: 9781416596585: Amazon.com: Books — Written with full cooperation from top management, including cofounders Sergey Brin and Larry Page, this is the inside story behind Google, the most successful and most admired technology company of our time, told by one of our best technology writers.
• Meta Debuts Llama 3, Latest AI Model for Powering Chatbots — Llama 3, unveiled Thursday, is an upgrade from an AI model that Meta released last summer. Chief Product Officer Chris Cox said that model, Llama 2, has been downloaded 170 billion times. Llama 2 was billed as open source, though its license included some restrictions such as requiring companies with more than 700 million users to ask for permission to use it.
• Meta's Mark Zuckerberg says energy constraints are holding back AI data center buildout - DCD — "Now I think that's getting less, and you're seeing a bunch of companies thinking 'wow we should really just invest a lot of money in building out these things.' I think that will go [on] for some period of time."
• Private LLM: Secure Offline & Local AI Chatbot for iOS & macOS — Private LLM is a local AI chatbot for iOS and macOS that works offline
• Microsoft launches Phi-3, its smallest AI model yet - The Verge — Phi-3 learned from ‘bedtime stories’ created by other LLMs.
• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• Dylan Patel on X: — Synthetic data pipelines are massive improvements over internet data Flywheel only continues with big models too when these techniques are applied
• Aran Komatsuzaki on X: "A few caveats about Phi-3:" — Phi-3-medium performs well on TriviaQA but noticeably underperforms rel. to GPT-3.5. We can guess that Phi-3 recipe doesn't magically make it understand more random factoids. It's more focused on valuable knowledges (e.g. STEM), which makes sense.
• Meta Expands Its Mixed Reality Beyond the Quest Headsets - CNET — Meta opened the Quest Horizon OS to others, and now VR headsets from Asus, Lenovo and even an Xbox variant are on the way.
• A New Era for Mixed Reality | Meta Quest Blog | Meta Store