AI Scaling Laws, Tech Updates, and the Future of Innovation: Good Morning from OWITH.ai

OWITH.ai - Only What's Important to Hear around AI and Tech

Good morning from OWITH.ai: the podcast that gives you only what's important to hear in AI and tech world.The evolving understanding of scaling laws in AI has led to the emergence of new scaling laws related to test time or inference time compute. Previously, scaling laws were seen as predictable mathematical laws that could determine performance based on data, parameters, and compute. However, recent large-scale models have not aligned with these predictions, leading to a reevaluation of scaling laws. These new scaling laws are seen as empirical generalizations rather than physical laws, with a focus on how long a model is trained, post-trained, or allowed to work on a problem. It is essential to avoid blindly investing in these new scaling laws, as they may not hold true for all cases and can be unreliable and expensive. New ideas and inventions in AI beyond relying solely on scaling are called for. Ultimately, AI should be viewed as part of artificial cognition rather than a one-size-fits-all solution.Moving on to the tech world, Google has upgraded their Gemini AI for premium users to include persistent memory capabilities, allowing the AI to remember user preferences across conversations. Perplexity has introduced a new shopping feature for pro users in the US, while Mistral AI has launched a new multimodal model called Pixtral Large. Intel is leading advancements in deepfake detection and improved battery life for AI PCs. Pickle is creating digital clones of users for virtual meetings, raising ethical questions about authenticity and human connection. USCC Commissioner Jacob Helberg stresses the importance of the US keeping pace with China in AI development. The balance between AI innovation and human connection is explored through examples like Coca-Cola's controversial AI-created Christmas ad. Fei-Fei Li emphasizes using AI to amplify human creativity.Innovation in AI research is crucial, as highlighted by Gary Marcus's idea that deep learning may hit a wall and new approaches may be needed. Initially ridiculed, this idea is now being accepted as self-evident by tech CEOs like Satya Nadella. Resistance to new ideas has delayed progress in AI research, with billions of dollars invested based on assumptions that Marcus was wrong. Considering new architectures in AI research is essential to make genuine progress. Marcus believes symbol-manipulation will be part of the solution to deep learning limitations. The sociology of the field and the resistance to new ideas are crucial points to consider before dismissing alternative viewpoints.OpenAI is facing a crisis as their AI models struggle with basic tasks like accurately counting letters in words. This issue has sparked debate within the company and among users on platforms like Reddit and Twitter. CEO Sam Altman has called this a "crisis of faith" in the AI community and allocated resources to address the problem. Despite mixed reactions, OpenAI plans to release a patch to improve letter-counting capabilities in their models.Theory Ventures' successful fundraising for its second fund reflects Tomasz Tunguz's ability to combine theoretical insights with practical implementation and a data-driven approach in venture capital world. Tunguz predicts major disruptions for software giants like Salesforce and Zendesk due to advancements in AI technology.Millennials are optimistic about AI in the workplace, while Alphabet's stock dropped after proposed antitrust remedies were outlined by the Justice Department. Fintech IPOs are on the horizon, and regulatory changes are affecting digital wallet providers. In other news, Fortune hosted a video roundtable discussion with industry CEOs discussing matters such as climate change, cybersecurity, and the transformative

Thanks for listening! Follow us on Twitter, Instagram and Linkedin