Fabian Schaipp

fschaipp.bsky.social

14 February 2025 05:32

Learning rate schedules seem mysterious? Why is the loss going down so fast during cooldown? Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization. A short thread on our latest paper 🚞 arxiv.org/abs/2501.18965

Pierre Ablin

pierreablin.bsky.social

14 February 2025 05:28

Excited to share Soup-of-Experts, a new neural network architecture that, for any given specific task, can instantiate in a flash a small model that is very good on it. Made with ❤️ at Apple Thanks to my co-authors David Grangier, Angelos Katharopoulos, and Skyler Seto! arxiv.org/abs/2502.01804

Jeff Dean

jeffdean.bsky.social

14 February 2025 05:28

OK! My Google colleague Thang Luong shared some exciting updates about AlphaGeometry2! AG2 now has surpassed the average gold-medalist in solving Olympiad geometry problems, w/ a solve rate of 84% compared to 54% previously! Paper: arxiv.org/abs/2502.03544 See full list of authors on link

Pablo Samuel Castro

pcastr.bsky.social

14 February 2025 05:27

careless careless whisper

Kevin J Miller

kevinjmiller.bsky.social

14 February 2025 05:24

Cognitive models of behavior are a key part of neuroscience. But discovering them is hard! In new work from Google DeepMind Neuroscience and collaborators @hhmijanelia.bsky.social, we demonstrate an approach that uses LLMs and large datasets to discover models automatically.