Keshav Ramji

Aparições nos rankings

Os melhores dias em que posts desta conta entraram na lista diária do Debya.

Esta conta ainda não apareceu em um ranking diário.

Posts curtidos recentemente

Curtidas públicas da rede Bluesky.

• Giovanni Monea@giomonea.bsky.social06/10/2026 16:16

  Very excited to finally share my IBM work. It was an intense but rewarding summer, and I had a blast working with my talented collaborators at IBM & Cornell ( @keshavramji.bsky.social @yelkurdi.bsky.social @lastrasl.bsky.social @yoavartzi.com @nthngdy.bsky.social @ramon-astudillo.bsky.social )

  Ver post
• Giovanni Monea@giomonea.bsky.social06/10/2026 16:16

  So, what does memory sharing bring to the table to *improve* quality? Prediction recursions attend heavily to the shared memory. Each read sends a gradient signal back to the state recursion. We find that this "gradient highway" makes the state recursion learn much more.

  Ver post
• Giovanni Monea@giomonea.bsky.social06/10/2026 16:16

  We train LPT and Hybrid LPT at 150M-1B parameters, on 20-50B tokens at 2, 3, and 5 recursions. The more recursions, the more our memory-sharing models outperform their respective baselines.

  Ver post
• Giovanni Monea@giomonea.bsky.social06/10/2026 16:16

  Prior work laid the foundation for looped models and reduced their extra memory at inference. The result? Surprisingly strong despite the train-test mismatch. That made us ask: could this be a good inductive bias? One recursion memorizes, the others reuse it to predict.

  Ver post
• Giovanni Monea@giomonea.bsky.social06/10/2026 16:16

  What is memory sharing? Our simple recipe: at later recursions, just concatenate the attention keys of the first recursion and those of the current recursion before performing standard attention. Then, restrict the later recursions to a short window for memory savings.

  Ver post

Entre para ver mais

Contas que bloqueia

No Bluesky, bloqueios são registros públicos.

Esta conta não bloqueia ninguém.