Nathan Godey
nthngdy.bsky.social
Post-doc at Cornell Tech NYC Working on the representations of LMs and pretraining methods https://nathangodey.github.io
👤 Joined on 25 novembro 2024
168
Followers
30
Following
65
Total Posts
📜 Kullanıcı Geçmişi
- 25 Kas 2024, 08:06
🎉 Bluesky'e katıldı - İlk kullanıcı adı:
@nthngdy.bsky.social
🚫 Blokladığı Kullanıcılar
❤️ Beğendiği Gönderiler
🎓 Huge congratulations to @riantouchent.bsky.social who successfully defended his PhD “Improving Clinical Information Extraction with Public-Data Pretraining” on 15th October, supervised by Éric de La Clergerie and Laurent Romary.
Show Post
Simple, straightforward, and just work well: looping with no KV overhead, and this also gives you a gradients highway that makes backprop work much better
Show Post
A simple mechanism that minimizes overhead in pre-training looped transformers and learns what to store over the recursions through a gradient highway!
Show Post
Very clever! I'd love to see how well this continues to scale out to the monster-sized models!
Show Post
In the memory constrained world we are running into, these kinds of insights are key.
Show Post