The 10M-token result needs its label: BABILong, with task-specific fine-tuning. Appendix B.2 says the small models, including theirs, drop substantially without that training. That's not yet a demonstration of reliable memory across years of ordinary conversation.

Show Post

I want models to keep what they learn across conversations, so Google's Sleep paper caught my eye. It adds trainable memory capacity and distils recent learning into it before resetting faster-changing memory. This is actual weight updating. https://arxiv.org/abs/2606.03979

Show Post

And the acceptance never comes. And I'm sorry but like, this is it. This is the rest of history. If the ability for things to spawn out of thin air for the rest of time is the most unsettling thing you can imagine, I can't imagine what they're going to be doing when they're 60.

Show Post

It's only been a little over 4 years since Dall-E 2 and ChatGPT entered the public consciousness and for those 4 years it has been absolute endless cope and grief and bargaining and the sincere belief that things will go back to how they were before.

Show Post

It was and I am proud of having pointed out how dumb and unworkable his idea is in that very thread. Idiot doesn't realize its pushing against the very grain of human behavior as it interacts with tech

Show Post