mr. TIM

timkellogg.me

07 June 2025 21:18

seems like reasoning models are basically, rough draft + final draft i wonder if you could get decent writing by RL’ing an LLM into a loop of “draft => critique” until it was satisfied i wonder if it would lose some of the artifacts we associate with LLMs

Boo5

boy-5.bsky.social

07 June 2025 21:18

FAIR RL’d llama into being continuously better at coding that way. I think they’re looking for something more than a transformer now though.

Boo5

boy-5.bsky.social

07 June 2025 21:16

Well they cancelled Behemoth. They went in big on scale and didn’t get much noticeably better than Llama 3.3. They could just be re-examining their training approach to transformers but they’ve also released recent work on continuous thought. There’s a shift coming

mr. TIM

timkellogg.me

07 June 2025 21:16

Yann’s finally getting his JEPA? not a chance

mr. TIM

timkellogg.me

07 June 2025 21:16

i saw this months ago and i’m still thinking about it so much of my own personal breakthroughs are system 3 also. it’s not reasoning in the normal sense, a lot of the times i can’t even explain it intuition. slow intuition