Ai2

ai2.bsky.social

15 March 2026 21:01

🔎 Deep research agents like Asta ScholarQA are transforming how we perform literature review. But how do we know if the way we evaluate them is actually meaningful? Announcing our new paper: “Deep Research, Shallow Evaluation: A Case Study in Meta-Evaluation for Long-Form QA Benchmarks” 🧵

Juan Diego Rodriguez

juand-r.bsky.social

14 March 2026 05:13

I’m looking forward to reading this!

Greg Durrett

gregdnlp.bsky.social

14 March 2026 05:13

Check out Manya's benchmark for LLM creativity! Inspired by work on creativity in graphs (@adtraghunathan.bsky.social 's "roll the dice" paper), CREATE isolates testing of creative insights for discovery. Future: understanding how LLMs derive insights & how they can be better creative partners!

Jesse Raab

jraab.bsky.social

14 March 2026 05:12

Neat to see frameworks for evaluating this kind of stuff

Jessy Li

jessyjli.bsky.social

14 March 2026 05:12

Want to know how well the models can brainstorm connections across different concepts? Super excited about @manyawadhwa.bsky.social’s work on measuring associative creativity!