Ben Prystawski

benpry.bsky.social

26 June 2026 18:05

This is joint work with my fantastic collaborators @kushinm.bsky.social, @danielwurgaft.bsky.social @linasnasvytis.bsky.social, Michael Y. Li, @noahdgoodman.bsky.social, and @mcxfrank.bsky.social. You can read the preprint here: arxiv.org/abs/2606.26460 9/

Ben Prystawski

benpry.bsky.social

26 June 2026 18:05

We believe that auto-psych, and systems like it, unlock the ability to explore large spaces of possible cognitive models. But we’re not suggesting that humans be taken out of the scientific loop. 7/

Ben Prystawski

benpry.bsky.social

26 June 2026 18:05

Next, we let the system gather real human data by running experiments on Prolific! We ran three independent replicates of auto-psych, each of which posted three experiments sequentially. By the end, each found a model that beat the seed models, with *zero human intervention*! 6/

Ben Prystawski

benpry.bsky.social

26 June 2026 18:05

We first tested whether auto-psych could recover true cognitive models by observing data generated by these models. Our model recovery simulations recovered both “seed” models inspired by prior work or “alien” models designed to behave in non-human ways. 5/

Ben Prystawski

benpry.bsky.social

26 June 2026 18:05

A key part of our framework is two nested loops: - an outer loop for designing and running experiments - an inner loop for critiquing and improving cognitive models. Once an experiment has been run, the system iterates on the data to improve its models. 4/