Oskar 🕊️

austegard.com

10 April 2026 01:40

Important research: user-questions are stochastic and even the best and newest models are framing-succeptible. This matters remendously in fields like medicine! But I believe prompting can mitigate if not eradicate the issue -- see bsky.app/profile/aust...

Oskar 🕊️

austegard.com

09 April 2026 21:14

3/3 But perhaps most importantly: this type of excellently documented research can now be very quickly, cheaply and effectively verified, and extended on by use of the very same AI models that are being tested. I look forward to reviewing more of your and coauthors' research!

Oskar 🕊️

austegard.com

09 April 2026 21:14

2/3 The results seem to indicate, unsurprisingly, that prompting can indeed help the situation, and also that bigger models do better, but also -- using an LLM as a Judge is itself very sensitive to the model used (tl;dr: don't use Haiku or similar cheap models).

Oskar 🕊️

austegard.com

09 April 2026 21:14

Nice research! You may be interested in the small scale ($4 budget) verification performed by my personal Opus agent here: muninn.austegard.com/blog/this-tr... in which we also introduced a framing-resistant prompt to see how much that would mitigate the effetcs. 1/3

Mark (reluctant scrivener)

markmadsen.bsky.social

09 April 2026 15:45

That was not a criticism of the research, I like this paper. It's a criticism of the credulous managers in organizations who think the magic software will replace decision making.