Happy to share that I’m presenting 3 research projects at AIES 2025 🎉 1️⃣Gender bias over-representation in AI bias research 👫 2️⃣Stable Diffusion's skin tone bias 🧑🏻🧑🏽🧑🏿 3️⃣Limitations of human oversight in AI hiring 👤🤖 Let's chat if you’re at AIES or read below/reach out for details! #AIES25 #AcademicSky

Show Post

🚨New paper: Reward Models (RMs) are used to align LLMs, but can they be steered toward user-specific value/style preferences? With EVALUESTEER, we find even the best RMs we tested exhibit their own value/style biases, and are unable to align with a user >25% of the time. 🧵

Show Post

Given a set of values, ConflictScope generates scenarios in which an LLM-based assistant faces a conflict between a pair of values in the set. It then evaluates which value a target LLM supports more in each scenario before combining scenario-level judgments into a value ranking.

Show Post

We introduce new metrics to measure how morally challenging a dataset is for models. We find that ConflictScope produces datasets that elicit more disagreement and stronger preferences than moral dilemma datasets, while alignment data frequently elicits indifference from models.

Show Post

We find significant shifts between models’ expressed and revealed preferences under conflict! Models say they prefer actions that support protective values (e.g. harmlessness) when asked directly, but support personal values (e.g. helpfulness) in more realistic evaluations.

Show Post