← All projectsProject
PrefKit
A toolkit four of us built to ask: if you ask a model what it prefers in four different ways, do you get the same answer? Mostly, no — stated preference and actual choice diverge sharply at 14B parameters. Along the way we also caught a silent parsing bug that had been quietly misreading responses. Full paper linked below.
Preference ElicitationEvaluationLLM
Overview
A toolkit four of us built to ask: if you ask a model what it prefers in four different ways, do you get the same answer? Mostly, no — stated preference and actual choice diverge sharply at 14B parameters. Along the way we also caught a silent parsing bug that had been quietly misreading responses. Full paper linked below.