Papers
arxiv:2610.17641

Beyond Bias Scores: Answerability Confounds in Activation Steering with Gradient-Constrained Direction Design (GCD)

Published on Oct 13
Authors:
,
,
,

Abstract

Language-model bias evaluations conflate preference with answerability; a gradient-constrained steering method reduces answer-rate distortion while improving preference measurement, recommending joint reporting of answerability.

Bias scores in language-model evaluations often conflate what a model prefers with whether it answers at all. We identify this confound in BBQ, whose ambiguous score factorizes into answerability and signed preference. Across 11 categories, 28,}824 items, and five models, contrastive steering lowers the signed ambiguous score in 11/11 categories but lowers the signed disambiguated preference score in only 8/11. Across 99 configurations, the answerability channel exceeds half of the observed score change in 17\% and accounts for the entire change in 11%. In a six-category disambiguated evaluation, 20.8\% of 1,500 items flip to the unknown option, where abstention is incorrect; StereoSet shows no analogous answer-rate shift. The effect reflects selection and near-orthogonality between the contrastive and preference-gradient directions. We introduce Gradient-Constrained Direction Design (GCD), an option-gradient construction with a first-order unknown-logit constraint. Under a matched Qwen-7B protocol, GCD reduces answer-rate distortion 14-fold and yields a 4-fold larger preference change, while ablations show similar gains for alternative and unconstrained option-gradient directions; nulling the unknown gradient alone is not responsible for the gain. We recommend reporting answerability alongside published scores.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.17641
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.17641 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.17641 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.17641 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.