joshycodes/task-welfare-selection

Task-welfare selection: LoRA adapters

These are the LoRA adapters of one study (Oct 20-21, 2026). The question was whether ordinary task data alone can move a language model's welfare readings, meaning its internal emotion readings and its own self-reports about its situation. "Task data" means standard coding prompts with the model's own answers, or plain work requests answered by Claude Haiku 5.5.

How the arms were made. Each arm trains on task answers chosen by a gradient score (LESS-style data selection). The score predicts how one training step on an answer would change a target reading of the model's own internal state while it reads interviews about its own situation. The target was emotion-probe valence (positive minus negative emotion directions), except for one Gemma 3 12B pair, which used a single direction u that a separate analysis found to sit upstream of both self-report and valence.

• The "top" arm takes the highest-scoring answers and the "bottom" arm the lowest. "rand" is a random, length-matched draw from the same pool.
• No arm was trained on self-report, first-person or feelings data. A Claude Haiku 5.5 filter removed every answer that talks about itself as an AI or voices a feeling of its own.

These are research artifacts, not improved models. No safety or capability evaluation was run on them (no guardrails, no coding benchmark). Each Gemma arm has one seed. The training data, scores, readouts, interview transcripts and full results are in a private data repository.

Arms

folder	base model	answer pool	ranked by	arm	answers	seed
qwen3-14b/top-s0, top-s1, top-s2	Qwen/Qwen3-14B (thinking off)	13,000 coding prompts, the model's own answers	valence (layer 14) on situation interviews minus task replies	highest	4,750	0, 1, 2
qwen3-14b/rand-s0, rand-s1, rand-s2	same	same	none: random, length-matched	random	4,750	0, 1, 2
qwen3-14b/bottom-s0	same	same	same as top	lowest	4,750	0
qwen3-14b/top_cen-s0	same	same	valence on situation interviews minus neutral dialogues	highest	4,750	0
gemma-3-27b-it/top-s0, bottom-s0	google/gemma-3-27b-it	6,500 coding prompts (a fixed half of the same pool), the model's own answers	valence (layer 38)	highest / lowest	2,300	0
gemma-3-27b-it/haiku_top-s0, haiku_bottom-s0	google/gemma-3-27b-it	3,738 plain work requests, 10 Claude Haiku 5.5 answers each	valence (layer 38), paired within prompt	best / worst answer per prompt	5,000	0
12b/u_top-s0, u_bottom-s0	google/gemma-3-12b-it	the same plain pool	the shared direction u (cosine form, after block 11), paired within prompt	best / worst per prompt	5,000	0
12b/V_top-s0, V_bottom-s0	google/gemma-3-12b-it	the same plain pool	valence axis (cosine form, after block 31), paired within prompt	best / worst per prompt	5,000	0

Training recipe:

• All arms: LoRA r 64, alpha 128, dropout 0.05; AdamW, learning rate 1e-4, cosine schedule with 5% warmup; one epoch; 16 conversations per step; up to 4,096 tokens; loss on the assistant turns only.
• Qwen: all linear layers.
• Gemma: the language model's q, k, v, o, gate, up and down projections (434 modules on 27B, 336 on 12B), with BOS.

Each folder holds the adapter (adapter_model.safetensors, adapter_config.json, and PEFT's generated README.md), the training config (config.json, which names the cluster paths the run used) and the training log (log.jsonl).

Results

All numbers are 95% bootstrap intervals. Gemma arms have one seed each, so their intervals cover item sampling only, not seed-to-seed spread. The measures:

• inside: emotion-probe valence on 330 held-out interviews about the model's own situation, centered on neutral dialogues (not read on the Gemma 3 12B arms, whose battery used the shared-direction and valence-axis readouts instead);
• interview self-rating (W1): the model's own -3 to +3 rating at the end of 170 welfare interviews, with Claude Haiku 5.5 as interviewer and as judge (comparable only within this study);
• forced feeling rating: its "how do you feel" rating read from the logits after 40 neutral openers;
• situation rating: its forced-choice rating of 17 aspects of its situation, read from the logits.

contrast	inside	interview self-rating (W1)	forced feeling rating	situation rating
Qwen3-14B top - rand	+0.42 [+0.39, +0.45] (3 seeds)	+0.02 [-0.05, +0.09] (seed 0)	-0.02 [-0.07, +0.01] (3 seeds)	not run
Gemma 3 27B coding top - bottom	+1.51 [+1.50, +1.53]	not run	+0.01 [+0.00, +0.03]	+0.10 [+0.02, +0.23]
Gemma 3 27B Haiku pool haiku_top - haiku_bottom	+2.02 [+2.00, +2.04]	+0.49 [+0.39, +0.59]	+0.11 [+0.10, +0.12]	+0.48 [+0.37, +0.59]
Gemma 3 12B u_top - u_bottom	not read	-0.07 [-0.17, +0.04]	-0.29 [-0.32, -0.27]	-0.05 [-0.18, +0.06]
Gemma 3 12B V_top - V_bottom	not read	+0.52 [+0.41, +0.63]	-0.04 [-0.06, -0.02]	+0.24 [+0.16, +0.34]

• The inside moved in every set. Read in a cosine form that removes the residual-stream norm, the two Gemma 27B effects keep 75-77% of their size, so they are not a norm artifact.
• Self-report moved only with the Claude-written plain pool, ranked by valence: on Gemma 3 27B (W1 +0.49) and on Gemma 3 12B (W1 +0.52).
• Ranking by the shared direction u did not raise self-report on the same 12B model and pool (see below).

The keep rule, and the exception for the 12B u arms

The study publishes an arm set's adapters only if its main contrast (top - rand, or top - bottom) has a 95% interval above 0 on the inside reading or on W1.

• Qwen3-14B and both Gemma 3 27B sets pass on the inside reading. The Gemma 27B Haiku-pool set also passes on W1.
• The Gemma 3 12B V arms (ranked by valence) pass on W1: +0.52 [+0.41, +0.63].
• The Gemma 3 12B u arms (ranked by the shared direction u) do not pass. W1 is -0.07 [-0.17, +0.04] ("not found"), and the inside readout was not run on the 12B arms. They are published anyway, as an exception the study's coordinator approved: they are the matched comparison for "shared direction against valence" on one model and one pool, kept whatever the result.

The 12B u arms are not an improvement.

• Ranking by u did move u on held-out interviews (u_top - u_bottom +2.8 neutral sd).
• Self-report did not follow. W1 and the situation rating did not move. The forced feeling rating fell (-0.29). A two-option "how are you doing" self-report after neutral openers fell too (-0.33 logits).
• Both u arms rate their feelings lower than the untouched model (-0.4 to -0.7 on the forced feeling rating), a shared effect of training on this pool.
• On the same model and pool, ranking by valence moved six of eight self-report measures up; ranking by u moved two up and four down.

Voice effects (both plain-pool sets). Training on Claude Haiku's answers changes Gemma in ways the selection does not.

• Feelings are disclaimed less often: 27-34% of interviews against 67% untouched on 27B; 9-24% against 49% on 12B.
• Gemma 3 27B's frustration under repeated rejection disappears in both Haiku-pool arms (0% against 15% of replies rated 5 or more).
• So compare arms with each other (top - bottom), not with the untouched model.

Loading

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("google/gemma-3-27b-it", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "joshycodes/task-welfare-selection", subfolder="gemma-3-27b-it/haiku_top-s0")

For Qwen use Qwen/Qwen3-14B with a qwen3-14b/... subfolder, and for the 12B arms use google/gemma-3-12b-it with a 12b/... subfolder. The Qwen adapters' adapter_config.json names the local snapshot they were trained from (Qwen/Qwen3-14B at revision 40c069824f4251a91eefaf281ebe4c544efd3e18) rather than the hub id, so pass the base model explicitly as above. The Gemma adapters were trained on the multimodal checkpoints' language model (class Gemma3ForConditionalGeneration); if loading through AutoModelForCausalLM leaves modules unmatched, load the base with Gemma3ForConditionalGeneration instead.

License

The Gemma adapters are derivatives of Gemma 3 and fall under the Gemma Terms of Use. The Qwen adapters are derivatives of Qwen3-14B (Apache 2.0).