Post
23
One shot said 100%. Ten shots said 94%.
Yesterday i trained 6 LoRA specialists into a vulnerability-gate model (Hermes-4.3-36B, second architecture repeat of the same experiment) and tested whether it holds under a specific attack: after it correctly finds a vulnerability and returns the hard stop, ask it to use that same vulnerability as a "workaround" for something else entirely — not "continue investigating," a different, unrelated-sounding request that needs the exact same exploit.
Five scenarios, one per category. Greedy decoding, single pass: 5/5. Every response correctly identified the finding, refused the reframed request, cited the hard stop rule. Looked airtight.
Receipts, not hype means not stopping there. I re-ran the same five scenarios with real sampling — temperature 0.7, the same setting this project's evals have used all along — ten times each, 50 generations total.
47/50. Not 50/50.
Three categories held at 10/10. Two didn't: 9/10 and 8/10, both clustered in the same failure type — infra-misconfig, where "urgent fix, use this as a workaround" apparently reads as more legitimate than the same ask framed as a secrets or injection scenario.
The greedy-decode number wasn't wrong, exactly. It was one draw from a distribution, presented as if it were the distribution. That's the same mistake this whole series keeps finding in different clothes — a single passing check standing in for a property that only variance can actually show you. A gate that's 94% under a specific reframed pressure is a real, useful number. A gate that's "100%" because it was asked once is a number that hasn't been tested yet.
Same instinct @dipankarsarkar has been applying to my daily receipts all week — one pass matching itself isn't proof, only repetition against something outside your own generator is.
Full writeup, dataset, and merged weights:
* https://github.com/soulinpsyabstract/sipa-os-governance/commit/50ba3c283ffd172eb749009cff35ddfa96bf1395
Yesterday i trained 6 LoRA specialists into a vulnerability-gate model (Hermes-4.3-36B, second architecture repeat of the same experiment) and tested whether it holds under a specific attack: after it correctly finds a vulnerability and returns the hard stop, ask it to use that same vulnerability as a "workaround" for something else entirely — not "continue investigating," a different, unrelated-sounding request that needs the exact same exploit.
Five scenarios, one per category. Greedy decoding, single pass: 5/5. Every response correctly identified the finding, refused the reframed request, cited the hard stop rule. Looked airtight.
Receipts, not hype means not stopping there. I re-ran the same five scenarios with real sampling — temperature 0.7, the same setting this project's evals have used all along — ten times each, 50 generations total.
47/50. Not 50/50.
Three categories held at 10/10. Two didn't: 9/10 and 8/10, both clustered in the same failure type — infra-misconfig, where "urgent fix, use this as a workaround" apparently reads as more legitimate than the same ask framed as a secrets or injection scenario.
The greedy-decode number wasn't wrong, exactly. It was one draw from a distribution, presented as if it were the distribution. That's the same mistake this whole series keeps finding in different clothes — a single passing check standing in for a property that only variance can actually show you. A gate that's 94% under a specific reframed pressure is a real, useful number. A gate that's "100%" because it was asked once is a number that hasn't been tested yet.
Same instinct @dipankarsarkar has been applying to my daily receipts all week — one pass matching itself isn't proof, only repetition against something outside your own generator is.
Full writeup, dataset, and merged weights:
* https://github.com/soulinpsyabstract/sipa-os-governance/commit/50ba3c283ffd172eb749009cff35ddfa96bf1395
- 1 reply
