code-daemon-denoise-v1

A bilingual (EN + RU) word filter: given one word form, it answers whether that word is a meaningful technical term worth keeping in a search vocabulary, or ballast to drop.

It is deliberately small and one-purpose. A frozen multilingual-e5-small encoder produces a 384-dim vector, and a single trained affine turns that vector into P(keep). No fine-tuning of the encoder, no classification head with its own weights to load — the entire learned decision is 384 numbers and a bias, shipped as a 6 KB JSON file.

That buys throughput: ~800 words/sec on a CPU core, ~17 800/sec on a laptop GPU.

emb = session.run(None, {"input_ids": ids, "attention_mask": mask})[0]   # [B, 384] pooled + L2
p_keep = 1 / (1 + np.exp(-(emb @ w + b)))                               # the whole classifier

---

1. What it is for

Vocabulary hygiene. Harvest every word form out of a codebase — identifiers, doc prose, comments, commit messages — and most of what you get is not worth indexing: inflected function words, chopped identifier fragments, transliteration noise, boilerplate. Keeping them inflates a search vocabulary and dilutes term statistics; dropping them by frequency alone throws away rare-but-real technical terms, which are exactly the ones worth searching for.

This model makes that call per word, in both English and Russian, at a rate that keeps up with a full-repository scan.

Suited to

• Filtering a harvested vocabulary before indexing.
• Any per-token keep/drop decision over short, single-word inputs.
• Mixed EN/RU corpora — including Cyrillic identifiers and comments.

Not suited to

• Sentences or phrases. Inputs are single word forms; the sequence budget is 40 tokens.
• Languages outside Latin/Cyrillic scripts — the vocabulary was pruned to those on purpose.
• Domain term-vs-stopword calls outside software; the label set is technical-corpus flavoured.

---

2. Architecture

Encoder	intfloat/multilingual-e5-small — XLM-RoBERTa, frozen, unchanged
Embedding dim	384, mean-pooled and L2-normalised inside the graph
Vocabulary	142k pieces, pruned from 250k by character class (Latin + Cyrillic + punctuation)
Classifier	one affine: P(keep) = sigmoid(w·e + b), w ∈ ℝ³⁸⁴
Sequence	40 tokens, batch 64
Inputs	input_ids, attention_mask
Output	[batch, 384] — pooled, normalised, ready for the dot product

Two decisions that make it small

The encoder is frozen. The head is a logistic regression fitted on top of fixed embeddings, then folded — its StandardScaler and the LR coefficients are multiplied out into a single (w, b) pair. There is no scikit-learn at inference, and no second model to keep in sync: the decision boundary is a dot product you can apply in any language.

The vocabulary is pruned by script. Cutting the 250k multilingual SentencePiece table to the Latin + Cyrillic + punctuation pieces removes 43% of the rows, and the embedding table is most of this model's weight. The pruned-vocab id remap is baked into the graph as a Gather at the input, so callers still feed ordinary SentencePiece ids and never see the mapping. INT8 weights drop from ~121 MB to **76 MB** — lossless for the two languages it targets, because nothing outside those scripts was reachable anyway.

The "vocab: " prefix

Words are embedded with a fixed "vocab: " prefix. The head was trained on prefixed embeddings, so reproduce the prefix for standalone use or the decision boundary will not line up.

---

3. How it was made

1. Encoder — export the frozen mE5-small to ONNX with mean-pooling and L2-norm fused into the graph, prune the embedding table to the kept character classes, and PTQ-quantize to INT8 (NNCF).
2. Head — embed a bilingual labelled word set (English: WordNet / BNC mid-frequency lemmas; Russian: Taiga / OpenCorpora / Nerus mid-Zipf) plus per-language hand-checked gold, fit LogisticRegression(class_weight="balanced"), then fold the scaler and the LR into one affine.

strip_threshold (default 0.95) sets where you cut. It is high on purpose: dropping a real technical term is the expensive error, keeping a bit of ballast is not.

---

4. Speed

Measured on one laptop: Intel Core Ultra 9 275HX / NVIDIA RTX 5060 Laptop, batch 64 × seq 40.

lane	per batch	throughput	per word
TensorRT FP16, RTX 5060 Laptop	3.60 ms	17 790 words/s	0.056 ms
OpenVINO INT8, iGPU (OV 2026.3)	56.1 ms	1 140 words/s	0.88 ms
OpenVINO INT8, CPU (OV 2026.3)	75-83 ms	770-850 words/s	1.18-1.30 ms
OpenVINO INT4, NPU (OV 2026.3)	57.8 ms	277 words/s	3.61 ms
ONNX Runtime FP32, CPU	188 ms	341 words/s	2.93 ms

The NPU number is per batch 16, not 64 — it is a low-batch part, so its per-word cost is the highest of the three even though its per-batch latency looks similar to the CPU's. Running all three Intel devices at once yields ~87 % of the sum of their solo rates (they share one memory controller), so a host with no discrete GPU can still denoise ~2 000 words/s.

The INT8 CPU lane is the intended default — 800 words/sec is enough to filter a repository's whole harvested vocabulary in seconds without touching a GPU, and it is 2.3× the unquantized ONNX path. The GPU lane exists for hosts that have spare VRAM anyway.

---

5. Standalone use

import json, numpy as np, onnxruntime as ort, sentencepiece as spm

sp   = spm.SentencePieceProcessor(model_file="sentencepiece.bpe.model")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
head = json.load(open("denoise_head.json"))            # {dim, w[384], b, strip_threshold}
w, b, thr = np.array(head["w"], np.float32), head["b"], head["strip_threshold"]

def p_keep(words, max_len=40):
    toks = [[2, *sp.encode("vocab: " + x)[: max_len - 2], 3] for x in words]   # bos … eos
    L    = max(len(t) for t in toks)
    ids  = np.array([t + [0] * (L - len(t)) for t in toks], dtype=np.int64)    # pad = 0
    mask = (ids != 0).astype(np.int64)
    emb  = sess.run(None, {"input_ids": ids, "attention_mask": mask})[0]       # [B, 384]
    return 1.0 / (1.0 + np.exp(-(emb @ w + b)))

scores = p_keep(["mutex", "tensorrt", "пожалуйста", "asdfgh"])
keep   = scores >= thr

The ONNX bakes in the fairseq +1 id offset and the pruned-vocab remap, so feed raw SentencePiece ids — do not remap them yourself.

---

6. Evaluation

On a frozen held-out bilingual word set, at strip_threshold = 0.95:

metric	value
SAFE (keep) F1	0.79
BALLAST (drop) F1	0.84
Strip precision	0.88

The INT8 vocab-pruned build scores the same as the full-vocabulary FP build (F1 0.79 vs 0.79) at 38% of the size — the pruning removes rows the two target languages never reach, so there is nothing to lose by it.

Strip precision is the number to watch if you tune the threshold: it says how often a word the model drops really was ballast.

---

7. What is in this repo

• OpenVINO INT8 — code-daemon-denoise-v1-s_ov2026.3_{cpu,igpu_lnl}_int8_b64_s40.{xml,bin} — the default lane (CPU) and an Intel iGPU build.
• OpenVINO INT4, NPU — code-daemon-denoise-v1-s_ov2026.3_npu_int4_b16_s40.{xml,bin} — weight-only INT4 at batch 16 for Intel NPUs.
• TensorRT FP16 — code-daemon-denoise-v1-s_{win_x64,linux_x64}_trt11.0_sm_120.engine.
• TVM Vulkan — code-daemon-denoise-v1_{win_x64,linux_x64}_tvm0.25_vulkan.{dll,so} — GPU fallback for non-NVIDIA hardware.
• Head — denoise_head.json. Required: the ONNX alone emits embeddings, not a decision.
• Tokenizer — sentencepiece.bpe.model, tokenizer_config.json.
• ONNX — model.onnx, FP32, pruned, with mean-pool + L2-norm + id-remap fused. The build source for every engine above and the path for standalone onnxruntime use.

---

8. License & attribution

The encoder weights are intfloat/multilingual-e5-small (Apache-2.0), redistributed here in compiled form unchanged; this repository is therefore released under Apache-2.0. The trained head and the build/quantization tooling are original. Backbone: XLM-RoBERTa. Not legal advice.

Used by the UltraCode code assistant, though nothing about the model is specific to it.