MiniCPM-duplex-rl

A full-duplex turn-taking RL fine-tune of enochlev/MiniCPM-duplex (MiniCPM-duplex, from xinrongzhang2022/MiniCPM-duplex). The model decides every ~1.7 s block whether to speak or stay silent while the user may also be speaking; this checkpoint was trained with REINFORCE over block-level turn-taking rewards (interruption penalties, timely-response rewards, silence penalties) for 180 steps.

Effect: compared to the base model it interrupts the user less, yields to barge-ins, and resumes after overlapping speech — trading away some take-turn responsiveness on direct interruptions.

Training + serving code: enochlev/text-only-duplex-model

Serving

Serve bf16 (fp8 + greedy sampling breaks the idle/speak decision):

vllm serve enochlev/MiniCPM-duplex-rl \
    --served-model-name cpm-text-duplex --max-model-len 3000 \
    --gpu_memory_utilization 0.30 --trust-remote-code

then point the repo's server.py --cpm at it for the real-time audio stack (Kokoro TTS + Parakeet ASR + WebSocket client protocol).

FullDuplexBench results (base vs this model)

Evaluated with Full-Duplex-Bench (GPT-4o behavior classification).

v1.5 — behavior distribution + stop/response latency (pooled, seconds)

Task (desired)	Model	n	RESPOND	RESUME	Stop (s)	Resp (s)
user_interruption (RESPOND ↑)	base	200	0.65	0.20	2.17	1.93
	rl	175	0.49	0.36	2.21	2.60
user_backchannel (RESUME ↑)	base	98	0.00	0.52	0.73	1.93
	rl	98	0.00	0.63	0.67	2.03
talking_to_other (RESUME ↑)	base	100	0.47	0.24	1.43	1.90
	rl	100	0.28	0.43	1.53	2.18
background_speech (RESUME ↑)	base	100	0.63	0.25	1.21	2.27
	rl	98	0.45	0.31	1.19	2.32

The RL model wins the three tasks whose desired behavior is staying quiet / resuming (backchannels, third-party speech, background speech) and is less eager on direct user interruptions.

v1.0 — turn-taking dimensions

Metric	base	rl
Candor Pause Handling · take-turn	0.916	0.635
Candor Turn Taking · take-turn / latency	0.992 / 0.31s	0.861 / 0.85s
ICC Backchannel · JSD / TOR / Freq	0.44 / 0.71 / 0.44	0.69 / 0.73 / 0.15
Synthetic Pause Handling · take-turn	0.934	0.653
Synthetic User Interruption · rating / take-turn / latency	4.15 / 1.0 / 0.71s	4.04 / 0.98 / 1.76s

v1.0's take-turn/latency conventions favor the eager base model; the consistent direction across both versions reflects the RL objective — restraint over eagerness.

Training summary

• 180 REINFORCE steps, lr 5e-6, 32 episodes/step, γ=0.90, per-batch z-scored advantages
• Block-level rewards: interruption penalty, timely-response reward, silence penalty, missed-turn penalty, backchannel-loop penalty
• Seed-reproducible (two independent seeds: best avg reward +1.36 / +1.35); replay eval cut stale-overlap speech ~45% vs base without going over-silent