Refinement Buys Intelligibility, Search Buys Identity:
What Test-Time Compute Buys in Masked-Diffusion TTS

Nityanand Mathur^* Hamees Sayed^** Ayush Pratap Singh

^*Work done outside of Blackstar Inc. ^**Work done at Smallest AI.

NeurIPS 2026 Workshop on Diffusion Language Models (DiffuLM), Sydney Poster

TL;DR A diffusion language model buys sequential computation twice: depth, paid once in parameters, and denoising steps, rented per utterance at serve time. Over one iso-parameter width–depth grid of masked-diffusion codec TTS models, with steps swept at inference only, refinement from T=1 to T=16 closes 86.2% of the reachable intelligibility range but only 46.4% of the reachable identity range, measured against floors rather than against zero. At a matched number of function evaluations, best-of-K search moves speaker similarity where refinement cannot. Identity is bought with training and with selection, not with more refinement.

Abstract

A diffusion language model buys sequential computation twice: depth, paid once in parameters, and denoising steps, rented per utterance at serve time. We ask whether the rented kind helps every capability equally. We train an iso-parameter width–depth grid of 15 masked-diffusion codec TTS models (3 seeds, 19–133 M non-embedding parameters) on 2,000 h of Emilia-EN and sweep the refinement budget T at inference only, evaluating cross-sentence zero-shot synthesis on 400 items from 174 held-out speakers. Our central result is model-free and referenced to floors we measure rather than to zero: the word error of an ASR system on the real recordings, and the speaker similarity of a codec round trip. Against those floors, refinement from T=1 to T=16 closes 86.2% of the reachable intelligibility range but only 46.4% of the reachable identity range, a 1.86× asymmetry that holds between 1.68× and 1.86× under every monotone reparameterisation of the error we tried. Retraining the same runs to 3× and 6× the schedule attenuates that factor monotonically (1.84×, 1.36×, 1.23×) while never reversing it, because refinement’s intelligibility share saturates (−0.09 pp across the last doubling, and −0.06 pp under a second recogniser from a different family) while its identity share keeps rising by +7.4 pp. The two capabilities have different bottlenecks, and we therefore offer the ordering, not the factor, as the result. We also report the pre-registered power-law fit, and we report that the step-exponent contrast it yields (Δτ = τ_WER − τ_SIM = 0.110) changes sign under those same reparameterisations: it is a property of the error coordinate, and we therefore decline to build the paper on it. The limit is on what refinement buys, not on what inference buys. At a matched number of function evaluations (NFE), best-of-K search moves speaker similarity where refinement cannot, and 4 speaker-encoder families outside the selector’s lineage agree, with per-item win rates of 64.6–79.0% and every interval above one half. Depth and steps are not interchangeable: the substitution form B(d T^κ)^−β fits worse than the separable form by ΔAICc = +69.3. Finally, 62% of the remaining identity gap belongs to the codec, not the model, so most of what identity has left to gain is not the generator’s to win.

Key figures

Pipeline: phonemes and a 3-second Mimi-encoded prompt are always visible; a bidirectional masked-diffusion transformer of depth d and width w is run by a MaskGIT confidence decoder for T steps per codebook level, and the Mimi decoder produces the waveform. — **Figure 1. The two ways a masked-diffusion TTS model buys sequential computation.** The backbone spends depth (*owned* serial compute, fixed at training time), and the sampler spends T refinement steps per codebook level (*rented* serial compute, chosen at serve time). One training grid over (w, d) therefore yields a whole (w, d, T) surface at inference.

Fitted iso-error contours over width and depth at T=16 for WER and for 1 minus SIM-o, with the 15 configs overlaid. — **Figure 2.** *Left:* iso-error contours in (w, d) at T=16. The two capabilities do not share a preferred shape, but the width amplitude saturates its bound, so the anisotropy ratio ρ is not identified. *Right:* iso-WER contours in (log T, log d): a single substitution factor d T^κ cannot reproduce both the depth ordering at fixed T and the T ordering at fixed depth.

Iso-WER contours in the plane of log steps and log depth. — **Figure 2.** *Left:* iso-error contours in (w, d) at T=16. The two capabilities do not share a preferred shape, but the width amplitude saturates its bound, so the anisotropy ratio ρ is not identified. *Right:* iso-WER contours in (log T, log d): a single substitution factor d T^κ cannot reproduce both the depth ordering at fixed T and the T ordering at fixed depth.

Panel A: bar chart of the SIM-o gain bought by each axis against the remaining headroom to the codec ceiling. Panel B: of the distance from the best system to real audio, 62% is lost in the codec. — **Figure 3. Identity is mostly not the model’s to win.** (A) What each axis buys in speaker similarity, against the headroom that remains to the codec ceiling. Refinement’s bar is the largest only because it is measured from one-step decoding, which is not an operating point anyone ships; from the deployable T=16 default it buys nothing further, and test-time search is what moves identity from there. (B) Of the distance between real audio and our best system, 62% is lost in the codec itself.

Two panels of error versus refinement steps T from 1 to 16, normalised to T=1: WER error falls steeply, 1 minus SIM-o falls only slightly. — **Figure 4. Test-time scaling is metric-selective.** Error versus refinement steps for all 15 configs, each normalised to its own T=1 value; thick lines are grid means. (A) WER error falls 0.952 in absolute terms over T=1→16 and is still falling at T=16. (B) SIM-o error falls 0.161 over the same range, 5.9× less.

Key results

Result	Value	Paper
Share of the reachable intelligibility (WER) range closed by refinement, T=1→16	86.2% [82.8, 89.1]	Table 1
Share of the reachable identity (SIM-o) range closed by refinement, T=1→16	46.4% [45.0, 47.8]	Table 1
Ratio of the two shares	1.86× [1.82, 1.90]	Table 1
The same ratio under monotone reparameterisations of the error (log(1+err), √err, err²)	1.68× to 1.86×	Table 2
The same ratio after 30k / 90k / 180k training steps (9 runs retrained to 3× and 6× the schedule)	1.84× / 1.36× / 1.23×	Table 3
Change in refinement’s shares from 90k to 180k steps: intelligibility (Whisper, wav2vec2) and identity	−0.09 pp, −0.06 pp; +7.4 pp	Table 3
Share of the remaining identity gap lost in the codec	62%	Figure 3
Best-of-K search over refinement at matched NFE: per-item win rate under 4 speaker-encoder families outside the selector’s lineage	64.6–79.0%	Table 6
Substitution form B(d T^κ)^−β against the separable form, ΔAICc (WER / SIM-o)	+69.3 / +71.3	Table 11
Pre-registered step-exponent contrast Δτ, which changes sign in 3 of the reparameterisations	0.110 [0.092, 0.131]	Table 2

The floors are measured: Whisper-large-v3’s word error on the real recordings (0.0345) and the speaker similarity of a Mimi round trip of the real target with no model in the loop (0.5554, the codec ceiling). Brackets are 95% bootstrap intervals; every search-win-rate interval lies above one half.

Research notes

Working documents from the study, kept as they were written. They predate the camera-ready paper; for the current results, read the paper.

Motivation: the pre-registration-era theory and motivation page. Its headline claim was later withdrawn.
Protocol: the v1.0 experimental protocol (thresholds, gates, pivots, statistics, deliverables), frozen before any training run.
Implementation: what was built and run, every implementation choice the protocol left open, and reproduction commands.
v1.0 results: the measured surface, fitted laws with intervals, the declared outcome class and the gate history.
v1.1 extensions: steps past T=16, NFE allocation across codebook levels, a fourth parameter budget, a training-compute control and a robustness panel.
v1.4 coordinate audit: why Δτ was demoted from the headline, and the floor-referenced contrast that replaced it.
Pre-registration: the v1.0 pre-registration and its addenda (v1.1, v1.2, v1.3, v1.5).

BibTeX

@inproceedings{mathur2026refinement,
  title     = {Refinement Buys Intelligibility, Search Buys Identity: What Test-Time
               Compute Buys in Masked-Diffusion {TTS}},
  author    = {Mathur, Nityanand and Sayed, Hamees and Singh, Ayush Pratap},
  booktitle = {NeurIPS 2026 Workshop on Diffusion Language Models: Foundations,
               Efficiency, and Reasoning (DiffuLM)},
  year      = {2026},
  url       = {https://openreview.net/forum?id=E659lrDKOx},
  note      = {Poster}
}