Project ASPECT-D · Document 1 of 2 · Theory & motivation · target: DiffuLM @ NeurIPS 2026 (Aug 29 AoE)

Denoising steps rent depth, not width

Per-metric (width, depth, steps) scaling laws for masked-diffusion TTS. Diffusion LMs have the one lever autoregressive models lack — serial compute purchasable at inference time. We test whether that lever moves every capability, or only the compositional ones.

TL;DR A masked-diffusion codec LM buys sequential computation two ways: depth d (paid in parameters, forever) and refinement steps T (paid per utterance, adjustable at serve time). We train one iso-parameter shape grid (20M/50M/125M × 5 aspect ratios, seeds ×2, +1 stretch), then sweep T ∈ {1…16} at evaluation only — one training grid yields a 75-point (w, d, T) response surface per metric for free. Pre-registered primary claims: H-D2 — steps improve intelligibility (WER) far more than speaker identity (SIM): test-time scaling is metric-selective; H-D3 — for WER, steps and depth are exchangeable at a measurable rate κ. Cost ≈240–420 GPU-h, capped 500, calendar-gated against the Aug 29 deadline. Every outcome (including the null) is a 4-page workshop paper. Outcomes (v1.0, measured): H-D2 supported in its declared coordinate — Δτ = +0.1102, CI [0.0921, 0.1312] — but Δτ is not coordinate-free: refitting the same measurements under log(1+err) or √err flips its sign, so v1.4 replaces it as the headline with a model-free, floor-referenced contrast (refinement closes 86.2% of the reachable intelligibility range against 46.4% of the reachable identity range, stable at 1.68–1.86× across those same maps). H-D1, H-D3 and H-D4 not supported; no κ is quotable. Full results in results.html, v1.1 extensions in extensions.html, the coordinate audit in coordinate-audit.html.

The two claims, one plane each. Panel 1 is the anisotropy inherited from the ASPECT program: at fixed steps, iso-WER contours run nearly horizontal, iso-SIM nearly vertical. Panel 2 is the diffusion-native claim: in the (steps, depth) plane, iso-WER contours have slope −κ — depth and steps exchange at rate κ for the compositional metric — while iso-SIM is flat: no number of refinement passes buys identity, because identity is representational, fixed by width. The dashed iso-latency line (serial cost ∝ 8·T·d) turns κ into a serving decision rule. All contour values shown are illustrative placeholders; the fitted versions land in results.html.

§1The question only a diffusion LM can ask

In an AR model, serial computation per emitted token is fixed at training time: it equals depth. Test-time compute can widen the search (sampling, reranking) but cannot add refinement passes over the sequence itself. A masked-diffusion LM decouples this: each unmasking iteration is one more full round of global, bidirectional conditioning — architecturally, one more "layer" of sequential composition, rented per utterance instead of bought in parameters.

TTS is the ideal probe because its quality axes plausibly live in different computational regimes. Intelligibility requires the compositional chain text → phonemes → durations → acoustics, and its characteristic failures (skips, repeats, garbled alignment) are failures of sequential planning that later refinement passes could correct. Speaker identity is representational — timbre lives in per-token channel capacity, and no amount of re-reading a draft adds channels. Hence the original thesis, that steps would rent depth and not width, and so move WER while leaving SIM untouched. The data withdrew it as stated: read as a rate claim it is backwards, since WER in fact converges sooner than identity, not faster-for-longer. What survives is a magnitude statement, and only against measured floors — see coordinate-audit.html.

Layer-wise probing of speech models backs the split: speaker/acoustic attributes decode from early, shallow layers while content requires the deep compositional stack (Pasad et al.; cross-model analyses 2024–25). ASPECT-D tests whether the same anisotropy governs generation, with steps as the third axis.

§2What exists, and the exact gap

Work	What it varies	What it measures	What's missing
Text MDM scaling (AR-vs-masked-diffusion compute parity; LLaDA-line)	N, data, compute	single loss / task suites	no shape axis, no per-metric decomposition, text only
SoundStorm	steps (qualitatively)	quality vs speed demos	no scaling law, no shape, single-model
MaskGCT	—	SOTA zero-shot TTS, per-metric tables	one shape, one size, no T law
Llasa	N + inference compute (AR)	per-metric, coarse	AR search, not refinement; no shape
Test-time-scaling literature (LLM reasoning)	tokens/steps at inference	task accuracy	single metric; no capability-selectivity question
ASPECT-D (this)	w × d at fixed N × T at inference	separate law per metric + exchange-rate test	—

The selectivity question — which capabilities can test-time compute buy? — is unasked even in text, where per-metric decomposition is harder to define. Speech hands us two cleanly separable capabilities with standard automatic metrics.

§3Mechanism in one picture

Effective serial computation: owned (depth) vs rented (steps). H-D3 quantifies the exchange rate κ in d_eff = d·T^κ per metric.

§4Formalization

M_sep: err_m(w, d, T) = E_m + A_m·w^−α_m + B_m·d^−β_m + C_m·T^−τ_m

M_sub: err_m(w, d, T) = E_m + A_m·w^−α_m + B_m·(d·T^κ_m)^−β_m

H-D2 (primary): Δτ = τ_WER − τ_SIM > 0 — steps are metric-selective. H-D3 (primary, not supported): κ_WER > 0 and κ_WER > κ_SIM with M_sub competitive on AICc — a measurable depth–step exchange rate for intelligibility. M_sub was decisively worse (ΔAICc +69.3 WER / +71.3 SIM-o), so no κ exists to quote. H-D1 (secondary): the fixed-T anisotropy Δρ = α_SIM/β_SIM − α_WER/β_WER > 0. H-D4: small-budget fits extrapolate to the largest budget. Full tests, bootstrap scheme (run-level resampling — the 5 T-points of one run share a trained model), and the four pre-registered outcome classes are in protocol §7.

§5Design: one grid, a free surface

Training: the validated iso-N shape grid — budgets 20M/50M/125M (N = 12·d·w², head_dim 64), five aspect ratios each, aspect span matched across budgets, μP for LR transfer, everything else identical between runs. Seeds: 2 mandatory + 1 stretch (calendar-gated). The T axis costs no training: T is swept only at synthesis, T ∈ {1, 2, 4, 8, 16} steps per codebook level with generation RNG fixed across T — so within an item, T is the only difference. 15 configs × 5 T = 75 surface points per metric from at most 45 trained models.

Backbone recipe (frozen in configs/grid.json): bidirectional transformer over Mimi tokens, SoundStorm-style coarse-to-fine masked training, MaskGIT confidence decoding, no CFG, no duration model — total length from a corpus char-rate estimate, the conditioning strategy MaskGCT validated at scale. No speaker encoder: prompt continuation is the only identity pathway, so backbone shape is the only variable.

§6Known failure modes, planned for

The T-points of one run are correlated. They share a trained model, so naive point-wise bootstrap would overstate certainty. All resampling happens at the run level (protocol §7.2). This is the statistical detail most likely to be botched; it is frozen in writing.

"Steps do nothing" must be separable from "steps aren't wired." A sampler-integrity check (outputs must differ across T under matched RNG) runs before any metric; genuinely flat step-curves after a passing check are a finding (outcome F1), not a bug.

CFG is excluded on purpose. Guidance strength is a second inference-time axis that interacts with both metrics; v1 freezes it off. It is the first listed follow-up, not a silent knob.

LR confounding across shapes. μP for width, explicit depth-transfer gate, per-depth fallback; nothing tuned per shape or per T after Phase 1.

Floor/ceiling regimes. Pilot gate G2 checks intelligibility/similarity floors at T=16 before the grid launches; ladder-shift and recipe pivots each fire at most once; the recipe pivot stays inside the diffusion-LM family because the venue is DiffuLM.

The deadline is a failure mode. Unique to this project: calendar gate G6 with dated milestones, an ordered scope-cut list, and a writing floor — the paper starts by Aug 26 no matter what state the grid is in.

§7The exact benefit

B1 · A workshop-shaped first: capability-selective test-time scaling If H-D2 holds, "more denoising steps" is not a uniform quality dial — it is a targeted intelligibility dial. That lands in all three DiffuLM tracks at once: a scaling law over architecture and steps (Foundations), a sampling/serving allocation result (Efficiency), and quantitative evidence about what global self-correction can and cannot fix (Reasoning). The null is equally at home there: "refinement lifts all metrics equally" is a clean negative about self-correction selectivity.

B2 · κ is a serving decision rule Serial latency of masked decoding is 8·T·d·c_layer(w). Conditional on H-D3, a fitted κ would answer whether to spend a latency budget on owned depth or rented steps. Outcome: H-D3 was not supported (ΔAICc(M_sub − M_sep) = +69.3 WER / +71.3 SIM-o against a ≤ +4 bar), so no κ was fitted and no κ-derived serving rule is shipped. results.html §4 reports budget-constrained minima on the measured grid instead — with T censored at 16, so they support no depth-versus-steps ordering. Plus the SIM corollary: identity is bought with width at zero serial-latency cost.

B3 · The serving contrast with AR, quantified on the same tokens Same Mimi tokens, same data as the AR ASPECT program: masked decoding carries no per-token KV growth (activations recomputed per pass) while AR KV grows linearly at d·w per frame. ASPECT-D's latency/quality frontier per metric, next to AR's KV-bound frontier, is the first same-tokenizer comparison of the two serving regimes' quality-per-serial-second — directly relevant to the workshop's long-context generation and serving topic.

§8Cost & calendar

Projected 240–420 B200-equivalent GPU-hours (hard cap 500, gate G5), which is a few days of wall-clock on available hardware. The binding constraint is the calendar, not compute: milestones M1–M6 with an ordered cut list run to the Aug 29 AoE wall (gate G6, protocol §8–9). Designed end state on the worst path: a smaller but complete grid, honest limitations, submitted on time.

§9Reading order for an autonomous agent

1) task.md — operations and decision routing. 2) protocol.html — binding thresholds, gates, statistics. 3) configs/grid.json — every architecture, recipe, sampler, and calendar number. 4) aspect-d-paper/OUTLINE.md — the 4-page structure Phase 7 must fill. 5) This document — for why, never for numbers.

§10Project documents

Document	Role
protocol.html	Canonical thresholds, gates G0–G6, pivots, statistics, deliverables.
implementation.html	What was actually built and run: harness architecture, every Phase-0 implementation decision, file-by-file reference, reproduction commands.
results.html	Measured surface, fitted laws with CIs, declared outcome class, design-rule table at measured latency, gate/cut history. Real numbers only.
coordinate-audit.html	v1.4 coordinate audit: why Δτ was demoted from the headline, the floor-referenced contrast that replaced it, the five-encoder agreement that closed the scorer-lineage rival, and three further scope conditions (χ²/dof, weight-resampled CI, local-slope range).
extensions.html	v1.1 extensions (`task-v1.md`): steps past the T=16 cap, NFE allocation across codebook levels, a fourth parameter budget, training-compute control, robustness panel. Hypothesis scoreboard H-E1…H-E6, gates, and the defects found and fixed.
`DECISION.md`	The single declared outcome class, headline CIs, submission-readiness.
`LOG.md`	Append-only decision trail: every gate with measured values, every fallback, every deviation.

ASPECT-D v1.0 · theory document · all figure values above are hypothesis illustrations; results.html and the paper carry fitted values only.