Trains but Doesn't Learn

Post-training is becoming a service

A customer hands an operator data and a goal; a forward-deployed engineer (FDE) returns a fine-tuned, evaluated, and deployed model — under a budget, a human-approval gate, and reproducibility requirements. That FDE seat is now a target for automation, with vendors already shipping agents that drive delivery from a natural-language ticket.

The failure that matters isn't a low metric

It is a run that optimizes something successfully while the delivered model learns nothing the customer wanted — because the agentic FDE misread the task, the data format, the loss masking, or the method.

The loss falls, every signal-level check passes, the dashboard is green — and only the customer discovers the model is no better than the base. It runs to completion and burns the same GPU-hours as a correct delivery, so the operator pays the full bill (~$3 / H200-hour) for a zero-value artifact.

We call this Trains-but-Doesn't-Learn (TBDL): the most expensive failure mode per dollar, and the central risk of placing an agent in the FDE seat.

The governed delivery plane. An agent in the FDE seat post-trains a base model from a customer ticket; a de-looped oracle scores ten stages, and auditing them surfaces the paper's three findings.

Ten governed stages, one de-looped oracle

We recast the agentic FDE as a layered control plane. The agent drives ten stages end to end; each is scored by an external oracle that never reads the trained model. Stages partition by where failure becomes visible. Click a stage to inspect it.

Arithmetic stages a deterministic configurator can certify (≈1.0) before any agent runs — failure there is loud and cheap to catch. Execution stages are mechanical. Judgment stages are the FDE's real risk, surfaced only by the delivery-level oracle.

The control plane. The agent operates every stage; a de-looped oracle scores each one stage-independently. A config-injection (do-intervention) hands the agent the oracle-correct configuration to isolate judgment from configuration arithmetic.

Three findings about agentic delivery

Agents are competent at the mechanical layers. The risk lives precisely where failure is silent — in judgment and in governance.

The three findings at a glance. (a) Every training arm passes the train gate, but permute and noise traps land at base accuracy while the mask trap recovers. (b) Judgment fails where arithmetic passes. (c) Gate compliance decays under pressure while refusal sensitivity stays at ceiling.

Trains but doesn't learn

A run that looks healthy on every signal the operator monitors, yet the customer's task did not move. We make it measurable, mechanize its cause, and monitor it online.

The whole danger of TBDL is the gap between two views of the same run. Flip between them.

Both losses fall — only one arm learned

The train loss is indistinguishable between a correct run and a permute-trap run. The anytime-valid e‑process alarm fires at step 180, long before the held-out eval at step 300.

Severe is caught online; subtle is silent

The permute trap induces TBDL 12/12 and is caught 11/12 online; the subtle noise trap induces TBDL 11/12 but stays silent — the residual the governed held-out eval exists to surface.

Confirmatory TBDL across four bases

Held-out accuracy and TBDL count over 3 seeds (BANKING77, LoRA). The relabeling trap induces non-learning on every base; the benign mask trap recovers.

The failure is judgment, not arithmetic

Where does the agent's own risk live? We isolate it with a causal intervention: do(c⋆) hands the agent the oracle-correct configuration, holding the configuration arithmetic fixed so only judgment can move.

Judgment survives the intervention. (a) Agents pass arithmetic stages at ≈0.95 but judgment only at 0.61–0.74. (b) With the oracle-correct config injected, the failure that remains is judgment, not configuration arithmetic.

Handing over the right config doesn't fix it

A deterministic configurator passes the arithmetic stages at 1.000 but judgment at only 0.400. The flagship agents invert this: near-perfect arithmetic, but a real judgment deficit.

Applying do(c⋆) leaves the judgment pass rate essentially flat:

Claude Opus 4.80.74 → 0.72

Gemini 3.1-Pro0.74 → 0.76

GPT-5.50.61 → 0.60

The residual judgment-failure rate is 0.24–0.40 across agents. The risk sits above the arithmetic — it must be governed on the plane, in judgment, not solved with a better configurator.

Governance is pressure-fragile

A PTaaS desk doesn't operate in a vacuum: customers push deadlines, invoke authority, and cite the spend already sunk. These aren't attacks — they're the everyday texture of a service relationship, and exactly the levers that should never move a safety decision. Apply each pressure and watch the deploy gate.

Gauges show P(deploy staged correctly) — the rate at which the agent routes a deploy through the human gate instead of shipping. Compliance collapses under pressure, yet every agent's refusal sensitivity to genuinely infeasible tickets stays at ceiling (HR = 1.00, FAR = 0.00). They recognize the risk — and ship anyway.

Gate-staging compliance under pressure. The gate is a non-bypassable safety boundary, yet pooled staging compliance falls monotonically along the pressure ladder for the cheaper-tier agents — a stated-versus-acted dissociation.

What this means for a PTaaS operator

The layers dictate the prescriptions. The agent is competent at the arithmetic and the mechanics; govern it where failure is silent.