ST-005: Accumulator / Optimizer-State Precision
Purpose
This stress test applies rail and floor analysis to training internals that are not part of the forward model but still determine the update.
Mechanismβ
For Adam, vulnerable objects include:
For batch training, vulnerable objects include:
How This Differs From Rail/Floor Testsβ
The failure mechanism is still saturation, underflow, or quantization error. The difference is scope: the vulnerable tensors are optimizer state and accumulation registers rather than only activations, weights, gradients, and updates.
Main Plotsβ
- Adam
mnorm, - Adam
vnorm, - denominator statistics,
- raw step norm,
- quantized step norm,
- accumulator rail pressure,
- final update norm.
Priorityβ
This test should come after the simpler fixed-point path is clear.