ST-004: Quantization Floor / Update Dead-Zone
Purpose
This stress test targets the opposite of explosion. Learning can be numerically stable but useless because the update is too small to be represented.
Mechanism
The dead-zone condition is:
Expected symptom:
Why This Matters
A conservative throttle can accidentally kill learning. A useful controller must avoid both sides:
Main Plots
- update norm,
- update underflow rate,
- ,
- effective update quantum threshold,
- loss plateau behavior.
Controller limitation
If the stable-useful interval is empty, no scalar throttle can save the selected precision. The dtype, learning rate, or update format must change.