Skip to main content

ST-003: Rail Saturation / Overflow

Status:Preliminary
Tags:Ablation TestFixed PointSaturation
Last modified:

Purpose

This is the core fixed-point hardware failure family. It tests what happens when activations, gradients, weights, updates, or accumulators approach the representable rails.

Mechanism​

Fixed-point quantization clips values to the representable range:

Q(z)=clip⁑(round⁑(z),zmin⁑,zmax⁑).Q(z) = \operatorname{clip} \left( \operatorname{round}(z), z_{\min}, z_{\max} \right).

Objects to test separately:

ObjectFailure symptom
activationsforward information clips
gradientsbackpropagated learning signal clips
weightsstored model state clips
updatesparameter increments clip
accumulatorsbatch sums clip before averaging

Rail Pressure​

Log rail pressure per object:

rz(t)=#{∣zi(t)∣β‰₯ρzmax⁑}#{zi(t)}.r_z(t) = \frac{ \#\{|z_i(t)| \geq \rho z_{\max}\} }{ \#\{z_i(t)\} }.

where ρ\rho is a near-rail threshold such as 0.950.95.

Controller Hypothesis​

The throttle should reduce update pressure before the system repeatedly hits rails:

Ξ±t↓⇒βˆ₯Δθtβˆ₯↓⇒rz(t)↓.\alpha_t\downarrow \Rightarrow \|\Delta\theta_t\|\downarrow \Rightarrow r_z(t)\downarrow.
Information loss

If the input itself is irreversibly clipped before the model sees it, the throttle cannot recover the lost information. A wider input dtype or input cap may still be required.