ST-006: Naive Safety Mechanisms
Purpose
This family compares global throttling against simple stabilizers from the literature and from practical engineering.
Candidate Baselinesβ
| Baseline | What it does | Main risk |
|---|---|---|
| input clipping | caps incoming samples | destroys input information |
| activation clipping | caps layer outputs | changes model function |
| gradient clipping | caps gradient magnitude | may distort descent direction |
| update clipping | caps parameter increments | changes optimizer dynamics |
| weight clipping | caps stored parameters | projects onto a box, can harm convergence |
Geometry Riskβ
Elementwise clipping does not generally preserve direction:
Global norm clipping is less destructive:
but it still acts directly on the gradient rather than on the closed-loop stability margin.
Questionsβ
- Can simple clipping prevent numerical failure?
- Does it preserve useful learning?
- Does it distort the actual update direction?
- Does global throttling recover better because it scales the full update uniformly?
Main Plotsβ
- loss,
- rail pressure,
- update norm,
- update cosine / phase,
- number of clipped elements,
- comparison against global throttle.