🗂️ How this fits into κ-budgeting
-budgeting is still useful as a static safety rail , since it bounds:
- fixed-point dynamic range,
- layer gain,
- representational Lipschitzness,
- overflow risk,
- static hardware safety.
On the other hand, the throttle controls:
It bounds:
- learning speed,
- closed-loop adaptation stability,
- overshoot under drift,
- optimizer stiffness.
So:
A very clean combined formulation is:
Then:
But the key is that should be a loose guard rail, not the primary stabilizer applied aggressively every step.
Key insight
The throttle should prevent the system from hitting the rails too violently. The rails are still there as final protection. So if we want, we can actually disable the projection to spare hardware and computation and keep the global throttle as the main stabilizer.