đž Current Snapshot
đENABOL now has a cleaner formulation that connects the ICCAD -budgeting story to the global throttle work. -budgeting defines the fixed-point safety envelope: the admissible region where weights, activations, gradients, and updates remain bounded. Global throttling defines the trajectory controller inside that envelope: one scalar scales the full parameter update so learning slows down when curvature or fixed-point pressure makes the current step too aggressive.
Current formulationâ
At each online step, the raw update is:
The controlled update is:
The controller estimates local closed-loop sensitivity with:
Then is selected so the effective update is damped during high-curvature or numerically fragile regimes. The key design rule is that is global across all trainable parameters.
How Îș and global throttle fit togetherâ
The project should no longer frame global throttling as replacing -budgeting. The combined view is:
Let be the set of parameters satisfying the selected budgets. A direction-preserving controller can compute two global limits:
and
The applied scalar is:
This keeps one global update direction:
The distinction is useful:
| Limiting signal | Meaning |
|---|---|
| The proposed update would hit a fixed-point gain or rail budget first. is limiting the step. | |
| The update fits the envelope, but the local learning dynamics are too stiff. Curvature is limiting the step. | |
| both near 1 | The update is inside the envelope and dynamically safe enough to apply normally. |
This reconciles the submitted -budgeting formulation with the new global-throttle direction-preservation result. The old row/column projection path remains useful for reproduction and comparison, but the preferred direction-preserving formulation is a -aware global alpha.
What is currently provenâ
- Valid The floating-point one-layer sanity test shows that a global throttle can prevent divergence after input-gain drift.
- Valid The fake-fixed-point path works for staged quantization experiments: weights, updates, activations, rails, saturation counters, and update distortion plots.
- Valid The update-direction argument is clear: multiplying the flattened update by one scalar preserves the optimizer direction before fixed-point rounding and saturation.
- Valid The CSIM CTRL-NONE pipeline (half_mse â dense_backpass â sgd â global_throttle_none â apply_dense_update) compiles and trains a 1-Dense linear model with 0 errors (
CSIM-001). - Preliminary CTRL-GT-ORDER-0 is implemented in hls4ml as
global_throttle_order0with internal curvature sensing; CSIM validation is pending (CSIM-002,ENB-017). - Preliminary The -aware global-alpha formulation reconciles safety rails with global throttling, but has not yet been implemented or validated in hls4ml.
- Preliminary The phase/cosine diagnostics are useful as software observability tools, but should not be treated as direct hardware controller inputs yet.
Tested systemsâ
| Item | Current value |
|---|---|
| Dataset | DS-AFFINE-LINEAR-000 |
| Model | MDL-DENSE1-LINEAR-NOBIAS-000 |
| Float experiment | EXP-000A |
| Quantized experiment | EXP-000B |
| Workspace | workspace/ablations/exp_000_global_throttle_sanity/ |
What currently worksâ
- Custom training loop (loss, norms, curvature proxy, throttle, metrics...).
- Global throttle controller with a simple curvature proxy (software: GT-0/1/2/2-QA).
- Single-layer affine dataset with input gain drift.
- Fake quantization of weights, updates, activations, and rails.
- hls4ml firmware chain for Dense: half_mse loss, dense_backpass, sgd optimizer, CTRL-NONE (α=1) controller, apply_dense_update.
- hls4ml CTRL-GT-ORDER-0 C++ kernel with curvature sensor + algebraic safe-alpha law (pending CSIM validation).
- CSIM-001: first compilation validation (1-Dense, batch-size-8, half_mse, sgd, CTRL-NONE, 0 errors).
Main limitationsâ
- Only the one-layer linear model has been tested end-to-end.
- Current quantization tests are software fake-fixed-point, not firmware-equivalent HLS traces.
- The global throttle law is still heuristic; we have not analytically tuned , rail tightness, or precision maps.
- Legacy row/column projection may rotate the update direction when it rescales different rows, columns, or layers differently.
- The -aware global alpha is still a formulation target, not a validated hardware implementation.
Next actionsâ
- Start the hls4ml implementation with the direction-preserving trainable path:
HLS4ML-001throughHLS4ML-006(done). - Keep the first correctness target narrow: one Dense layer, half-MSE loss, SGD,
CTRL-NONE,CTRL-GT-ORDER-0. - Implement pure global throttle (CTRL-GT-ORDER-0) as the first non-trivial controller in hls4ml.
- Add the trainable config schema so
SafetyBudgetandControllercan be selected independently later. - Preserve legacy row/column projection as a selectable reproduction/comparison mode, not as the default learning controller.
- Run one-layer Dense CSIM comparison with CTRL-GT-ORDER-0 active (
CSIM-002/ENB-017). - Add -aware global alpha as the reconciliation mode.
Things not to repeatâ
-
Do not describe global throttle as abandoning -budgeting. The better framing is envelope plus trajectory controller. -
Do not lead with legacy row/column projection as the main implementation path. It is useful as a reproduction/comparison mode, but it should not be the default controller. -
Do not mix curvature instability and rail/saturation instability in the first diagnostic experiment. The plots become hard to interpret. -
Do not treat update cosine as a hardware-available controller signal. It is a software diagnostic for update distortion.