CTRL-GT-ORDER-1
CTRL-GT-ORDER-1 treats as a first-order dynamic state. It smooths the instantaneous jumps of CTRL-GT-ORDER-0. The firmware uses the same division-free inequality search as GT-0, replacing the division-based margin with an attractor toward the largest feasible candidate.
Definition
The controller searches the candidate table to find the largest satisfying the stability constraint:
Denote this value . The first-order update toward it:
Then clamp:
This replaces the continuous margin with the gap , avoiding the division in .
Inputs
| Input | Meaning |
|---|---|
| raw optimizer update | |
| base learning rate | |
| squared parameter delta | |
| squared gradient delta | |
| target stability margin | |
| controller adaptation gain | |
| numerical guard |
Outputs
| Output | Meaning |
|---|---|
| global throttle scalar | |
| effective learning rate | |
| globally scaled update |
State
| State | Meaning |
|---|---|
| controller gain state |
Expected Behavior
This controller reacts more smoothly than GT-0, at the cost of slower response to sudden spikes. The attractor toward acts as a first-order low-pass on the GT-0 decision, with controlling the time constant.
hls4ml Implementation
Kernel: nnet::global_throttle_order1_law<CONFIG_T>(raw_update_norm_sq, dgrad_norm_sq, alpha, ..., reset_numerator).
Shares the raw_update_sensor_order0 (Phase 1) with GT-0. The law uses raw optimizer update geometry, not actual/throttled parameter movement. For SGD, raw_update_norm_sq already includes the learning rate because the raw update is -eta * gradient. Maintains a static alpha_state variable. On reset_numerator = true, alpha_state reinitializes to 1.
Law (division-free, on squared norms):
Config field controller_k_alpha (default 0.1, key KAlpha in YAML).
The candidate table does not use zero as a normal candidate. If no candidate satisfies the inequality, the law uses the nonzero minimum candidate and logs controller_feasible = 0.
Trace logging provides raw, controlled, and actual update squared norms, ||ΔG||², the squared stability terms, α_feasible, α_state, α_code, α_min, and feasibility.
Implementation Status
Implemented in hls4ml-trainable (HLS4ML-022, inprogress). The original division-based GT-1 law was replaced with the division-free inequality-search approach, matching GT-0's fix. HLS4ML-039 replaced actual-update denominator geometry with raw-update control geometry to avoid alpha self-locking.