Skip to main content

SGD Optimizer Template

Status:In progress
Tags:Hls4mlTrainableOptimizerSgd
Last modified:
Source:hls4ml/templates/vivado/trainable/optimizers/sgd.h

sgd.h implements the minimal optimizer stage:

raw_update = -learning_rate * gradient

It does not update parameters directly.

Why Raw Updates​

Global throttling needs to inspect or bound the update vector before it is applied. For that reason, the optimizer emits raw update proposals:

  • weight_update,
  • bias_update.

The controller/update stage later applies:

parameter += alpha * raw_update

For CTRL-NONE, alpha=1, so this behaves like plain SGD.

Config Contract​

The generated config struct must provide:

  • n_in, n_out,
  • weight_grad_t, bias_grad_t,
  • raw_update_t,
  • learning_rate_t.

The learning rate may eventually be either a compile-time constant or a runtime input. The current template accepts it as an argument so the top-level writer can support both policies.