Dense Backprop Template
nnet_dense_backprop.h implements the first supported trainable layer backpass: a Dense layer in the sequential one-output path.
Kernel Roleβ
The kernel computes three things:
| Output | Meaning |
|---|---|
grad_out | Input-side gradient, dL/dx_i = sum_j dL/dy_j * W_ij. |
weight_grad_accum, bias_grad_accum | Running batch sums for dL/dW and dL/db. |
weight_grad, bias_grad | Batch-averaged gradients emitted when batch_end=True. |
The kernel does not update weights. That separation is intentional: backprop computes gradients, SGD proposes raw updates, and the controller later applies a global alpha.
Batch Averagingβ
The config struct must provide:
static const unsigned batch_size_log2 = ...;
ENABOL rounds the requested batch size up to a power of two before emitting hls4ml config. The C++ kernel then averages accumulated gradients with a shift:
avg >>= batch_shift;
This avoids synthesizing a divider in the minimal path.
Trace Pointsβ
The file contains trace hooks for:
- forward input activation,
- incoming loss/activation gradient,
- outgoing previous-layer gradient,
- weight/bias accumulators,
- final weight/bias gradients at
batch_end.
These hooks are inactive unless HLS4ML_TRAINABLE_TRACE is defined by generated firmware.