Phase 3: Templates and Kernels
Port the useful deprecated trainable kernels into the new hls4ml backend structure, starting with Dense, loss endpoints, and raw update storage.
Linked Tasksโ
- HLS4ML-004: static trainable headers.
- HLS4ML-005: Dense backpass templates.
- HLS4ML-006: loss endpoint templates.
- HLS4ML-007: two-phase raw update storage.
First Kernelsโ
The first C++ path should include:
forward Dense
loss endpoint
Dense backpass
raw SGD update proposal
apply update with scalar alpha
This gives us a correctness baseline before adding controller complexity.
Static Template Treeโ
The first concrete headers now live under:
hls4ml/templates/vivado/trainable
โโโ backprop
โ โโโ nnet_dense_backprop.h
โโโ common
โ โโโ trainable_trace.h
โโโ controllers
โ โโโ global_throttle.h
โโโ losses
โ โโโ mse.h
โโโ optimizers
โโโ sgd.h
See Trainable Vivado Templates for the file-by-file implementation notes.
Template Policyโ
Templates should consume config structs and layer attributes. They should not inspect arbitrary strings emitted by earlier templates.
Deprecated headers can be used as source material, but the new implementation should remove assumptions that only existed for the old ENABOL bridge.
Loss Scaling Decisionโ
We need to decide whether the first endpoint uses mse or half_mse. The important requirement is not the name; it is matching the software trace scale exactly:
half_mse: dL/dy = y_hat - y
mse: dL/dy = 2 * (y_hat - y)
The selected convention must be recorded in config and test traces.
Exit Criteriaโ
Phase 3 is done when generated C++ contains Dense backpass, loss, and update kernels that compile in CSIM and match ENABOL software gradients for the one-Dense case.