Skip to main content

hls4ml Trainable Integration

Status:PreliminaryIn progress
Tags:Hls4mlImplementationTraining
Last modified:

hls4ml logo

Purpose of this section

This section is the filtered map of hls4ml that matters for making ENABOL trainable in hardware. It focuses on the path from model conversion to emitted firmware, then records the objects we expect to extend for backpasses and global throttling.

The hls4ml Path We Care About​

For our purposes, hls4ml is a compiler pipeline that turns a frontend model into a C++ HLS project. The exact frontend can vary, but the useful mental model is:

The converter creates a list of hls4ml layer descriptions. ModelGraph turns that list into graph nodes, registers inputs, outputs, variables, and weights, and applies backend flows. Vivado flows initialize layers, run graph optimizations, infer or transform types, attach C++ templates to layers, and finally invoke the writer. The writer mostly copies project templates and fills insertion points with C++ snippets already stored on graph/layer attributes.

That last sentence is the main rule for our integration: training semantics should be resolved before the writer. The writer should emit already-resolved facts.

Relevant Objects​

The objects we will modify or extend are:

AreaWhy it matters
HLSConfigHolds model, layer, precision, flow, and backend configuration. This should become the source of truth for Trainable, losses, optimizer, batch size, and controller config.
ModelGraph and flowsOwns graph construction and flow application. This is where trainable flows should be registered and applied, not where C++ should be hand-written.
Layer attributeshls4ml already stores per-layer facts as attributes. Backpass types, trainability, update types, and controller metadata should become explicit attributes.
Backend template passesCurrent forward C++ is generated by backend templates. Backward C++ should follow the same pattern.
Vivado writer hooksThe writer needs small top-level hooks for trainable IO and emission of generated backpass code, but should not infer graph semantics.
Training endpoints and lossesLoss computation and output-gradient seeding are model-level generated endpoints, not ordinary forward layers.
Optimizer and controller stateGlobal throttling is a model-level controller over raw updates, not row/column projection hidden inside individual layers.
ENABOL bridgeCurrent ENABOL should produce hls4ml config cleanly from PrecisionDict, model metadata, controller settings, and dataset/loss choices.

Old Implementation Lesson​

The deprecated trainable hls4ml fork proved that the hardware structure is feasible: add ground-truth IO, compute loss, seed output gradients, walk layers backward, call layer-specific backpass kernels, and update weights. The fragile part was where that knowledge lived. Much of it was embedded in vivado_writer.py, including C++ string parsing, config patching, logits rewiring, typedef injection, loss config generation, and trainability checks.

The new integration should keep the working ideas and move them into hls4ml-native extension points:

config -> graph attributes -> backend passes/templates -> writer emission

Integration Plan​

The first upstreamable version should be conservative:

  1. Add a first-class trainable configuration schema under HLSConfig.
  2. Parse trainable config into explicit model and layer attributes.
  3. Add a Vivado trainable flow that runs after normal forward templates and before writing.
  4. Generate loss endpoint metadata and backpass function snippets through backend passes.
  5. Add static Vivado headers under templates/vivado/trainable/, split into common, losses, backprop, optimizers, and controllers.
  6. Keep writer changes small: top-level trainable signature, IO pragmas, includes, loss configs, and backpass function emission.
  7. Keep ENABOL as the config producer, not as a generated-C++ patcher.

Implementation details can live in later pages once code starts changing. These pages should remain the design map: what hls4ml does today, why that matters, and where ENABOL should attach its trainable behavior.