Skip to main content

hls4ml Implementation Roadmap

Status:PreliminaryIn progress
Tags:Hls4mlImplementationTraining
Last modified:

How to use this page

This is the ordered implementation map for hls4ml-trainable. Each row links back to the task board so the roadmap stays connected to project status. Add phase-specific implementation notes under this subsection as work starts.

Phase Notesโ€‹

Use these pages as the implementation log for each block of work:

Milestone Shapeโ€‹

The first target is not the full trainable hls4ml backend. The first target is the smallest upstreamable path:

Keras Dense model
-> hls4ml ModelGraph
-> trainable HLSConfig
-> Dense forward
-> loss endpoint
-> Dense backpass
-> raw SGD update
-> global throttle alpha
-> two-phase update
-> CSIM agreement with ENABOL software traces

Only after that path is stable should we generalize to two dense layers, activations, convolution, BatchNorm, and more aggressive pipelining.

Roadmapโ€‹

PhaseStepTaskMain files / objectsExit criteria
0. OrientationFreeze the first trainable architecture map.HLS4ML-001site/docs/hls4ml/*, site/src/data/tasks.jsThe config, flow, writer, endpoint, and controller boundaries are documented enough to start code changes.
1. ConfigDefine the normalized Model.Training schema.HLS4ML-001hls4ml/model/graph.py, ENABOL bridge codeTrainable, losses, optimizer, controller, batch size, and trainable precision fields have stable config names.
1. ConfigAdd parser/accessor methods for trainable config.HLS4ML-001HLSConfigCode can ask is_trainable(), get_training_config(), get_loss_config(), get_optimizer_config(), and get_controller_config() without reading raw dict paths everywhere.
1. ConfigAdd early validation for impossible training configurations.HLS4ML-002HLSConfig, trainable validation passMissing losses, unsupported layers, invalid controller names, and ambiguous loss/output counts fail before writing C++.
2. Flowโœ… Register a Vivado trainable flow.HLS4ML-002hls4ml/backends/vivado/vivado_backend.py, hls4ml/model/flowvivado:trainable runs after normal forward template resolution and before writer emission.
2. Flowโœ… Add trainable validation pass.HLS4ML-002hls4ml/backends/vivado/passes/trainable.pyvivado:validate_trainable_config checks the first Dense trainable subset without generating C++.
2. Flowโœ… Resolve reverse traversal order.HLS4ML-002hls4ml/backends/vivado/passes/trainable.py, ModelGraphSequential Dense graphs produce trainable_backward_order; unsupported branches are rejected.
3. Layer metadataAdd trainable layer attributes.HLS4ML-003model/attributes.py, backend layer wrappers or trainable passesLayers can hold trainable, grad_in_t, grad_out_t, update types, backward function code, backward config code, and headers.
3. Layer metadataNormalize ENABOL semantic precision into hls4ml trainable fields.HLS4ML-003 / HLS4ML-012enabol/precision.py, ENABOL hls4ml bridgePrecisionDict fields like gradient, update, and accumulator map to explicit hls4ml typedefs.
4. Static assetsPort deprecated trainable source material into the new layout.HLS4ML-004templates/vivado/trainable/{common,losses,backprop,optimizers,controllers}Static headers copy into generated projects from one trainable umbrella folder without relying on old writer-specific insertion logic.
4. Static assetsTrim old headers to generic template-driven kernels.HLS4ML-004trainable/**/*.hDense/loss headers depend on generated config structs, not hardcoded ENABOL assumptions.
5. Loss endpointโœ… Generate endpoint metadata for each model output.HLS4ML-006hls4ml/backends/vivado/passes/trainable.pyThe first one-output half_mse path has ground-truth name, scalar loss name/type, loss input, gradient seed name/type, output layer, and gradient scale metadata.
5. Loss endpointGenerate loss typedefs and config structs.HLS4ML-006loss template pass, parameters.h emissionmse or half_mse config compiles and creates loss_*_t and loss_*_grads_t.
5. Loss endpointDecide and document half_mse versus full mse scaling.HLS4ML-006loss headers, ENABOL comparison tracesHardware gradients match the software reference scale.
6. Dense backpassImplement Dense backward config template.HLS4ML-005backends/vivado/passes/*, nnet_dense_backprop.hGenerated Dense config includes data, gradient, update, accumulator, and raw-update typedefs.
6. Dense backpassImplement Dense backward function template.HLS4ML-005Dense backward template passWriter can emit a Dense backpass call from layer attributes without parsing config_cpp.
6. Dense backpassVerify Dense gradient math against ENABOL software.HLS4ML-013CSIM traces, ENABOL trace dumpdL/dW, dL/db, loss, and forward outputs match within fixed-point expectations.
7. Raw updateImplement two-phase raw update storage for Dense.HLS4ML-007Dense backpass kernel, update buffersDense can compute/store raw update separately from applying it.
7. Raw updateAdd explicit update-application phase.HLS4ML-007trainable/optimizers and trainable/controllers headersStored updates can be applied with a supplied scalar alpha.
8. ControllersImplement CTRL-NONE.HLS4ML-008controller helper headerAlpha is always 1 and the path behaves like ordinary trainable SGD.
8. ControllersImplement CTRL-GT-ORDER-0.HLS4ML-008controller helper headerAlgebraic safe alpha is computed from learning rate and curvature signal.
8. ControllersImplement CTRL-GT-ORDER-1.HLS4ML-008controller helper headerPersistent alpha state updates synthesizably.
8. ControllersImplement CTRL-GT-ORDER-2.HLS4ML-008controller helper headerAlpha and velocity state update synthesizably.
8. ControllersImplement CTRL-GT-ORDER-2-QA.HLS4ML-008controller helper headerDynamic lower alpha bound from update quantum is represented explicitly.
9. Global syncAdd global norm/reduction accumulators.HLS4ML-009trainable reduction helpersLayer-local gradient/update/parameter-change contributions reduce into global metrics.
9. Global syncAdd alpha synchronization point.HLS4ML-009controller + update phaseThe design makes clear which alpha is used to apply which stored update.
9. Global syncAdd fixed-point precision policy for controller metrics.HLS4ML-009HLSConfig, controller configNorms, curvature, EMA, alpha, and velocity have explicit types and rails.
10. WriterAdd trainable top-level IO hooks.HLS4ML-011writer/vivado_writer.py, firmware templatesGround truth, loss outputs, train/apply-update flags, and controller inputs/outputs are emitted only when trainable.
10. WriterEmit trainable config/function attributes.HLS4ML-011write_project_cpp(), write_parameters()Writer emits trainable loss/backward config structs, including batch shift and static learning-rate constants, without semantic inference in kernels.
10. WriterUpdate bridge/testbench path for inference-only calls.HLS4ML-011myproject_test.cpp, bridge writerExisting prediction flow still works with dummy trainable arguments or train=false.
11. ENABOL bridgeBuild config producer in current ENABOL.HLS4ML-012enabol, new compile/export moduleCurrent ENABOL can call hls4ml with trainable config without patching generated C++.
11. ENABOL bridgeWrite dataset/testbench export for trainable CSIM.HLS4ML-012 / HLS4ML-020 / HLS4ML-037enabol/dataset.py, vivado_writer.pyCSIM receives inputs, ground truth, epoch/shuffle settings, compact console progress, and tb_data/training/*.dat traces for first firmware validation.
12. ValidationRun one-layer Dense trainable CSIM.HLS4ML-013generated Vivado project, CSIM logsForward output, loss, gradients, alpha, and updated weights match ENABOL software traces.
12. ValidationDebug mismatch reports into targeted tests.HLS4ML-013hls4ml tests, ENABOL fixturesEvery observed mismatch maps to a small reproducible fixture.
13. GeneralizationExtend to Dense -> activation -> Dense.HLS4ML-014activation backpass, reverse traversalBackward traversal through a hidden activation works before convolution/BatchNorm work resumes.
13. GeneralizationRevisit Conv2D and BatchNorm tasks after Dense path is stable.ENB-014 / ENB-015conv and BN backpassesExisting conv/BN work is reattached to the new hls4ml-native flow instead of the old writer-heavy path.
14. OptimizationEvaluate delayed-alpha pipelined update schedule.HLS4ML-010schedule design, HLS experimentsWe know whether one-step stale alpha is acceptable and what storage policy avoids weight/backpass inconsistency.
14. OptimizationDecide on update-buffer versus weight double-buffer strategy.HLS4ML-010hardware schedule designThe design either delays per-layer update until backpass consumes old weights or double-buffers weights safely.

The practical order is:

HLS4ML-001
-> HLS4ML-002
-> HLS4ML-003
-> HLS4ML-004
-> HLS4ML-006
-> HLS4ML-005
-> HLS4ML-007
-> HLS4ML-008
-> HLS4ML-009
-> HLS4ML-011
-> HLS4ML-012
-> HLS4ML-013
-> HLS4ML-014
-> HLS4ML-010

HLS4ML-010 is intentionally late. The delayed-alpha schedule is promising, but it should not be allowed to obscure the first correctness target. The basic two-phase path gives us a reference implementation. The pipelined schedule can then be judged against it.

First Correctness Targetโ€‹

The first correct trainable build should be boring:

one Dense layer
one MSE or half-MSE loss
SGD
CTRL-NONE and CTRL-GT-ORDER-0
io_parallel
single batch size policy
no branches
no convolution
no BatchNorm
no delayed-alpha optimization

Once this is correct in CSIM, every extra feature has a known-good baseline to compare against.