Skip to main content

Layer Attributes

Status:PreliminaryIn progress
Tags:Hls4mlImplementationPrecisionTraining
Last modified:

Role

hls4ml layers are attribute containers. This is the cleanest place to store resolved trainable facts: gradient types, update types, trainability, backward calls, and required headers.

What hls4ml Does Today​

Every hls4ml layer derives from Layer in hls4ml/model/layers.py. During initialization:

  1. The layer stores raw converter attributes.
  2. It creates mappings for weights, variables, types, and code.
  3. It gets a unique index.
  4. It resolves layer config from HLSConfig.
  5. It copies matching config keys into the layer attribute dictionary.
  6. It calls initialize().
  7. It validates required attributes.

The attribute system is defined in hls4ml/model/attributes.py.

Attribute classMeaning
AttributeBasic expected attribute.
ConfigurableAttributeUser-overridable field.
TypeAttributeStores a named precision type.
WeightAttributeStores a weight variable.
CodeAttributeStores generated source code.
WeightMappingView of all weight variables.
VariableMappingView of all tensor variables.
TypeMappingView of all named types.
CodeMappingView of generated source blocks.

Forward code generation already uses this model. A template pass stores:

layer["config_cpp"]
layer["function_cpp"]
layer["include_header"]

The writer later emits these strings.

What The Old Trainable Writer Had To Infer​

The deprecated writer inferred or patched several things that should be attributes:

Old behaviorBetter owner
Parse config_cpp to find the struct name.layer.backward_config_name or layer.config_name.
Look for weight_t and bias_t in generated C++.Layer weight_t and bias_t attributes.
Add grad_in_t and grad_out_t by string injection.TypeAttribute or generated trainable type attributes.
Synthesize reshape/flatten configs if missing.Reshape/flatten trainable template pass.
Add layer_name inside C++ config structs.Template pass field.
Add kappa/log2 enums.Trainable/controller config attributes.
Decide pass-through behavior for unsupported layers.Backward template pass.

Proposed Trainable Attributes​

At minimum, trainable layers should receive:

AttributeMeaning
trainableWhether this layer updates weights and allows gradient flow through it.
backward_supportedWhether a true backpass implementation exists.
backward_passthroughWhether unsupported behavior is an intentional pass-through.
backward_config_cppC++ config additions or complete backward config struct.
backward_function_cppC++ call for the backpass.
backward_include_headerHeaders needed by the backpass.
grad_input_nameName of incoming gradient tensor.
grad_output_nameName of outgoing gradient tensor.
forward_input_nameForward input tensor used by the backpass.
forward_output_nameForward output tensor used by the backpass.
requires_gradient_castWhether incoming gradient needs a cast buffer.

Trainable type fields should include:

Type fieldMeaning
data_in_tType of forward input seen by the backpass.
grad_in_tType of dL/dy.
grad_out_tType of dL/dx.
gW_tPer-sample weight-gradient product.
gb_tPer-sample bias-gradient product.
dW_accum_tBatch accumulation type for weight gradients.
db_accum_tBatch accumulation type for bias gradients.
update_W_tType used for weight update deltas.
update_b_tType used for bias update deltas.
optimizer_state_tDefault optimizer state type when layer-specific names are not needed.

Precision Naming Policy​

Use semantic fields in ENABOL and normalize to hls4ml names during config conversion:

ENABOL fieldhls4ml trainable field
weightweight_t
biasbias_t
activationresult_t
accumulatoraccum_t
gradientgrad_in_t and/or grad_out_t
updateupdate_W_t, update_b_t
gradient_accumdW_accum_t, db_accum_t

Avoid forcing ENABOL users to know every C++ typedef name. ENABOL should expose semantic fields; hls4ml should resolve the C++ names.

Layer-Specific Notes​

Dense and convolution layers need real backpass templates and update handling. Activation, reshape, flatten, pooling, upsampling, and padding layers can start with backpass templates that compute gradient transport. Unsupported layers should fail during validation unless explicitly marked as pass-through.

Batch normalization needs special handling because hls4ml forward config uses scale/bias naming rather than weight/bias naming. That should be handled in a batchnorm trainable template, not by regex editing generated C++.