Custom Training Loop
What this page covers
This page summarizes the custom online-training loop used by the ablation harness. The public entry point is BaseModel.train_instrumented(...); the implementation is orchestrated by enabol.nn.training.InstrumentedTrainer, with controller logic in enabol.nn.controller.
The first target is Experiment 1A: a one-layer, no-bias, floating-point linear regression test with scale drift.
Design Goals
- Use Keras models for the trainable layers.
- Avoid
model.fit. - Own the update step explicitly.
- Flatten trainable parameters into one global vector
theta. - Flatten gradients into one global vector
G. - Compute the dynamic global throttle
alpha(t). - Apply the full update vector with one shared scalar throttle.
- Log every diagnostic needed to explain closed-loop stability.
Minimal Loop Shape
Algorithm: MinimalInstrumentedTrainingLoop
- for each online step do
-
with
GradientTapedo - end with
- end for
This is the conceptual loop;
BaseModel.train_instrumented(...) adds quantization hooks and diagnostics.First Model
No bias is included in the first test. Bias is added only after the pure scale-drift Hessian story is validated.
First Controller
Diagnostics
The first implementation should log:
- loss,
- output error,
||theta||,||G||,- raw update norm,
- actual update norm,
- curvature proxy
C(t), - EMA signal
S(t), alpha(t),- effective learning rate
alpha(t) * eta, - true Hessian metrics where feasible,
- stability margin
eta * lambda_max(H), - throttled stability margin
alpha(t) * eta * lambda_max(H), - update cosine.
Notes
Keep this page updated with code snippets once the harness stabilizes.