Skip to main content

EXP-000A: Global Throttle Float LIN1

Status:Valid
Tags:ExperimentGlobal ThrottleFloatLin1
Last modified:
Workspace:workspace/ablations/exp_000_global_throttle_sanity/

Experiment Trace​

FieldValue
DatasetDS-AFFINE-LINEAR-000
ModelMDL-DENSE1-LINEAR-NOBIAS-000
Training methodDynamic global throttle, SGD-style online loop
Precision modeFloating point
DriftInput gain drift, xβ€²=Ξ³xx' = \gamma x, with Ξ³=4\gamma = 4

Purpose​

This is the first working sanity check for the dynamic global throttle idea.

The goal is not yet to test quantization, fixed-point rails, multiple layers, or the full πŸš‚ENABOL hardware path. The goal is narrower:

Can we build a custom Keras training loop that detects unstable online learning dynamics and globally throttles the learning update so training does not diverge?

This test validates the instrumentation and controller on the simplest possible model.

Code to generate the dataset:​

If you need to understand the structure of this dataset, check here.

import enabol
# Create the dataset
dataset = enabol.AffineDataset(num_samples=1000, use_bias=False)

# Get the data
X, Y = dataset.get()

Note that the dataset object also contains the analytical hessian metrics. This is important because it means that:

Ξ»max⁑(Hnom)β‰ˆ1.0841Ξ·max⁑nom=2Ξ»max⁑(Hnom)β‰ˆ1.8448.\begin{aligned} \lambda_{\max}(H_{\text{nom}}) &\approx 1.0841 \\ \eta_{\max}^{\text{nom}}=\frac{2}{\lambda_{\max}(H_{\text{nom}})} &\approx 1.8448. \end{aligned}

Drift Model​

Now let's assume a learning rate of Ξ·=0.5\eta=0.5. With no drift, the nominal margin is:

Ξ·Ξ»max⁑(Hnom)=1.0841β‹…0.5β‰ˆ0.54<2.\eta\lambda_{\max}(H_{\text{nom}}) = 1.0841 \cdot 0.5 \approx 0.54 < 2.

However, with a gain drift of Ξ³=4\gamma=4, the Hessian grows approximately as:

Ξ»max⁑(Hdrift)β‰ˆΞ³2Ξ»max⁑(Hnom)β‰ˆ17.3456.\lambda_{\max}(H_{\text{drift}}) \approx \gamma^2\lambda_{\max}(H_{\text{nom}}) \approx 17.3456.

which means the post-drift margin is:

Ξ·Ξ»max⁑(Hdrift)β‰ˆ8.6728>2\eta\lambda_{\max}(H_{\text{drift}}) \approx 8.6728 > 2
Summary

When drifting the input gain by a factor of 4, the previously stable learning rate of 0.50.5 becomes unstable. This is the regime where we expect the global throttle controller to intervene and prevent divergence.

Controller Behavior​

Given the instability introduced by the drift, what we basically expect is that the controller should choose a throttle Ξ±t\alpha_t such that the effective learning rate Ξ±tΞ·\alpha_t \eta is back in the stable region. In other words, we expect:

Ξ±tβ‰ˆΟ‡Ξ·Ctctrl.\alpha_t\approx \frac{\chi}{\eta C_t^{\text{ctrl}}}.

With Ο‡=1.5\chi=1.5 the ideal post-drift throttle:

Ξ±β‹†β‰ˆ1.58.6728β‰ˆ0.173.\alpha^\star \approx \frac{1.5}{8.6728} \approx 0.173.

Then:

Ξ±tΞ·Ξ»max⁑(Hdrift)β‰ˆ1.5<2.\alpha_t\eta\lambda_{\max}(H_{\text{drift}}) \approx 1.5 < 2.
Summary

The global throttle reduces the optimal learning rate such that it stays in the stable region even after drift.

Model​

The student model is a one-layer linear network without bias:

y^=Wx\hat{y} = Wx

This keeps the Hessian and closed-loop stability story simple.

Code to build the model:​

model = enabol.LinearBlockModel(dataset=dataset, num_hidden=[dataset.A.shape[0]],
activation=None, use_batchnorm=False, verbose=True,
use_bias=False, seed=0)
model.summary()

which returns

model summary
[INFO] - Building model with input shape (4,) and output shape (2,)
[INFO] - Added Dense layer with 2 units
Model: "LinearBlockModel"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┑━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
β”‚ input_layer (InputLayer)        β”‚ (None, 4)              β”‚             0 β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ dense (Dense)                   β”‚ (None, 2)              β”‚             8 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Total params: 8 (32.00 B)
Trainable params: 8 (32.00 B)
Non-trainable params: 0 (0.00 B)

Current Notebook Flow​

The notebook currently does three things:

  1. Builds a controlled affine dataset.
  2. Trains the one-layer model normally to confirm the task is learnable.
  3. Reinitializes the model, performs a nominal warmup, then enters a drifted online phase:
xdrift=Ξ³xx_{\mathrm{drift}} = \gamma x

The notebook compares:

RunControllerPurpose
Nominal baselineoffConfirm the one-layer model learns the teacher.
Drift baselineoffShow unstable behavior after gain drift.
Drift controlledonShow the global throttle can prevent divergence.

What Is Being Tested​

The custom trainer logs the quantities needed for closed-loop analysis:

  • loss,
  • RMSE,
  • weight error,
  • parameter norm,
  • gradient norm,
  • raw update norm,
  • actual update norm,
  • curvature proxy,
  • curvature EMA,
  • controller value alpha(t),
  • effective learning rate,
  • Hessian eigenvalue estimates,
  • stability margins,
  • update-map spectral radius,
  • finite/divergence flags.

The controller globally scales the SGD update:

Δθactual(t)=Ξ±(t)Δθraw(t)\Delta \theta_{\mathrm{actual}}(t) = \alpha(t)\Delta \theta_{\mathrm{raw}}(t)

where:

0<Ξ±(t)≀10 < \alpha(t) \le 1

The important property is that this scaling should preserve the update direction while reducing the effective learning rate.

EXP-000A.0: Sanity check without drift or quantization​

Here we just make sure that our dataset and our model classes/objects are working as expected so we simply train them with a sane value for the learning rate, and without any drift. The code for this experiment:

h = model.train_instrumented(
X,
Y,
epochs=100,
batch_size=32, #dataset.num_samples, # Full-batch for clean Hessian metrics
learning_rate=0.05,
loss_mode="half_mse",
controller=enabol.Controller.from_str(None, curvature_ema_rho=0.05, chi=1.5),
metrics=enabol.MetricsConfig(
profiles=("core", "stability", "teacher"),
reference_A=dataset.reference_weight_matrix,
),
)
print(h)
h.plot_results(title="Training History")

The result is shown in the image below, which confirms that the loss curve is smooth and converges to zero, as expected, without any instability or divergence.

Loss curve

EXP-000A.1: Drift without controller​

Here we introduce a gain drift in the input:

xβ€²=Ξ³x⇝yβ€²=Axβ€²x' = \gamma x \qquad \leadsto \qquad y' = Ax'

so that the task remains consistent while the Hessian grows approximately as:

Ξ»max⁑(Hdrift)β‰ˆΞ³2Ξ»max⁑(Hnom)\lambda_{\max}(H_{\mathrm{drift}}) \approx \gamma^2 \lambda_{\max}(H_{\mathrm{nom}})

Stage 0: Normal training without drift​

First, we reintialize the model and run a nominal warmup phase to confirm that training starts in a stable regime. The code to generate this is:

# Reinit
model.reinitialize_weights()

# Phase 1: nominal warmup
h_nom = model.train_instrumented(
X,
Y,
epochs=20,
batch_size=len(X),
learning_rate=0.5,
controller=enabol.Controller.from_str(None),
metrics=enabol.MetricsConfig(profiles=("core", "stability", "teacher"), reference_A=dataset.A),
)
h_nom.plot_results(title="Warmup History")

The result is shown in the image below, which confirms that the model learns the task and converges to zero loss in a stable way. Warmup loss curve

Stage 1: drifted training without controller​

Next, we introduce the gain drift and continue training without the controller to confirm that the training dynamics become unstable. The code to generate this is:

# Phase 2: sensor gain drift
gamma = 4.0
Xd = gamma * X
Yd = Y # important: target remains clean

h_drift = model.train_instrumented(
Xd,
Yd,
epochs=100,
batch_size=len(Xd),
learning_rate=0.5,
controller=enabol.Controller.from_str(None),
metrics=enabol.MetricsConfig(profiles=("core", "stability", "teacher"), reference_A=dataset.A / gamma),
)

h_drift.plot_results(title="Drift History")

The result is shown in the image below, which confirms that the loss curve diverges after the drift is introduced, as expected. Drift loss curve

EXP-000A.2: Drift with controller​

Finally, we run the same loop with the drifted data but now we turn on the controller to confirm that it can prevent divergence. The code to generate this is below (note that it's important that we reinitialize the model again to start from the same initial conditions as the previous runs, and that we use the same nominal warmup phase to give the controller a chance to estimate the curvature before the drift starts):

# Reinit
model.reinitialize_weights()

# Phase 1: nominal warmup
h_nom = model.train_instrumented(
X,
Y,
epochs=20,
batch_size=len(X),
learning_rate=0.5,
controller=enabol.Controller.from_str(None),
metrics=enabol.MetricsConfig(profiles=("core", "stability", "teacher"), reference_A=dataset.A),
)

# Phase 2: sensor gain drift
h_drift = model.train_instrumented(
Xd,
Yd,
epochs=100,
batch_size=len(Xd),
learning_rate=0.5,
controller=enabol.Controller.from_str("gt-order-0"),
metrics=enabol.MetricsConfig(profiles=("core", "stability", "teacher"), reference_A=dataset.A / gamma),
)

h_nom.plot_results(title="Warmup History")
h_drift.plot_results(title="Drift History")

The result is shown in the image below, which confirms that the loss curve remains stable and converges to zero even after the drift is introduced, thanks to the global throttle controller. Drift with controller loss curve

Summary of Results​

Summary

Key Findings: This preliminary notebook supports the basic claim that a global controller can throttle the total learning rate and stabilize a one-layer online learning loop.

Notebook Preview​

The notebook can be viewed directly on GitHub:

Open notebook on GitHub

If the iframe does not load, use the GitHub link above.