Skip to main content

Experiments Index

Status:ValidIn progress
Tags:Experiment Registry
Last modified:

The first month of experiments focuses on small synthetic systems that expose when fixed-point online learning becomes unstable and whether dynamic closed-loop throttling prevents divergence while preserving update geometry.

IDNameStatusDataset IDModel IDWorkspace
workspace/ablations
Main Question
EXP-000AOne-layer global throttle sanity checkValidDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000.../exp_000_global_throttle_sanityCan a global controller throttle the total learning rate and prevent one-layer divergence?
EXP-000BOne-layer global throttle with quantizationValidDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000.../exp_000_global_throttle_sanityDoes throttling still help when the one-layer loop includes fake-fixed-point weights, updates, activations, and rails?
EXP-001Global throttle parameter sweepValidDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000.../exp_001_lin1_stability_phase_diagramHow does the stability region and controller behavior change as we sweep drift parameters?
CSIM-001One-layer no controller sanity check in csimValidDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000.../csim_001_global_throttle_csim_sanityCan we synthesize and run training for the simplest case?
EXP-001AOne-layer no-bias float scale driftPlannedDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000notes.mdDoes the controller enforce the known Hessian stability boundary under pure scale drift?
EXP-001BOne-layer fake fixed-point railsPlannedDS-AFFINE-LINEAR-000MDL-DENSE1-LINEAR-NOBIAS-000TBDDoes throttling still help when instability is caused by quantization and saturation?
EXP-002ATwo-layer linear networkPlannedTBDTBDTBDCan global throttling stabilize inter-layer coupling without activation nonlinearities?
EXP-002BTwo-layer ReLU teacher/studentPlannedTBDTBDnotes.mdCan global throttling stabilize coupled nonlinear layer dynamics without rotating the update?

Shared Hypothesis​

There exists a region of drift parameters (alpha, beta) where ordinary fixed-point online training fails because some combination of activations, gradients, updates, or weights saturates. A useful dynamic controller should keep training bounded in this region while preserving the global descent direction.

Drift Model​

The starting drift model is affine input drift:

x_drift = alpha x + beta

Two versions should be tested:

x_drift = alpha x + beta
x_drift = clip(alpha x + beta, x_min, x_max)

The unclipped version isolates range expansion. The clipped version models sensor rails and information loss.

Common Metrics​

Every experiment should report:

MetricWhy it matters
Loss curveShows recovery, divergence, or training death.
Saturation countDirect evidence of fixed-point failure.
Activation rangeShows forward rail pressure.
Gradient rangeShows backward rail pressure.
Weight normShows parameter growth.
Gradient normShows update-field magnitude.
Update normShows effective step size.
Curvature proxy C(t)Estimates local closed-loop gain.
Global throttle alpha(t)Shows controller intervention.
Update cosineMeasures whether budgeting preserves descent direction.
Hessian metricsValidates whether C(t) tracks true curvature in toy models.

Near-Term Roadmap​

StepGoalAdds
EXP-000Sanity-check instrumentation and controllerOne layer, no bias, float, exploratory drift.
EXP-000BAdd the first fake-fixed-point pathQuantized weights, updates, activations, saturation counters, and update underflow checks.
EXP-001AClean curvature-only stability testPure scale drift, Hessian-selected learning rates.
EXP-001BNumerical hardware-style failureFake fixed-point quantization, rails, saturation counters.
001CMore realistic affine driftBias and scale+shift drift.
EXP-002ACoupled linear dynamicsTwo Dense layers, no activation.
EXP-002BNonlinear hidden representationTwo Dense layers with ReLU.
003πŸš‚ENABOLcomparisonLoose kappa rails, legacy row/column projection if useful.

Initial Variant Matrix​

VariantPurpose
Floating referenceEstablish expected behavior without fixed-point limits.
Fixed-point baselineFind drift regimes where online learning fails.
Dynamic global throttleTest closed-loop stabilization with a single shared update scalar.
Loose kappa + throttleTest static safety rails plus dynamic update control.
Global static kappa scaleTest representational gain control without row/layer direction changes.
Legacy row/column projectionOptional comparison only; do not rebuild first.

Documentation Rule​

Each experiment workspace must contain:

  • config.yaml: settings, seed, dataset ID, model ID, precision map, drift grid, and enabled controller mechanisms.
  • notes.md: hypothesis, procedure, status, interpretation, and links to results.
  • notebooks/analysis.ipynb: exploratory run and plots.
  • results/: exported logs, CSV files, figures, or summaries.

Each experiment documentation page should start with an Experiment Trace table that names the dataset ID, model ID, training method, precision mode, and drift/corruption model. Do not duplicate full dataset or model derivations unless the experiment intentionally specializes them.

When an exploratory notebook becomes stable, add run.py for reproducible batch sweeps.