Experiments Index
The first month of experiments focuses on small synthetic systems that expose when fixed-point online learning becomes unstable and whether dynamic closed-loop throttling prevents divergence while preserving update geometry.
| ID | Name | Status | Dataset ID | Model ID | Workspaceworkspace/ablations | Main Question |
|---|---|---|---|---|---|---|
| EXP-000A | One-layer global throttle sanity check | Valid | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | .../exp_000_global_throttle_sanity | Can a global controller throttle the total learning rate and prevent one-layer divergence? |
| EXP-000B | One-layer global throttle with quantization | Valid | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | .../exp_000_global_throttle_sanity | Does throttling still help when the one-layer loop includes fake-fixed-point weights, updates, activations, and rails? |
| EXP-001 | Global throttle parameter sweep | Valid | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | .../exp_001_lin1_stability_phase_diagram | How does the stability region and controller behavior change as we sweep drift parameters? |
| CSIM-001 | One-layer no controller sanity check in csim | Valid | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | .../csim_001_global_throttle_csim_sanity | Can we synthesize and run training for the simplest case? |
| EXP-001A | One-layer no-bias float scale drift | Planned | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | notes.md | Does the controller enforce the known Hessian stability boundary under pure scale drift? |
| EXP-001B | One-layer fake fixed-point rails | Planned | DS-AFFINE-LINEAR-000 | MDL-DENSE1-LINEAR-NOBIAS-000 | TBD | Does throttling still help when instability is caused by quantization and saturation? |
| EXP-002A | Two-layer linear network | Planned | TBD | TBD | TBD | Can global throttling stabilize inter-layer coupling without activation nonlinearities? |
| EXP-002B | Two-layer ReLU teacher/student | Planned | TBD | TBD | notes.md | Can global throttling stabilize coupled nonlinear layer dynamics without rotating the update? |
Shared Hypothesisβ
There exists a region of drift parameters (alpha, beta) where ordinary fixed-point online training fails because some combination of activations, gradients, updates, or weights saturates. A useful dynamic controller should keep training bounded in this region while preserving the global descent direction.
Drift Modelβ
The starting drift model is affine input drift:
x_drift = alpha x + beta
Two versions should be tested:
x_drift = alpha x + beta
x_drift = clip(alpha x + beta, x_min, x_max)
The unclipped version isolates range expansion. The clipped version models sensor rails and information loss.
Common Metricsβ
Every experiment should report:
| Metric | Why it matters |
|---|---|
| Loss curve | Shows recovery, divergence, or training death. |
| Saturation count | Direct evidence of fixed-point failure. |
| Activation range | Shows forward rail pressure. |
| Gradient range | Shows backward rail pressure. |
| Weight norm | Shows parameter growth. |
| Gradient norm | Shows update-field magnitude. |
| Update norm | Shows effective step size. |
Curvature proxy C(t) | Estimates local closed-loop gain. |
Global throttle alpha(t) | Shows controller intervention. |
| Update cosine | Measures whether budgeting preserves descent direction. |
| Hessian metrics | Validates whether C(t) tracks true curvature in toy models. |
Near-Term Roadmapβ
| Step | Goal | Adds |
|---|---|---|
| EXP-000 | Sanity-check instrumentation and controller | One layer, no bias, float, exploratory drift. |
| EXP-000B | Add the first fake-fixed-point path | Quantized weights, updates, activations, saturation counters, and update underflow checks. |
| EXP-001A | Clean curvature-only stability test | Pure scale drift, Hessian-selected learning rates. |
| EXP-001B | Numerical hardware-style failure | Fake fixed-point quantization, rails, saturation counters. |
| 001C | More realistic affine drift | Bias and scale+shift drift. |
| EXP-002A | Coupled linear dynamics | Two Dense layers, no activation. |
| EXP-002B | Nonlinear hidden representation | Two Dense layers with ReLU. |
| 003 | πENABOLcomparison | Loose kappa rails, legacy row/column projection if useful. |
Initial Variant Matrixβ
| Variant | Purpose |
|---|---|
| Floating reference | Establish expected behavior without fixed-point limits. |
| Fixed-point baseline | Find drift regimes where online learning fails. |
| Dynamic global throttle | Test closed-loop stabilization with a single shared update scalar. |
| Loose kappa + throttle | Test static safety rails plus dynamic update control. |
| Global static kappa scale | Test representational gain control without row/layer direction changes. |
| Legacy row/column projection | Optional comparison only; do not rebuild first. |
Documentation Ruleβ
Each experiment workspace must contain:
config.yaml: settings, seed, dataset ID, model ID, precision map, drift grid, and enabled controller mechanisms.notes.md: hypothesis, procedure, status, interpretation, and links to results.notebooks/analysis.ipynb: exploratory run and plots.results/: exported logs, CSV files, figures, or summaries.
Each experiment documentation page should start with an Experiment Trace table that names the dataset ID, model ID, training method, precision mode, and drift/corruption model. Do not duplicate full dataset or model derivations unless the experiment intentionally specializes them.
When an exploratory notebook becomes stable, add run.py for reproducible batch sweeps.