MDL-DENSE1-LINEAR-NOBIAS-000: One Dense Linear Student
Summaryβ
This is the smallest student model used in the first global-throttle ablations. It has one Dense layer, no activation, and no bias. It is designed to match the linear teacher in DS-AFFINE-LINEAR-000.
| Field | Value |
|---|---|
| Model ID | MDL-DENSE1-LINEAR-NOBIAS-000 |
| Model family | Dense linear |
| Input dimension | d_in = 4 |
| Output dimension | d_out = 2 |
| Layers | 1 Dense |
| Activation | None |
| Bias | Disabled |
| Stable layer name | dense0 |
| Trainable parameters | 4 x 2 = 8 Keras kernel parameters |
Forward Passβ
The student model computes:
In Keras Dense storage convention, the kernel has shape:
The math convention used in the experiment docs is:
Diagramβ
Training Roleβ
This model is useful because the one-layer no-bias regression loss has an exact Hessian. That lets the ablation compare:
- analytic stability margin,
- online curvature proxy,
- selected global throttle
alpha_t, - quantized update distortion.
Used Byβ
| Experiment | Role |
|---|---|
| EXP-000A | Float global-throttle sanity check. |
| EXP-000B | Quantized global-throttle sanity check. |
Implementationβ
Implemented through LinearBlockModel with:
LinearBlockModel(
dataset=dataset,
num_hidden=[],
activation=None,
use_bias=False,
)
Reference: Model implementation