Skip to main content

DS-AFFINE-LINEAR-000: One-Layer Linear Teacher

Status:ValidIn progress
Tags:DatasetAffineLinear
Last modified:

Summary​

This is the baseline synthetic regression dataset for the first global-throttle ablations. It is intentionally simple so the Hessian and stability boundary are analytically interpretable.

FieldValue
Dataset IDDS-AFFINE-LINEAR-000
FamilyAffine / linear teacher
TaskRegression
Input dimensiond_in = 4
Output dimensiond_out = 2
BiasDisabled
Default samplesN = 1000
Default driftx_drift = gamma x, with gamma = 4 in EXP-000A and EXP-000B

Generation​

Inputs are sampled as:

x∼U([0,1]4).x \sim \mathcal{U}([0,1]^{4}).

Targets are generated by:

y=Ax.y = A x.

The default teacher matrix is:

A=[1.25βˆ’0.750.500.20βˆ’0.400.901.10βˆ’0.60].A = \begin{bmatrix} 1.25 & -0.75 & 0.50 & 0.20 \\ -0.40 & 0.90 & 1.10 & -0.60 \end{bmatrix}.

So:

[y1y2]=[1.25βˆ’0.750.500.20βˆ’0.400.901.10βˆ’0.60][x1x2x3x4].\begin{bmatrix} y_1\\ y_2 \end{bmatrix} = \begin{bmatrix} 1.25 & -0.75 & 0.50 & 0.20 \\ -0.40 & 0.90 & 1.10 & -0.60 \end{bmatrix} \begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}.

Code to generate the dataset:​

If you need to understand the structure of this dataset, check here.

import enabol
# Create the dataset
dataset = enabol.AffineDataset(num_samples=1000, use_bias=False)
# Plot it
dataset.plot()
dataset.plot_histogram()
print(dataset)

# Get the data
X, Y = dataset.get()

which renders:

dataset summary
AffineDataset(
[Input] X: 
  Shape: (1000, 4)
  Dtype: DataType.D2_FLOAT
  X <- Uniform(-1, 1)
[Output] Y: 
  Shape: (1000, 2)
  Dtype: DataType.D2_FLOAT
  Y <- X @ A.T + b
---------------------------------
A = [[ 1.25 -0.75  0.5   0.2 ]
     [-0.4   0.9   1.1  -0.6 ]]
b = [0. 0.]
---------------------------------
Analytic Hessian:
  Lambda max: 1.0841
  Eta max: 1.8448
)

The following image shows scatter plots of the input xx and target yy distributions, as well as their histograms at the bottom.

Dataset Dataset histograms

Drift Model​

The first drift model is pure input-gain drift:

xdrift=Ξ³x.x_{\mathrm{drift}} = \gamma x.

For the first ablations:

Ξ³=4.\gamma = 4.

The target remains consistent with the drifted input:

ydrift=Axdrift.y_{\mathrm{drift}} = A x_{\mathrm{drift}}.

This isolates curvature-driven closed-loop instability without changing the underlying teacher rule.

Why This Dataset Exists​

For one-layer no-bias linear regression, the loss is quadratic and the Hessian is tied to the input covariance. If the input is scaled by gamma, the dominant curvature grows approximately as:

Ξ»max⁑(Hdrift)β‰ˆΞ³2Ξ»max⁑(Hnom).\lambda_{\max}(H_{\mathrm{drift}}) \approx \gamma^2 \lambda_{\max}(H_{\mathrm{nom}}).

That makes this dataset useful for checking whether a controller reacts coherently when a learning rate that was stable before drift becomes unstable after drift.

Used By​

ExperimentRole
EXP-000AFloat global-throttle sanity check.
EXP-000BQuantized global-throttle sanity check.

Implementation​

Implemented by AffineDataset in the dataset module.

Reference: Dataset implementation