🧮 Fixed-point quantization
Eventually, the real hardware update must be quantized to the fixed-point format. This can be modeled as:
Where:
- quantizes/clips weights,
- quantizes/clips updates,
- is the global throttle.
This introduces quantization error:
Stability analysis (Lyapunov) under quantization
The Lyapunov/descent condition becomes roughly:
This tells us two things. First, stability requires an upper bound:
Second, fixed-point usefulness requires the update not to underflow. If update quantum is q_\Delta, then approximately:
So:
Therefore, useful stable fixed-point learning requires a nonempty interval:
Key insight
Fixed-point precision gives a minimum useful update size. Stability gives a maximum safe update size. Online learning is possible only when these bounds overlap.