Direction-Preserving Number Representations

Bardia Zadeh and George A. Constantinides · Imperial College London

Accepted to Asilomar 2026

Paper · Code and Lean proofs · Interactive demo · George’s blog post

As part of my PhD research, I worked with my supervisor George Constantinides to investigate how low-precision number formats preserve vector directions. In block-scaled arithmetic, a shared scale factor handles magnitude, leaving the values inside a block to encode its direction. This led us to ask: how should we choose those values to represent directions well?

We treat a number format as a finite set of scalar values, or an alphabet. Choosing each vector component from this alphabet creates a product code. Normalising its nonzero vectors gives a set of points on the unit sphere, whose largest angular gaps measure the format’s worst-case directional error.

We prove that, for any fixed alphabet size, product codes have worse worst-case directional coverage in sufficiently high dimensions than unrestricted spherical codes with the same number of raw codewords. We also establish that standard floating-point, fixed-point and two’s complement alphabets are asymptotically suboptimal within the product-code class. Our theorems are formally verified in Lean, with the proofs openly available alongside the experimental code.

To explore practical block sizes, we numerically optimised 4-bit alphabets and compared them with standard formats.

For 16-element blocks, the size used in NVIDIA’s NVFP4, the sampled worst-case angular errors were:

These results, from Table I of the paper, are estimates over one million sampled directions. E2M1 comes close to the optimised alphabet, offering a geometric explanation for its strong performance in block-scaled machine learning. The experiments also show how directional coverage can guide the design of future number formats.

I also built the Directional Coverage Explorer, where you can change a scalar alphabet and see how its representable directions cover the circle.