INT8 Weight Quantization

Storing model weights as 8-bit integers instead of 32-bit floats makes models 4× smaller and much faster to serve. The simplest scheme is symmetric per-tensor quantization:

\text{scale} = \frac{\max |w|}{127}, \qquad q = \text{clip}\left(\text{round}\left(\frac{w}{\text{scale}}\right), -127, 127\right)

Implement:

quantize(w), returning (q, scale), where q is a NumPy array with dtype np.int8. If every weight is zero, use scale = 1.0.
dequantize(q, scale), returning the float approximation q * scale.

Use np.round.

Example

q, s = quantize(np.array([0.5, -1.27, 0.0, 1.0]))
# q = array([50, -127, 0, 100], dtype=int8), s = 0.01