$ #72Local and open models#72 · Local and open models · 6.4 minQuantization Explained: Q4 vs Q8 vs FP16 in Plain PythonFour bits in blocks of 32 lost about 11 percent; four bits with one scale lost almost 98.Python 3.12.15