Learn/Core Concept Why do FP4 kernels speed up inference? FP4 uses 4-bit floating point for weights and activations, cutting memory bandwidth and letting more math run per cycle on supporting GPUs. The new B300 FP4 attention kernels claim 1.69x speedup by trading a little precision for a lot of throughput. QuantisationThroughput |