Learn/Core Concept Why quantise a model's KV cache? KV cache quantisation shrinks the memory used by attention's key/value tensors during inference by storing them at lower precision (Q8, Q6, Q5), trading a little accuracy for much longer context windows. A community test on Qwen3.6-27B measured this tradeoff directly with KL divergence, useful if we're squeezing long contexts onto limited VRAM. QuantisationAttention |