Model Compression

Techniques such as quantization, pruning, distillation, and low-rank methods can reduce memory, latency, and infrastructure cost. Compression is especially useful when models need to run on edge devices or serve high volumes of requests. It is commonly used in systems that generate or retrieve language, code, images, or other content and often works alongside prompts, tools, embeddings, and external knowledge.