Read more at:
- Knowledge distillation: A larger “teacher” model trains a small “student” model so that it can learn to mimic strong reasoning capabilities, but at a much smaller scale.
- Pruning: Redundant or irrelevant parameters are removed from neural network architectures.
- Quantization: Values are reduced from high-precision to lower-precision (that is, floating-point numbers are converted to integers) to reduce data size, speed up processing, and optimize energy consumption.
Larger models can also be modified and distilled into smaller, more specialized models through techniques like retrieval-augmented generation (RAG), when they are trained to pull from trusted sources before generating a response; fine-tuning and prompt tuning to guide responses to specific areas; or LoRa (low-rank adaptation), which adds lightweight pieces to an original model to reduce its size and scope, rather than retraining or modifying the entire model.
Ultimately with SLMs, enterprise data becomes a “key differentiator, necessitating data preparation, quality checks, versioning, and overall management to ensure relevant data is structured to meet fine-tuning requirements,” notes Sumit Agarwal, VP analyst at Gartner.
Benefits of small language models
The core driver of SLMs is economic, analysts note. “For high-volume, repetitive, scoped tasks (such as customer service triage), the costs of using a trillion-parameter generalist cannot be justified,” Info-Tech’s Randall points out.
