Small language models: 10 Breakthrough Technologies by 2025

Size does matter in the AI world. OpenAI launched GPT-3 in 2020. It was the largest language-model ever built. The firm proved that supersizing a model of this type was enough to boost performance. Since then, bigger models have been sustaining a technology boom. Noam Brown, a researcher at OpenAI told an audience in October at TEDAI San Francisco, “The incredible advances in AI over the last five years can all be summarized with one word: scale.” For certain tasks, models that are trained using more focused data can now perform as well as large ones–if they don’t do better. This is a great benefit for businesses that want to use AI in specific ways. If you are making the same request over and over, then you don’t have to include the entire internet into your model.

Many big tech firms have fun-size versions for this purpose. OpenAI offers GPT-4o as well as GPT-4o Mini; Google DeepMind offers Gemini Ultra and Gemini Nano, and Anthropic’s Claude 3 is available in three sizes: large Opus, medium Sonnet, or tiny Haiku. Microsoft has pioneered a new range of small language models, called Phi.

More and more smaller companies are also offering small models. The AI startup Writer claims its latest language model is on par with the top-tier models in many key metrics, despite having only a 20th of the parameters (the values calculated during training that determine how a system behaves).

See the complete list of 10 Breakthrough technologies for 2025.

Smaller, more efficient models are easier to train and operate. This is good news for those who want a more affordable on ramp. It could also be good for the environment: Smaller models use less energy because they require a fraction of their cousins’ computer power.

These models are also portable: They can be run directly in our pockets without having to send requests to a cloud. Small is the new big thing.