Large language overkill: How SLMs can beat their bigger, resource-intensive cousins

Two years on from the public release of ChatGPT, conversations about AI are inescapable as companies across every industry look to harness large language models (LLMs) to transform their business processes. Yet, as powerful and promising as LLMs are, many business and IT leaders have come to over-rely on them and to overlook their limitations. This is why I anticipate a future where specialized language models, or SLMs, will play a bigger, complementary role in enterprise IT.

SLMs are more typically referred to as “small language models” because they require less data and training time and are “more streamlined versions of LLMs.” But I prefer the word “specialized” because it better conveys the ability of these purpose-built solutions to perform highly specialized work with greater accuracy, consistency and transparency than LLMs. By supplementing LLMs with SLMs, organizations can create solutions that take advantage of each model’s strengths.

Trust and the LLM ‘black box’ problem

LLMs are incredibly powerful, yet they are also known for sometimes “losing the plot,” or offering outputs that veer off course due to their generalist training and massive data sets. That tendency is made more problematic by the fact that OpenAI’s ChatGPT and other LLMs are essentially “black boxes” that don’t reveal how they arrive at an answer. 

This black box problem is going to become a bigger issue going forward, particularly for companies and business-critical applications where accuracy, consistency and compliance are paramount. Think healthcare, financial services and legal as prime examples of professions where inaccurate answers can have huge financial consequences and even life-or-death repercussions. Regulatory bodies are already taking notice and will likely begin to demand explainable AI solutions, especially in industries that rely on data privacy and accuracy.

While businesses often deploy a “human-in-the-loop” approach to mitigate these issues, an over-reliance on LLMs can lead to a false sense of security. Over time, complacency can set in and mistakes can slip through undetected.

SLMs = greater explainability

Fortunately, SLMs are better suited to address many of the limitations of LLMs. Rather than being designed for general-purpose tasks, SLMs are developed with a narrower focus and trained on domain-specific data. This specificity allows them to handle nuanced language requirements in areas where precision is paramount. Rather than relying on vast, heterogeneous datasets, SLMs are trained on targeted information, giving them the contextual intelligence to deliver more consistent, predictable and relevant responses.

This offers several advantages. First, they are more explainable, making it easier to understand the source and rationale behind their outputs. This is critical in regulated industries where decisions need to be traced back to a source. 

Second, their smaller size means they can often perform faster than LLMs, which can be a crucial factor for real-time applications. Third, SLMs offer businesses more control over data privacy and security, especially if they’re deployed internally or built specifically for the enterprise.

Moreover, while SLMs may initially require specialized training, they reduce the risks associated with using third-party LLMs controlled by external providers. This control is invaluable in applications that demand stringent data handling and compliance.

Focus on developing expertise (and be wary of vendors who overpromise)

I want to be clear that LLMs and SLMs are not mutually exclusive. In practice, SLMs can augment LLMs, creating hybrid solutions where LLMs provide broader context and SLMs ensure precise execution. It’s also still early days even where LLMs are concerned, so I always advise technology leaders to continue exploring the many possibilities and benefits of LLMs. 

In addition, while LLMs can scale well for a variety of problems, SLMs may not transfer well to certain use cases. It is therefore important to have a clear understanding upfront as to what use cases to tackle. 

It’s also important that business and IT leaders devote more time and attention to building the distinct skills required for training, fine-tuning and testing SLMs. Fortunately, there is a great deal of free information and training available via common sources such Coursera, YouTube and Huggingface.co. Leaders should make sure their developers have adequate time for learning and experimenting with SLMs as the battle for AI expertise intensifies. 

I also advise leaders to vet partners carefully. I recently spoke with a company that asked for my opinion on a certain technology provider’s claims. My take was that they were either overstating their claims or were simply out of their depth in terms of understanding the technology’s capabilities. 

The company wisely took a step back and implemented a controlled proof-of-concept to test the vendor’s claims. As I suspected, the solution simply wasn’t ready for prime time, and the company was able to walk away with relatively little time and money invested. 

Whether a company starts with a proof-of-concept or a live deployment, I advise them to start small, test often and build on early successes. I’ve personally experienced working with a small set of instructions and information, only to find the results veering off course when I then feed the model more information. That’s why slow-and-steady is a prudent approach.

In summary, while LLMs will continue to provide ever-more-valuable capabilities, their limitations are becoming increasingly apparent as businesses scale their reliance on AI. Supplementing with SLMs offers a path forward, especially in high-stakes fields that demand accuracy and explainability. By investing in SLMs, companies can future-proof their AI strategies, ensuring that their tools not only drive innovation but also meet the demands of trust, reliability and control. 

AJ Sunder is co-founder, CIO and CPO at Responsive.