Can Small Language Models (SLMs) Be the Next Big Thing in EdTech?

Table of Contents

• The Rise of Small Language Models (SLMs)
• A Long Way to Go
• References

Generative AI solutions like ChatGPT and Bard (now Gemini) have changed the way the world looks for answers. And it’s not just cubicles and boardrooms, GenAI has reached classrooms as well. In a rapidly evolving world, AI-based learning solutions and AI in education are gaining interest with the use of Small Language Models (SLMs).

The cornerstone of any good GenAI is a large language model or LLM. An LLM is essentially a massive trove of data that a GenAI uses to create an output. The more data that’s fed into an LLM, the more it can do! But developing and maintaining an LLM is an immensely expensive task. 

Running on many billion parameters, the sheer amount of processing power an LLM needs is substantial and something only a few organizations can afford. The kind of resources it needs makes it next to impossible for smaller educational institutions or EdTech services to wholeheartedly adopt GenAI. 

The Rise of Small Language Models (SLMs)

And that’s where small language models or SLM’s come in. These are models that are designed for specific tasks and thus require less data to learn. This ‘minimized’ version of a language model uses significantly fewer resources and manpower, which makes it a practical and economic solution. 

In the education sector, there is also an added advantage of choosing a small language model: safety. The data sets used to train small language models are smaller. This makes content moderation easier and institutes have greater control over what their students can or cannot see. So, if a curious student does search for a topic that is sensitive in nature, the AI program can simply deny showing results as it does not have the required data. 

Small language models aren’t just about practicality, they also pave the way towards a more democratized AI environment. Scaling down GenAI makes it more inclusive as well as accessible.

AI in education also poses another challenge: the complexity of language. Entering prompts or for that matter, making sense of AI-generated answers, can be a challenge for children or new learners. 

So, these AI models too need to ensure that it takes simplicity into consideration. Many developers have taken up this task and are using innovative ways to create ‘simpler’ AI solutions. The BabyLM challenge is one such initiative.

With a small dataset of less than 100 million words, the BabyLM challenge urges developers to create GenAI solutions that are easier for humans to understand. The goal of BabyLM is also to find out how humans themselves learn language, thus looking at natural language processing in an all new way.

A Long Way to Go

AI adoption in education is still in its nascent stages. Building personalized solutions can still be an expensive affair and open source solutions may not cater to the needs of specialized institutions. In developing countries, there are other cultural and linguistic challenges that need to be accounted for. 

As AI-led education gains greater acceptance worldwide, it is now time for developers to address these barriers and make technology truly inclusive. Going forward, there are lot of surprises in store for us when it comes to AI led education. To learn more about them, stay tuned and follow The Inclusive AI!

References

Bertelson, R. (2024, February 5). The Future is Inclusive: Leveraging Generative AI for PoC Empowerment and Growth. The Inclusive AI. https://theinclusiveai.com/the-future-is-inclusive-generative-ai/

Github. (n.d.). BabyLLM Challenge. Retrieved on February 14, 2024, from https://babylm.github.io/