What Are Large Language Models (LLMs) and How Do They Work?
Large language models (LLMs) are the underlying technology that has powered the meteoric rise of generative AI chatbots. Tools like ChatGPT, Google Bard, and Bing Chat all rely on LLMs to generate human-like responses to your prompts and questions.
But just what are LLMs, and how do they work? Here we set out to demystify LLMs.
What Is a Large Language Model?
In its simplest terms, an LLM is a massive database of text data that can be referenced to generate human-like responses to your prompts. The text comes from a range of sources and can amount to billions of words.
Among common sources of text data used are:
Of course, having a huge database of text is one thing, but LLMs need to be trained to make sense of it to produce human-like responses. How it does this is what we cover next.
How Do LLMs Work?
How do LLMs use these repositories to create their responses? The first step is to analyze the data using a process called deep learning.
Deep learning is used to identify the patterns and nuances of human language. This includes gaining an understanding of grammar and syntax. But importantly, it also includes context. Understanding context is a crucial part of LLMs.
Let’s look at an example of how LLMs can use context.
The prompt in the following image mentions seeing a bat at night. From this, ChatGPT understood that we were talking about an animal and not, for instance, a baseball bat. Of course, other chatbots likeBing Chat or Google Bardmay answer this completely differently.
However, it isn’t infallible, and as this example shows, sometimes you will need to supply additional information to get the desired response.
In this instance, we deliberately threw a bit of a curve ball to demonstrate how easily context is lost. But humans can misunderstand the context of questions too, and it only needs an extra prompt to correct the response.
To generate these responses, LLMs use a technique called natural language generation (NLG). This involves examining the input and using the patterns learned from its data repository to generate a contextually correct and relevant response.
But LLMs go deeper than this. They can also tailor replies to suit the emotional tone of the input. When combined with contextual understanding, the two facets are the main drivers that allow LLMs to create human-like responses.
To summarize, LLMs use a massive text database with a combination of deep learning and NLG techniques to create human-like responses to your prompts. But there are limitations to what this can achieve.
What Are the Limitations of LLMs?
LLMs represent an impressive technological achievement. But the technology is far from perfect, and there are still plenty of limitations as to what they can achieve. Some of the more notable of these are listed below:
There is also an argument that ethical concerns can be considered a limitation of LLMs, but this subject falls outside the scope of this article.
3 Examples of Popular LLMs
The continuing advance of AI is now largely underpinned by LLMs. So while they aren’t exactly a new technology, they have certainly reached a point of critical momentum, and there are now many models.
Here are some of the most widely used LLMs.
Generative Pre-trained Transformer (GPT) is perhaps the most widely known LLM. GPT-3.5 powers the ChatGPT platform used for the examples in this article, while the newest version, GPT-4, is availablethrough a ChatGPT Plus subscription. Microsoft also uses the latest versionin its Bing Chat platform.
This is the initial LLM used by Google Bard, Google’s AI chatbot. The version Bard was initially rolled out with was described as a “lite” version of the LLM. The more powerful PaLM iteration of the LLM superseded this.
BERT stands for Bi-directional Encoder Representation from Transformers. The bidirectional characteristics of the model differentiateBERT from other LLMs like GPT.
Plenty more LLMs have been developed, and offshoots are common from the major LLMs. As they develop, these will continue to grow in complexity, accuracy, and relevance. But what does the future hold for LLMs?
The Future of LLMs
These will undoubtedly shape the way we interact with technology in the future. The rapid uptake of models like ChatGPT and Bing Chat is a testament to this fact. In the short term,AI is unlikely to replace you at work. But there is still uncertainty about just how big a part in our lives these will play in the future.
Ethical arguments may yet have a say in how we integrate these tools into society. However, putting this to one side, some of the expected LLM developments include:
These are just a few of the areas where LLMs are likely to become a larger part of the way we live.
LLMs Transforming and Educating
LLMs are opening up an exciting world of possibilities. The rapid rise of chatbots such as ChatGPT, Bing Chat, and Google Bard is evidence of the resources being poured into the field.
Such a proliferation of resources can only see these tools becoming more powerful, versatile, and accurate. The potential applications of such tools are vast, and at the moment, we are only scratching the surface of an incredible new resource.
Yes, you can run an LLM “AI chatbot” on a Raspberry Pi! Just follow this step-by-step process and then ask it anything.
The fix was buried in one tiny toggle.
It saves me hours and keeps my sanity intact.
Who asked for these upgrades?
Your phone is a better editor than you give it credit for.
You don’t need to fork out for expensive hardware to run an AI on your PC.
