How Large Language Models Work: From Prompt to Answer

A large language model (LLM) is an AI system that has learned patterns from a huge amount of text. Its core job is simple to state: given some text, predict the next token. Google’s machine learning course describes LLMs as, in effect, very powerful autocomplete systems that can predict thousands of tokens in a row.

What makes them “large” is the number of parameters, the adjustable numbers inside the network that are tuned during training. Google notes that today’s biggest models have hundreds of billions or even trillions of them.

Computers work with numbers, not letters, so the first step is tokenisation. The model’s tokeniser splits your text into tokens, which can be whole words, pieces of words or single characters. For English, Google’s guide puts one token at roughly four characters, or about three-quarters of a word, so 400 tokens is around 300 words. A long word such as “unbelievable” might be split into a few pieces, but the exact split depends on the model.

Each token has an ID number in the model’s vocabulary. That ID is then converted into an embedding: a long list of numbers that the network learns during training and that captures something about the token’s meaning.

Most modern LLMs are built on the Transformer, an architecture introduced by Google researchers in the 2017 paper Attention Is All You Need. Its key idea is self-attention: for each token, the model asks how much every other token in the text should affect its meaning.

Take the sentence “The auto-rickshaw could not climb the hill because it was too steep.” Self-attention helps the model link “it” to “hill” rather than to “auto-rickshaw”. Change “steep” to “old”, and the link shifts to the auto-rickshaw. Because every token can look at every other token, the model can use context from anywhere in your prompt.

After the Transformer layers have processed your prompt, the model produces a probability for every token in its vocabulary. For “The capital of India is”, the token for “New” would get a very high probability and “bicycle” almost none.

The model picks a token, usually one of the most likely, adds it to the text, and runs the whole process again to choose the next one. This loop repeats until the reply is complete or a length limit is reached. That is why chatbot answers often appear on screen a few words at a time.

Training happens in stages. In pre-training, the model reads an enormous amount of text and repeatedly guesses tokens that have been hidden from it, adjusting its parameters a little after each guess. This needs huge amounts of computing power.

A pre-trained model is good at continuing text but not at following instructions. So developers fine-tune it. In OpenAI’s 2022 InstructGPT research, people first wrote example answers, then ranked different model outputs, and those rankings were used to train the model further. This is called reinforcement learning from human feedback (RLHF), and it made answers more truthful and less toxic.

Because an LLM predicts plausible text rather than checking a database of facts, it can hallucinate: give answers that sound confident but are wrong. Like other machine learning models, it can also reflect biases in its training data. A student in Chennai using a chatbot to revise for a Class 10 exam should treat it like a helpful study partner and check key facts against the NCERT textbook.