Ask a chatbot to write a poem, debug your code or summarise a 40-page PDF and it answers in seconds. It feels like magic, but under the hood a large language model (LLM) is doing something surprisingly simple \u2014 repeated billions of times, very fast.<\/p>\n\n\n\n
This guide walks through what actually happens between pressing Enter<\/strong> and seeing a reply. No maths required.<\/p>\n\n\n\n Computers don’t read words; they read numbers. So the first thing an LLM does is chop your text into small pieces called tokens<\/strong>. A token is often a whole word (“cat”), sometimes part of a word (“un” + “believ” + “able”), and sometimes punctuation.<\/p>\n\n\n\n As a rough rule of thumb, 100 tokens is about 75 English words. That’s why AI tools talk about “context windows” in tokens rather than pages.<\/p>\n\n\n\n Languages written in non-Latin scripts \u2014 like Urdu, Arabic or Hindi \u2014 usually need more tokens for the same sentence. That’s one reason some models feel slower or “forget” sooner in those languages.<\/p>\n\n\n\n Each token is converted into a long list of numbers called an embedding<\/strong>. Think of it as coordinates on a giant map of meaning, where similar ideas sit close together:<\/p>\n\n\n\n The model learns these coordinates during training \u2014 nobody types them in by hand.<\/p>\n\n\n\n Here’s the breakthrough that made modern AI possible. In 2017, Google researchers published a paper called “Attention Is All You Need”<\/em>, introducing the Transformer<\/strong> architecture.<\/p>\n\n\n\n The key idea is self-attention<\/strong>: for every token, the model looks at every other token in the text and decides how much each one matters. In the sentence:<\/p>\n\n\n\n The trophy didn’t fit in the suitcase because it was too big.<\/p>\n<\/blockquote>\n\n\n\n attention helps the model work out that “it” refers to the trophy<\/em>, not the suitcase. A Transformer stacks dozens of these attention layers, each one building a richer understanding of the text.<\/p>\n\n\n\n Strip away everything else and an LLM does one thing: given some text, it predicts what token comes next.<\/strong> That’s it.<\/p>\n\n\n\n When you ask a question, the model:<\/p>\n\n\n\n It’s autocomplete on a colossal scale. The intelligence we perceive emerges from having learned patterns across trillions of words of books, code, websites and conversations.<\/p>\n\n\n\n Building a model like ChatGPT or Claude happens in stages:<\/p>\n\n\n\n Pre-training is the expensive part \u2014 it can take thousands of specialised chips running for months.<\/p>\n\n\n\n Because the model predicts plausible<\/em> text, not verified<\/em> text. If it has never seen the answer, it may still produce something that sounds right \u2014 a fake citation, a function that doesn’t exist, a confident but wrong date.<\/p>\n\n\n\n Always double-check facts, numbers, legal or medical information and code that an AI gives you. Treat it like a brilliant intern: fast and helpful, but it needs review.<\/p>\n\n\n\n Modern tools reduce hallucinations by letting the model search the web<\/strong>, read your documents<\/strong> (a technique called retrieval-augmented generation, or RAG) or run code<\/strong> to check its work.<\/p>\n\n\n\n You’ll often see models described with two numbers:<\/p>\n\n\n\n An LLM is a next-token predictor trained on an enormous amount of text, using attention to understand how words relate. It doesn’t “know” things the way you do, but it has absorbed so many patterns that it can reason, write and code remarkably well.<\/p>\n\n\n\n Understanding this helps you use these tools better: give clear context, ask for step-by-step reasoning, and verify anything important. Do that, and an LLM becomes one of the most powerful tools you’ve ever had.<\/p>\n","protected":false},"excerpt":{"rendered":" Tokens, embeddings, attention and next-word prediction \u2014 the ideas behind ChatGPT, Claude and Gemini explained without a single equation.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[9,10,15,24],"class_list":["post-110","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning","tag-ai","tag-beginners","tag-chatgpt","tag-llm"],"_links":{"self":[{"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=\/wp\/v2\/posts\/110","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=110"}],"version-history":[{"count":0,"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=\/wp\/v2\/posts\/110\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=110"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=110"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nexooraclub.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=110"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}1. It all starts with tokens<\/h2>\n\n\n\n
2. Tokens become vectors (embeddings)<\/h2>\n\n\n\n
\n
3. Attention: figuring out what matters<\/h2>\n\n\n\n
\n
4. The only real job: predict the next token<\/h2>\n\n\n\n
\n
5. Training: from autocomplete to assistant<\/h2>\n\n\n\n
Stage<\/th> What happens<\/th> Result<\/th><\/tr><\/thead> Pre-training<\/td> The model reads a huge slice of the internet and learns to predict the next token<\/td> A knowledgeable but unruly “base model”<\/td><\/tr> Fine-tuning<\/td> It’s trained on high-quality example conversations<\/td> Follows instructions and answers questions<\/td><\/tr> Human feedback (RLHF)<\/td> People rank answers; the model learns which ones are more helpful and safe<\/td> A polished, helpful assistant<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n 6. Why do LLMs “hallucinate”?<\/h2>\n\n\n\n
7. What “parameters” and “context window” mean<\/h2>\n\n\n\n
\n
The takeaway<\/h2>\n\n\n\n