How a language model works · Indu Learning Platform

A model is a very well-read autocomplete

A large language model (LLM) is trained on huge amounts of text to do one thing: predict what text comes next. Training tunes billions of numbers (parameters) that store patterns, not documents. After training they are frozen: the model does not learn from your chat. It writes text that looks right; whether it is right depends on what you put in front of it.

Tokens: the unit the model reads and writes

Models read tokens, chunks of text from a fixed vocabulary. In English one token is roughly four characters. SAP codes and data extracts split into many tokens. Tokens matter because you are billed per token, speed is measured in tokens per second, and the context window is counted in tokens.

Counts vary by tokenizer. Use the provider's token counter for real numbers.

Generation, one token at a time

The answer is built left to right, one predicted token per step. The model never looks anything up.

Step through · 1 of 6

Prompt: 'IDoc CREMAS status 51 usually means'. The text is split into tokens and the model processes all of them together.

An answer is many small predictions, each conditioned on everything before it.

The context window is the model's whole world

The context window is the most tokens the model can handle in one request: instructions, documents, history and its own answer. For that call it is everything the model knows about your project. If your BRFplus table is not in there, you get a generic answer about BRFplus, not your rule.

What goes in the window is all the model can use.

Why fluent answers can be wrong

The model always produces a plausible continuation, even without the facts. That is hallucination: an invented T-code, a field that is not on BUT000, a made-up SAP Note. It follows from how generation works, so you design for it: supply the real material, ask for quotes, allow 'not in the material', verify anything that touches a system.

The right column must come from the context you provide.

Try it: match the terms

Try it · Match

Match each term to what it means.

Pick an item on the left, then its match on the right.

Temperature and max_tokens

Temperature sets how adventurous token picking is. Near 0 the model takes the likeliest token, so answers are repeatable; higher values add variety for brainstorming. Low temperature makes answers consistent, not correct. max_tokens caps answer length; if an answer stops mid-sentence, it hit this cap.

Suggested starting temperatures (0–1). Tune with your own test set.

Try it: pick a temperature

Try it · Sort

Which temperature fits each task?

Low (0–0.2)

Higher (0.6+)

Click an item, then a bucket (or drag it).

Try it: the invented rule

Try it · What would you do?

You ask an assistant which BRFplus rule rejects supplier CRs without a VAT number. It confidently names rule ZBP_VAT_CHECK_01 and lists its conditions. Nobody on the project has heard of it. What do you change?

Remember

An LLM predicts the next token, repeatedly; it does not look things up.
The context window is all the model knows about your project for that call, and every token costs.
Hallucination is plausible output without support; low temperature gives consistency, not correctness.

Check yourself

The model gives a generic answer about your BRFplus validation rule. What is the best fix?

Why does pasting a 5,000-row supplier extract into every request get expensive?

You set temperature to 0. What can you expect?

An answer cuts off mid-sentence. Most likely cause?