AI glossary

A program that uses a language model to decide which steps to take and which tools to call, such as searching a database, sending a message or updating a record, to complete a task with limited human input.

Sending many model requests as one job that is processed asynchronously rather than answered immediately. OpenAI and Anthropic both charge 50% less for batch jobs completed within 24 hours.

The maximum amount of text, measured in tokens, that a model can consider in one request, including the prompt, any retrieved documents and the answer. Usable context on modest hardware is often smaller than the advertised maximum.

The total model cost of completing one unit of work, such as answering one ticket or processing one document, including every model call, retry and agent step involved. A more useful measure than cost per token.

A list of numbers that represents the meaning of a piece of text, so that similar texts have similar numbers. Embeddings power semantic search and retrieval-augmented generation.

A fixed collection of real inputs paired with approved outputs, used to score a model or prompt before and after any change. It is the safeguard that stops cost savings from lowering quality.

Further training of an existing model on your own examples so it performs a specific task, format or vocabulary better than with prompting alone.

Checks placed around a language model to keep its inputs and outputs within safe and expected limits, such as input filtering, schema validation, content policies and limits on what actions it can take.

An answer from a language model that sounds plausible but is false or unsupported by its sources. Grounding answers in retrieved documents and testing on evaluation sets reduce it.

A neural network trained on very large amounts of text to understand and generate language. The largest models are usually offered as hosted APIs and are strongest at open-ended reasoning and writing.

Sending each request to the cheapest model that can handle it well, for example simple classification to a small model and complex questions to a large one. Routing rules should come from tests on your own examples.

A process split into steps handled by several AI agents, each with its own instructions and tools, coordinated by an orchestrator or a workflow tool such as n8n or Make.

Running models on servers an organisation controls, in its own data centre or private cloud, so that data never reaches a third-party AI provider.

A provider feature that bills repeated prompt prefixes at a reduced rate. OpenAI applies it automatically to prompts over 1,024 tokens; Anthropic lets developers mark cache points. It only works when the start of the prompt stays identical between requests.

An attack in which text given to a model, typed by a user or hidden in a document, email or web page, overrides the developer's instructions. It is listed as LLM01 in the OWASP Top 10 for LLM Applications 2025.

Storing a model's weights with fewer bits, for example 4 instead of 16, so it needs less memory and runs faster on modest hardware, usually with a small loss of quality.

Training a model while simulating low-precision weights, so that it keeps more of its accuracy when it is later quantized for deployment.

A technique in which relevant documents are retrieved from your own data and given to the model with the question, so answers are grounded in that content rather than the model's memory.

A language model with roughly 0.5 to 7 billion parameters, small enough to run on a single server, workstation or edge device. SLMs suit narrow, high-volume tasks and private deployments.

The unit of text a language model reads and writes, often a word or part of a word. Providers price input and output by the number of tokens.