Tech in Five - What is a Harness

John Savill explains what an AI “harness” is and why it matters when working with large language models, focusing on how models behave (including their lack of native memory) and how the harness provides the surrounding logic and context needed to make an LLM useful in real applications.

Overview

What a harness is (and why you care)

• The video frames an LLM as a component that generates output tokens based on input tokens.
• A harness is the surrounding layer that makes the model practical to use, by:
  • Supplying the right context and instructions.
  • Managing the interaction loop between user input and model output.
  • Handling the “application” responsibilities that the model itself does not provide.

How AI models don’t work (common misconceptions)

• The presenter highlights that models do not “understand” in a human sense; they generate tokens based on patterns learned during training.
• The model’s behavior is driven by the input it receives at inference time.

Models have no native memory

• The model does not retain state across interactions by itself.
• Any sense of continuity across turns must be provided externally.

What the harness does

• The harness provides the missing pieces around the model, such as:
  • Maintaining conversation history or relevant context.
  • Deciding what to send to the model each turn.
  • Coordinating the overall flow of an interaction.

Example: harness and model interaction

• The video walks through an example interaction to show the separation of responsibilities:
  • The model produces token output.
  • The harness manages what goes in, what comes out, and how the system uses the result.

How models generate tokens

• The presenter explains token generation at a high level:
  • The model predicts the next token(s) based on the provided input.
  • Output is produced incrementally as tokens are generated.

Why “model + harness” is the key

• The core takeaway is that useful AI solutions are not just “the model”; they are the model plus the harness that wraps it and makes it usable in a real workflow.