How to Design an Agent: Components, Runtime, and Lifecycle

This article is adapted from my PyCon China 2026 talk, “How to Design the Institution & Platform for Agents.” The presentation is available as Slides.

What makes up an agent, and what makes it a single individual? Starting with context and runtime, this article explores agent implementation, state management, and lifecycle, as well as the tension between freedom and manageability.

1. The Components of an Agent ​

This article discusses how mainstream agents are organized as of September 2026. We can understand them in terms of two aspects: context and runtime.

Context ​

Context consists of two parts:

• History
• Instructions

Runtime ​

The runtime needs to provide two capabilities:

• Execution: provided through tools, of which the shell is a particularly general-purpose example.
• State management: the file system (FS) provides general-purpose capabilities for storing and managing state.

2. Implementation Approaches ​

History ​

There are two main approaches to handling history: compression and retrieval.

Compression ​

Ways to compress history include discarding tool outputs, removing intermediate reasoning and tool calls entirely, and compressing or discarding the oldest parts of the context.

Retrieval ​

The retrieval approach stores history outside the context, looks up relevant information when needed, and brings it into the context. Retrieval methods can be described along two independent dimensions: the number of retrieval rounds and the type of indexing.

• By number of retrieval rounds:
  • Single retrieval: retrieves information once, typically with lower latency and lower recall.
  • Automatic multi-round retrieval: the agent continues retrieving based on the results it already has, typically with higher latency but the potential for higher recall.
• By type of indexing:
  • File-system-based retrieval without an index: historical data is stored directly in ordinary files. The agent uses tools such as sed to read, filter, and search them, without dedicated indexing infrastructure. This approach often has lower precision and is usually combined with automatic multi-round retrieval to improve recall.
  • Index-based retrieval: introduces techniques such as semantic embeddings, full-text indexes, and knowledge graphs, along with the corresponding infrastructure. These capabilities can also be organized into a memory system. Such approaches aim to improve precision. Ideally, they can be paired with a single retrieval for low latency, or with automatic multi-round retrieval to achieve both high precision and high recall.

Here, precision measures how much of the retrieved information is relevant, while recall measures how much of all the relevant information has been retrieved:

• Precision = relevant information retrieved / all information retrieved.
• Recall = relevant information retrieved / all relevant information.

High recall means the results include as much of the useful information as possible. Even when precision is low and most results are irrelevant, the in-context learning capabilities of current large language models may still allow an agent to pick out the information it needs and move the conversation forward correctly, at the cost of additional tokens. High recall therefore compensates for low precision by improving coverage of the information needed for subsequent tasks; it does not mean the precision of the retrieval results themselves has improved.

Instructions ​

Instructions can be organized in three ways. The main differences are when the content enters the context and who decides to read or inject it:

• Basic prompts: placed directly in the user's input and enter the context with the user message.
• Progressive prompts: the system predefines a prompt dictionary or key-value store and injects the corresponding values into the context when predefined conditions are met. For example, detecting a keyword in a basic prompt triggers injection of the prompt associated with that keyword.
• Agent-driven progressive prompts: the agent is given tools for reading prompts. During execution, it decides which instructions it needs and calls the tools to read them into the context. SKILL.md is one way to carry such instructions, but it is only one part of an Agent Skill.

Runtime ​

Defining a runtime begins with defining the tools an agent can use: for example, built-in tools, JSON Schema tools, MCP, and the ultimate general-purpose tool, the shell.

Next comes managing the agent's own state. Tools can produce side effects in external systems, but those changes do not necessarily belong to the agent's own state. For a runtime platform, the more important questions are which state belongs to the agent and should be saved, copied, and deleted with it, and how to create or restore an agent from saved state.

Tool protocols can provide mechanisms for associating state across calls, but applications still need to manage the actual state. For example, older versions of MCP supported protocol-level sessions. The July 2026 specification removed this mechanism, instead requiring applications to reference state across calls using explicit identifiers. These mechanisms do not themselves handle saving, copying, restoring, or deleting agents and their state.

The file system offers a direct way to manage this state: keep the programs, configuration, and runtime data that belong to an agent within a clearly defined set of directories. Saving, copying, or deleting those directories manages the corresponding persistent state, from which the agent can be created or restored.

The shell and file system are core tools provided by an operating system (OS). Using an OS directly to provide execution and state management is therefore the most straightforward solution. Its generality lets an agent freely combine tools, organize files, and carry out its work, giving it maximum flexibility in how it acts.

3. Current Solutions ​

Agent Skills ​

Agent Skills fully embrace agent-driven progressive prompts for instructions and an OS-based approach to the runtime. A skill packages SKILL.md, tools executable through the shell (usually CLI tools), and a file-system directory together into a cohesive unit of capability.

Limitations ​

• Tool distribution: CLIs in skills are typically written in languages such as Python. Distribution becomes more complex when compiled binaries need to be packaged.
• The boundary between content and state: the skill specification does not establish what should live outside the skill directory, such as where configuration and runtime data should be stored. If programs, configuration, and runtime data all live inside the skill directory, changing any of them changes that directory. Packaging and moving it to another location or machine therefore requires deciding whether configuration and runtime data should be included, which state should travel with it, and which should be excluded.

CoW Sandboxes ​

CoW sandboxes manage file systems using copy-on-write (CoW), combined with technologies such as microVMs or containers, to provide complete sandboxes that support snapshots and forks.

Limitations ​

CoW typically operates at the level of an entire file system, rather than allowing fine-grained management of individual components. For example, one might want to snapshot just a particular tool's .config and .local/state, then apply that snapshot to another sandbox.

Freedom and Manageability ​

Each of these individual problems can, in fact, be addressed:

• Tool installation and distribution: require tools packaged with skills to be distributed through package managers and run using mechanisms such as uvx or npx.
• Configuration and runtime data storage: require tools within skills to follow the XDG specifications, establishing where this content should be stored.
• Component-level state management: implement a fine-grained CoW sandbox system that supports snapshots and migration of individual components.

The central issue, however, is the tension between freedom and manageability. The original purpose of OS-based agents is freedom: letting agents work however they choose. Once we consider collaboration and sharing between users, or isolation between tools, we inevitably have to introduce conventions and restrictions.

In the past, OS distributions addressed this problem through package managers, the work of package maintainers, XDG specifications, and other measures. For today's rapidly growing agent ecosystem, this may be an old road we have to travel again.

4. Lifecycle ​

What Is “One” Agent? ​

Having discussed what an agent consists of, we can now ask what constitutes “one” agent and how its lifecycle should be defined.

Stateful Sessions ​

One basic approach is to define a stateful session as “one” agent. This design is reflected in today's mainstream agent implementations.

Here, state usually refers to context history and the workspace. An agent works within a directory, and the history it produces and the file changes it makes within that directory form the life of a session, or of “one” agent.

From Turns to Continuous, Real-Time Interaction ​

A session defines “one” agent and its persistent state, while turns divide up its activity. Traditionally, we have introduced the concept of a turn into the agent lifecycle based on how LLMs behave, treating it as the smallest unit of an agent's activity.

We are now seeing more complex forms of interaction, such as steering, where messages are inserted while an agent is running. Another example is asynchronous command execution: after a turn ends, commands launched by the agent may still be running while the agent itself rests.

Modeling agents in terms of turns is therefore giving way to continuous, real-time interaction, a shift also reflected in the updates to ACP v2.

Recap ​

• Agent = context + runtime
• Context = history (compression / retrieval) + instructions (basic / progressive / agent-driven progressive)
• Runtime = execution + state management; the OS provides general-purpose support through the shell and file system
• Current solutions = Agent Skills + CoW sandboxes
• Core tension = freedom ↔ manageability
• Agent boundary: a session containing context history and workspace state
• Interaction and execution model: moving from turns to continuous, real-time interaction that supports steering and asynchronous execution