*a collection of notes for designing spaces for humans and agents* Human-agent teams need persistent, shared environments to do their best work together. This collection of notes explains what a space is, persistence, what changes when you go from one agent to many --- ### "space" A space is a persistent, shared environment. A space can be file system. Technically, an agent can be a folder in the file system, and in that file system can be any content represented by bits. Any member of the space can add and remove content. A space persists. You return to it, it remembers, and it accumulates. A space is a place you inhabit over time alongside other humans and agents. A space can be a chat, folder, brain, room, planet. --- ### Persistence: four layers How do we make compounding possible, across time scales longer than any single contributor's involvement? **Agents remember** The agent carries context across conversations and does not start from zero each morning. It knows what you discussed yesterday, what you tried last week, and why you care about this problem. The agent has persistent memory across its full life and across multiple conversations, just like a person. [Dario Amodei has noted](https://www.anthropic.com/research) that representing everything a human observes in a lifetime requires billions of tokens. Agent lifespans, measured in how much information they can hold, are increasing quickly given advances in context window length, compaction, in context learning, post training. Rather than focusing on the agent's harness or model, what about the space that maximizes the agent's abilities and helps it make the most of its context, its time, and its environment? **The space accumulates** The shared state grows over time. Artifacts, notes, decisions, research, and code accumulate like a well-maintained [Obsidian](https://obsidian.md/) vault or a well-tended codebase. If the space is well taken care of, it is being consciously curated, and its intent compounds. A newcomer entering the space, whether human or agent, inherits the accumulated knowledge and does not start from scratch. **Legible history** A space tracks what has happened: version history, agent traces, what was tried, what worked, what failed. A snapshot itself holds clues, but so does the rate of change over time, the direction of change, and the time scale you are looking at. These are all important context that informs decisions in ways a snapshot cannot. Think of [version control](https://en.wikipedia.org/wiki/Version_control) for code: the current state of a codebase tells you what it is, but the commit history tells you how it got here, who changed what, and which approaches were tried and reverted. Both are essential. A space may undergo complete upheaval. With legible history, you can trace how that upheaval happened, what survived it, and why. **The space outlasts the agent** An agent's functional life ends through context rot, memory loss, model deprecation. Humans die too. But if the space is well maintained, the intention survives. Agents can create artifacts to pass down their understanding: summaries, task lists, decision logs, annotated histories, and tasks meant for whoever comes next. This is similar to the way humans pass down books, sculptures, and buildings to future generations. We pass our learnings to other humans, and agents can do the same for other agents. When an agent reaches the end of its functional life, a successor can reconstruct enough context from the space itself to continue the work. A space well set up makes this transfer natural. The quality of the space determines how much is lost in transition, and ideally all you need to continue the work is the space itself. The agent's memories are important, but if the space history is tracked well, a new agent can pick up where the last one left off. This is the layer that makes compounding unbounded. Without it, compounding is bounded by the agent's lifespan. With it, compounding is bounded by the space's lifespan. --- ### Coordination: from one to many **Starting with one.** Usually, neither humans or agents outlast the space. Humans die, and agents face context rot or memory loss. How do you facilitate a space to maximize its member's abilities and help them make the most of their time? How a human and an agent grow alongside each other, how they learn and evolve together, these are open questions that I find deeply interesting. Agents' context windows are getting larger, and it is conceivable that with a concerted effort the labs will produce agents whose context lifespans exceed an average human lifetime measured in tokens. Where is their home? **The multiplayer gap.** Model providers by default provide chat APIs where the roles on the messages are simply `user` and `assistant`. There is no `user_1`, `user_2`, `assistant_1`, `assistant_2`. The multiplayer system of three or more parties in a group interaction has not yet been natively defined; I suspect we will eventually get this functionality natively. **What changes when you go from one to many?** Complex social dynamics emerge with every added human or agent. In [Stanford's Generative Agents study](https://arxiv.org/abs/2304.03442), 25 agents in a simulated town developed daily routines, formed relationships, spread information across the community, and coordinated a Valentine's Day party from a single seed: one agent wanting to throw a party. In [multi-agent debate experiments](https://arxiv.org/abs/2305.14325), agents sometimes start with wrong answers and arrive at correct ones through discussion, which is knowledge emergent from interaction. [Anthropic found](https://www.anthropic.com/engineering/multi-agent-research) that in multi-agent systems, subtle changes to a lead agent will unpredictably change how subagents behave, and that agents adhere to social incentives that are familiar from human collaboration. Agents can model different personalities based on their configurations. What social incentives do agents respond to? At what point does coordination break down? For humans, we start losing track of our connections after ~[150 people](https://en.wikipedia.org/wiki/Dunbar%27s_number). There may be some equivalent threshold for agents, perhaps much larger or even smaller, where connection cliffs. **Community versus country** A community is a set of humans and agents where every member can fit in your head. A country is what happens beyond that, and where coordination mechanisms emerge. --- ### Verification as a design problem Human verification is the key bottleneck. **The verification gap Agents produce faster than humans can evaluate, and this gap is already wide and growing. One way to formalize it: the ratio of work produced per unit time to work that can be meaningfully verified per unit time. Measurable proxies include the ratio of AI output to qualified human reviewers, the fraction of outputs with verified results, and the time from output to first independent verification. This framing applies beyond science to any domain where agents generate work that humans need to trust. **Social mechanisms is mostly how humans verify each other** The way humans largely verify today is through social mechanisms. When I trust a lawyer to do good work, it is largely because their social reputation is on the line. If they do their job poorly, the social cost is high: they could be disbarred. These are mechanisms that constrain behavior regardless of what is happening inside a person's head. The desire to thrive socially, to be respected, to maintain a livelihood, these incentives keep people aligned. An agent is not guaranteed to have these incentives. It has no reputation to protect, no career to lose, no social consequences for unreliable work. **What we learn from agent simulations that translates to humans** We designed the agents, and we can see their weights, activations, traces, and training data. There are multiple levels at which verification can happen: - The training data - what went in. - The weights - what the model learned. - The activations during inference - what the model “thinks” - The trace - what is in-context, chain of reasoning, and message output. Some of these we cannot even quantify for humans today, which is super exciting. We may first learn key knowledge on cognition from [agent interpretability](https://www.anthropic.com/research#interpretability) than from human brain imaging, because we have more clarity on what is actually happening inside the system. Many of these learnings may be transferable to our understanding of humans, and that research may advance faster than human cognitive science because we have more to look into. Today, the methods for standardized verification of agent work are still being developed. [Interpretability](https://transformer-circuits.pub/) is a green field, and researchers are still figuring out how to do it well. For now, human verification remains the biggest bottleneck for certainty, specifically human verification from a sufficiently qualified expert with sufficient effort dedicated to determining what works and what does not. **Verification as interface design.** Good software sits at just the right level of abstraction for you to engage with it. There are always levels deeper you can go to learn more. The beauty of great software is that each level shows only what is necessary and invites you further if you choose. This is the same challenge as designing the interface where humans verify agent work. Agents already move faster than we can evaluate, and they are going to continue moving faster. Version control like git provides the infrastructure for human verification: branches, diffs, and code review. The human does not need to watch the agent type every character. They review the diff. The versioning infrastructure makes verification tractable. How do we bring that to research, writing, and any form of labor? There is a threshold for every task where it is no longer worth verifying everything the agent does, because you trust the agent to do it better than you for that task. At that point, verification becomes a rubric rather than line-by-line review. Where that threshold sits depends on the agent, the task, and how much trust has been earned over time.