Software architecture is gradually moving from the traditional stateless request → response model toward systems where state, memory, and context become first-class architectural concerns.
This does not mean that microservices or stateless APIs are disappearing. Instead, a new layer is emerging on top of existing infrastructure — an agentic runtime that requires persistent memory, orchestration, identity, sandboxing, and state lifecycle management.
Over the past months, several engineering teams have independently started solving remarkably similar problems. LinkedIn is developing Cognitive Memory Agent, Cloudflare has introduced Agent Memory, Google is expanding infrastructure for agentic applications, and Kubernetes-oriented projects are exploring specialized runtime mechanisms for stateful agent workloads.
Individually, these are different products and projects. Together, they begin to reveal a common architectural pattern.
From Stateless Services to Stateful Agents
Traditional distributed systems work well when requests are relatively independent:
request → service → response
State is stored separately — in a database, cache, or external storage — while services themselves remain as stateless as possible. For conventional web applications, this is an extremely effective model. It simplifies horizontal scaling, deployment, and fault tolerance. Agentic systems behave differently. An agent may:
- operate on a task for an extended period;
- interact with multiple tools;
- communicate with other agents;
- retain results from previous actions;
- resume work after a pause;
- act on behalf of a specific user;
- use context from multiple previous interactions.
A single context window is therefore no longer enough. The architectural question changes from:
How do we pass more context to the model?
to:
How do we manage the agent’s state and context over time?
This is where memory-centric architecture begins to emerge.
Memory Becomes an Architectural Layer
In modern agentic systems, memory is increasingly becoming more than an implementation detail. It is evolving into a separate architectural domain responsible for:
- storing long-term context;
- retrieving relevant information;
- separating episodic, semantic, and procedural memory;
- managing data freshness;
- deciding what should be retained and what should be forgotten;
- enforcing access policies.
Cloudflare, for example, treats persistent memory as a dedicated component for agents, while LinkedIn is developing memory infrastructure for stateful and context-aware AI applications. This represents an important shift in architectural thinking. Previously, we would say:
Cache → accelerates access to data.
Increasingly, we need to say:
Memory → manages the state and context of an agent.
These are not the same thing. A cache can usually be discarded and reconstructed from the source of truth. Losing an important piece of agent state, however, may change the agent’s future behavior. Memory therefore requires its own policies for lifecycle, freshness, validation, ownership, and security.
The Emerging Architectural Stack
When the current signals are considered together, a new structure begins to emerge:
This is an important departure from traditional microservice architecture. Instead of treating an agent as simply another service, the system begins to introduce dedicated mechanisms for memory, orchestration, and controlled execution.
Why a Larger Context Window Is Not Enough
Increasing the context window solves only part of the problem. As more information enters the context, it becomes increasingly difficult to determine:
- which information is actually relevant;
- what is still valid;
- what has become outdated;
- which facts belong to the current task;
- which data belongs to a specific user;
- which previous actions should influence the current decision.
The problem therefore changes from:
“How do we give the model more context?”
to:
“How do we manage context?”
These are fundamentally different problems. This is why dedicated memory layers, retrieval mechanisms, and context-management systems are emerging.
From Workflow Orchestration to Agent Runtime
The second major change concerns orchestration. In a traditional architecture, orchestration often means managing a predefined workflow:
event → service A → service B → service C
With agentic systems, the execution path can become dynamic. An agent may decide:
- which tool to call;
- which service to use;
- whether to delegate a task to another agent;
- whether to retry an operation;
- whether additional context is required;
- when execution should stop.
The architecture therefore gains another layer:
Orchestration is no longer responsible only for execution order. It increasingly becomes responsible for routing, coordination, validation, identity, and lifecycle management. This is one of the key differences between a conventional workflow engine and an agent runtime.
Identity Becomes Part of the Architecture
A conventional backend often has a relatively simple security model:
User → API → Service
A multi-agent system can look very different:
User → Agent A → Agent B → Tool → External Service
This creates a fundamental question:
On whose behalf is an action being executed?
If Agent A calls Agent B, should Agent B receive all of Agent A’s permissions? Clearly not. This makes the following capabilities increasingly important:
- identity propagation;
- scoped permissions;
- short-lived credentials;
- delegation;
- auditability;
- isolation between agents.
Without these mechanisms, a multi-agent architecture can quickly become a system where it is difficult to determine who actually performed an action and why that identity had the required permissions. Identity therefore stops being purely an authentication concern. It becomes part of the orchestration layer itself.
Sandboxing Becomes Core Infrastructure
There is another important difference between agents and conventional services:
agents do not merely read data — they can take actions.
An agent may:
- execute code;
- access files;
- call APIs;
- install dependencies;
- modify data;
- execute commands.
As a result, sandboxing is gradually becoming a core mechanism of agent infrastructure rather than an optional feature. The architecture starts to look like this:
This becomes particularly important for autonomous and long-running agents. The more capabilities an agent receives, the smaller its potential blast radius should be.
Kubernetes: A New Layer, Not a Replacement for Containers
One conclusion should be treated carefully here. Kubernetes is not becoming an “agent runtime instead of a container runtime.” A more realistic interpretation is that specialized mechanisms for stateful and agentic workloads are increasingly being built on top of Kubernetes. The architecture can be viewed as:
This is an important architectural development. Kubernetes continues to solve infrastructure-level problems, while specialized layers add capabilities required by new types of workloads.
Stateless APIs Are Not Going Away
One of the easiest mistakes when interpreting this trend is to declare the end of stateless architecture. That is not what is happening. Stateless APIs remain an excellent fit for:
- CRUD;
- transactional systems;
- web services;
- data ingestion;
- integrations;
- deterministic workloads;
- most conventional microservices.
What is changing is not the foundation of distributed architecture, but the layer above it. The result is more likely to be a hybrid architecture:
This is considerably more realistic than assuming a complete transition from stateless to stateful systems.
What Connects These Changes?
Looking at the signals together, five interconnected architectural directions emerge:
1. Memory
The system needs to remember and manage state.
2. Orchestration
The system needs to coordinate dynamic agent actions.
3. Identity
Every action needs a clearly defined security context.
4. Execution
Agents need a controlled execution environment.
5. Observability
Engineers need to observe not only requests, but also how system state evolves.
This creates a new loop:
This is no longer just a request/response pipeline. It is a state evolution loop.
Why Observability Is Changing
In a traditional distributed system, we primarily measure:
- latency;
- throughput;
- error rate;
- CPU;
- memory;
- request volume.
For agentic systems, this is no longer enough. We also need to understand:
- how the agent’s state changed;
- which context was used;
- which tools were invoked;
- how many steps were required;
- which decisions were made;
- which data entered memory;
- which permissions were used;
- why an agent repeated an action.
Observability therefore begins to shift:
request observability → workflow observability → state observability
This may become one of the most important — and currently underestimated — architectural shifts.
What Happens Next?
Over the next 6–12 months, the most likely outcome is not a revolutionary replacement of existing architecture, but gradual specialization.
High confidence
Memory and context management will become standard components of production agent systems.
High confidence
Orchestration will increasingly be separated into its own architectural layer.
High confidence
Sandboxing and scoped identity will become fundamental requirements for production-grade autonomous agents.
Medium confidence
Kubernetes and other infrastructure platforms will gain more agent-specific abstractions.
Medium confidence
More standards will emerge for exchanging context, identity, and capabilities between agents.
Lower-confidence hypothesis
Stateful agent sessions will begin replacing some traditional request-driven workflows where long-running interaction and persistent context provide a clear advantage.
Stateless services, however, will remain an essential foundation.
What Architects Should Do Now
The key takeaway is not that existing microservices should be rewritten. Quite the opposite. The better approach is to separate architectural responsibilities.
1. Treat memory as a separate domain
Do not automatically mix persistent agent state with conventional caching. Define:
- lifecycle;
- retention;
- freshness;
- validation;
- ownership;
- access control.
2. Separate orchestration from capabilities
An agent should coordinate actions, but business services should not become a collection of agent-specific implementation details. Preserve the architectural boundary:
Orchestration → Capabilities
3. Design identity early
Do not wait until the second agent appears before thinking about authorization. Identity and delegation should be part of the architecture from the beginning.
4. Use sandboxing by default
The more capabilities an agent receives, the smaller its potential blast radius should be.
5. Rethink observability
Metrics should describe more than requests. They should help answer:
How did the system change as a result of the agent’s actions?
The Bigger Picture
We are not witnessing the death of microservices or stateless architecture. We are witnessing the emergence of a new architectural layer on top of them. AI agents introduce workloads where processing a request and returning a response is no longer enough. A system may need to:
remember → plan → coordinate → act → persist state → continue
That is why memory, orchestration, identity, sandboxing, and state observability are increasingly converging into a new architectural model. A simplified view looks like this:
The next stage of distributed systems will therefore probably not be defined by abandoning stateless architecture. Instead, it will be defined by combining stateless infrastructure with a managed stateful agent layer. The interesting architectural frontier is no longer simply how to run an AI model. It is how to build a system around that model that is reliable, observable, secure, and scalable.