State Is the Hard Part

State Is the Hard Part is the observation that a design's difficulty is concentrated almost entirely below the service tier. Stateless services scale by copying, which is a solved problem with a well-known cost. State cannot be copied without deciding what a copy is allowed to know and when, and that decision is where designs go wrong.

Key takeaways

• A tier is stateless when any instance can serve any request because the request plus durable storage carries everything needed. That is the entire property, and it is what makes copying work.
• Scaling state means choosing losses. PostgreSQL 18 documents five positions of synchronous_commit, and the documentation is explicit that off can lose recently acknowledged transactions without leaving the database inconsistent.
• Replication lag is a product decision, because it decides whether a user sees their own write. Route by requirement, not by load balancer convenience.
• Abadi's PACELC formulation says the consistency and latency trade-off is present at all times in a replicated system, not only during a partition, and calls ignoring it a major oversight.
• Sticky routing is state you did not admit to having. Once a session lives in one process, that process is a failure domain and a deploy is an outage for its users.

Read this after Chapter 1, whose load model tells you how much write volume the durable layer must absorb, and before Chapter 3, which supplies the vocabulary for what a copy is allowed to show. Chapter 4 handles which model answers your queries, and deliberately does not discuss replication, so keep the two decisions separate in your own design document too.

An engineer adds a read replica to relieve a loaded primary. Read traffic moves across, the primary's CPU drops, and the change ships on a Thursday.

On Monday support reports that customers save a setting, reload the page, and see the old value. Not always. About one time in twenty.

Nobody broke anything. The design acquired a new behaviour, and nobody wrote down whether that behaviour was acceptable.

Stateless is a property of the request, not the process

A tier is stateless when any instance can serve any request, because everything the request needs arrives with the request or comes from durable storage.

That is the whole definition, and its usefulness is that it makes capacity a multiplication. Ten instances handle ten times the throughput of one, minus whatever the shared dependency underneath them can take. Chapter 1's arithmetic applies directly: concurrency divided by per-instance capacity gives instance count.

The reason this half is easy is that a stateless instance has no identity worth preserving. Kill it, replace it, deploy over it, move it to another zone. Nothing was there to lose.

State reverses every one of those properties. A durable store has an identity, cannot be replaced without moving its contents, and cannot be killed mid-write without a decision about what happens to the write.

Copying state means deciding what a copy may know

The instinct is to scale state the same way: add copies. Copies of state are not equivalent to each other, which is the entire difficulty.

Two copies of a stateless process are interchangeable at every instant. Two copies of data are interchangeable only if every write reaches both before either answers a read. Making that true costs a network round trip on the write path, every time. Making it false means a reader can see a stale value. There is no third option.

Gilbert and Lynch, in their 2012 perspectives paper on the CAP theorem, frame this as the general safety and liveness trade-off in an unreliable system: consistency is a safety property, availability is a liveness property, and in a network subject to communication failures a service cannot implement atomic shared memory and guarantee a response to every request. That is a proof, not opinion.

The engineering version is smaller and more useful. You are not choosing a philosophy. You are choosing, per write, how long to wait and what you accept losing.

Durability is a dial with five documented positions

PostgreSQL makes this unusually concrete, which is why it is worth reading its documentation even if you run something else.

The synchronous_commit setting, documented in the PostgreSQL 18 WAL configuration page, takes five values: off, local, remote_write, on and remote_apply. Each one waits for something different before the commit returns. The documentation states that setting the parameter to off does not create any risk of database inconsistency, and that an operating system or database crash might result in some recent allegedly-committed transactions being lost, with the database state the same as if those transactions had been aborted cleanly.

At the other end, remote_apply waits until the synchronous standby has received the commit record, applied it so it is visible to queries on the standby, and written it to durable storage. The documentation is blunt about the price: this will cause much larger commit delays than previous settings, since it waits for WAL replay.

Read that as a menu rather than a configuration detail. The same database offers you a write that can vanish and a write that a reader on another continent can already see. The difference between them is one word of configuration. That is the whole choice.

Replication lag is a product decision

Once a replica exists, lag exists. The question is not whether to have lag. It is which reads are allowed to see it.

Some reads genuinely do not care. An analytics dashboard, a search index, a monthly report, a list of items last updated by somebody else: seconds of staleness there is invisible. Other reads care enormously, and the sharpest case is a user reading back something they just wrote.

So route by requirement rather than by convenience. Reads that must reflect the caller's own writes go to the primary, or to a replica the system has confirmed has caught up past the caller's write position. Everything else goes to a replica. Write that split into the design as a list of read paths with their tolerance stated in units of time.

The failure mode when you skip this is not an outage. It is a slow accumulation of support tickets that nobody can reproduce, because the behaviour depends on which replica answered. Those are the expensive ones.

The trade you pay for continuously, not only in a crisis

CAP describes what happens during a partition, which is rare. That rarity makes it a poor guide to daily design, and Abadi's PACELC formulation fixes the gap.

Abadi, writing in IEEE Computer in February 2012, extends the question: if there is a partition, how does the system trade availability against consistency, and else, in normal operation, how does it trade latency against consistency. He argues that ignoring the consistency and latency trade-off is a major oversight, because unlike the partition case it is present at all times. His classification puts default Dynamo, Cassandra and Riak at PA/EL, VoltDB, H-Store, Megastore, BigTable and HBase at PC/EC, MongoDB at PA/EC and PNUTS at PC/EL.

The practical reading is that your durability dial is a latency setting on the write path and a staleness setting on the read path, and you are paying it on an ordinary Tuesday with no incident in sight. You pay it every day. That is the cost teams forget to include, because they filed the whole subject under disaster recovery.

Chapter 3 gives the vocabulary for describing what the reader is allowed to observe. This chapter only insists that the setting is chosen rather than inherited.

An entity has a home, and that home is one machine

There is a useful lower bound on how much distribution you can pretend to.

Helland's CIDR 2007 paper puts it plainly: an entity, meaning a collection of data with a unique key, lives on a single machine at a time. Scale-out systems partition entities across machines, and the machine an entity lives on can change. At any instant the authoritative copy is somewhere specific.

That fact constrains designs that assume otherwise. A workflow that needs two entities to change together needs them on the same machine, or needs a mechanism that is not a transaction. Chapter 5 takes that consequence and turns it into a rule about where service boundaries go.

For this chapter the point is narrower. State has a location, the location is a failure domain, and pretending it is uniform is the most common way a design acquires a fault it never modelled.

A worked example, composited and labelled

The details here are composited from ordinary production shapes rather than one system, and the shape is exact.

A scheduling product has three write classes. Appointment bookings, which a customer sees confirmed on screen and which cannot be silently lost. Audit entries, which are written alongside every action and read only during investigations. And presence updates, which say who is currently viewing a calendar and are worthless after ten seconds.

One durability level for all three is the mistake. Bookings warrant a synchronous commit that survives the loss of the primary, because a booking a customer was shown and the system then forgot is the worst outcome available. Audit entries warrant durability too, since their whole value is being there later, but they tolerate commit delay because nobody is waiting. Presence updates warrant almost none: losing ten seconds of presence is invisible, and paying replication latency for them is spending money on nothing.

The design that follows keeps bookings and audit in the relational store with a strict commit setting, and moves presence to a store that never touches a disk. The read routing follows the same split. A customer reading back their own booking reads the primary. A manager reading last month's audit reads a replica, and the design document records that this read may be up to five seconds stale, which nobody minds.

The objection: pick the managed default and move on

The objection is reasonable, and it has a real version and a lazy version.

The real version says that a managed database's defaults are chosen by people who understand the engine better than you do, and overriding them without measurement is how teams create problems. That is correct. Do not tune the dial for sport.

The lazy version says the default is therefore the decision. It is not, because a default is a compromise chosen without knowledge of your write classes. The managed default is frequently a durable synchronous commit on the primary with asynchronous replication, which is a sensible middle. It is also exactly the setting that produces the read-your-own-write bug that opened this chapter, and it will produce it silently.

So the answer is not to change the default. It is to write down what the default gives you, per write class and per read path, and to note the case where it is wrong. That sentence costs an afternoon and it is the difference between a known behaviour and a mystery ticket queue.

Chapter summary

Stateless tiers are the easy half of a design because a request plus durable storage carries everything an instance needs, so capacity is a multiplication and any instance can be killed without loss. State inverts all of that: a durable store has an identity, copies of it are not interchangeable, and making them interchangeable costs a network round trip on every write. Gilbert and Lynch's 2012 perspectives paper frames the underlying impossibility as the general safety and liveness trade-off in an unreliable network, and PostgreSQL 18 makes the engineering version concrete by documenting five positions of synchronous_commit, from off, which the documentation says may lose recently acknowledged transactions without leaving the database inconsistent, to remote_apply, which waits for a standby to apply and make the write visible and which the documentation warns causes much larger commit delays. Replication lag then becomes a product decision rather than an operational detail, because it decides whether a user can read back their own write, so read paths should be routed by stated tolerance rather than by load balancer convenience. Abadi's PACELC argument is that this consistency and latency trade is paid continuously in normal operation and not only during a partition, which is the cost teams misfile under disaster recovery. Helland's rule that an entity lives on one machine at a time bounds how much distribution a design can pretend to, and the chapter's decision is a durability and staleness setting recorded per write class and per read path.

State has a location and a durability setting. What a reader is allowed to observe of it needs its own vocabulary, and that vocabulary is what makes the next four chapters arguable rather than aesthetic. Chapter 3 is Consistency You Can Explain to the Business.