Load Testing That Predicts Production

Load Testing That Predicts Production is the chapter that decides whether the previous eleven produced anything. A test is a claim about behaviour you have not observed yet, and most Laravel load tests are built in a way that guarantees a reassuring answer.

Key takeaways

• Grafana's k6 documentation states that when the target system is stressed and responds more slowly, a closed-model load test will wait, giving longer iteration durations and a tapering arrival rate.
• The same page names that effect coordinated omission, and states that under open-model executors the response times of the target system no longer influence the load on the target system.
• Virtual users is not a unit of load. Arrival rate is, which is why constant-arrival-rate and ramping-arrival-rate are the executors that map onto Chapter 2's arithmetic.
• k6 thresholds take the form aggregation, operator, value, support p(N) percentile expressions on trend metrics, and make k6 exit non-zero when they fail.
• Google's SRE book, chapter 22, names cold caches and slow start as a distinct failure mode, which is why an unwarmed first run measures your cache-fill path rather than your application.

Read this after Chapter 2, whose utilisation model is what a load test is trying to locate, and after Chapter 10, because a test that only requests pages will miss most of the traffic a component-driven interface generates. The argument that a result you cannot defend is worse than no result at all belongs to the evidence-before-shipping argument.

A team runs a load test before a campaign. Five hundred virtual users, thirty minutes, one endpoint.

The report says the ninety-fifth percentile held inside the agreed budget and that no requests failed. The campaign launches. The application falls over in eleven minutes.

Nothing in the test was faked. It measured something real. It measured the wrong thing.

A closed-model test waits, and waiting hides the wall

This is the mechanism that invalidates most load tests, and Grafana's k6 documentation states it plainly enough to quote.

With virtual-user executors, each iteration starts only after the previous one finishes. So when the target system is stressed and starts to respond more slowly, a closed-model load test will wait, resulting in increased iteration durations and a tapering off of the arrival rate of new virtual-user iterations. The k6 documentation names this coordinated omission.

Read the consequence against Chapter 2. Your test was meant to push arrival rate up until utilisation approached one. Instead, the moment service time rises, the test reduces arrival rate, so utilisation is held near a comfortable value by the test itself.

The system under test is therefore protected by the instrument measuring it. A closed model asks what happens with five hundred concurrent users, which is a question about your test harness. It does not ask what happens at four hundred requests per second, which is the question production will ask.

Open-model executors decouple load from latency

The fix is a different executor rather than a different number, and the k6 documentation is explicit about what it buys.

Open-model executors, constant-arrival-rate and ramping-arrival-rate, start iterations on a schedule rather than on completion. The documentation's own summary is that the response times of the target system no longer influence the load on the target system.

That is precisely the property you need to find saturation. Load continues arriving while the system degrades, so the queue grows, the tail extends, and the failure you were looking for actually happens inside the test window instead of after the campaign starts.

Closed models are not useless. They are right for a system with genuinely fixed concurrency, such as a worker pool draining a queue. For an HTTP application taking arrivals it does not control, the open model is the honest one.

Virtual users is not a unit of load

The phrase concurrent users survives because it sounds like a business quantity, and it is not one. Chapter 2's arithmetic explains why in one line: utilisation is arrival rate times service time divided by pool size, and virtual users appear nowhere in it.

Translate the business question into arrivals before you write the test. Expected peak sessions per minute multiplied by requests per session, divided by sixty, gives arrivals per second. Chapter 10 is where the second term comes from, because in a Livewire or Inertia application most requests are not page loads.

Then set the arrival rate to what you expect, and a second scenario to what you would need to survive. Two numbers. Both defensible.

The test that says five hundred virtual users is untranslatable into either. Ask anybody who ran one what arrival rate it produced and the answer is a shrug.

A cold cache is a different system

Google's SRE book, chapter 22, names this as a failure mode in its own right, and it is the single most common way a Laravel load test produces a number nobody should act on.

The first minute of a test against a freshly started application is measuring the cache-fill path. Chapter 6's application cache is empty. The database buffer pool has none of your hot pages. Chapter 11's OPcache has not compiled the code, and if opcache.validate_timestamps was left on with the documented two second revalidation frequency, the runtime is still checking files.

That chapter's recommendation is to handle cold caches by overprovisioning or ramping traffic slowly, which is also the recipe for a valid test. Warm the system with a low-rate phase, discard those measurements, and only then apply the load you care about.

The opposite mistake exists and is rarer. A test that runs the same three parameter values for thirty minutes achieves a cache hit ratio production will never see, and reports a capacity you do not have. Vary the parameters across a realistic key space.

State the cache condition in the report. A result without it is uninterpretable.

A single-endpoint test predicts one endpoint

The reason a single-endpoint test is misleading is not that it is incomplete. It is that the interactions between routes are where the shared pools from Chapter 2 are actually contended.

A test of the product listing alone never opens a write transaction, so it cannot find the lock contention from Chapter 5. It never runs the report query, so it cannot find the buffer pool eviction from Chapter 4. It hits one route's cache keys repeatedly, so it never produces the eviction pressure from Chapter 6.

Build the mix from your access log rather than from intuition. Take the top routes by request count over a real peak hour, keep their relative proportions, and include the write paths even where they are a small percentage, because a small percentage of writes can dominate contention.

Include the component traffic too. If Chapter 10's reading found that most requests are polling updates, a mix without them is not your application.

There is a data-shape requirement alongside the route mix, and it is easier to get wrong. A test that requests the same twenty product identifiers exercises twenty rows and twenty cache keys, so the buffer pool and the cache both behave far better than they will. Draw identifiers from a distribution resembling production, with a hot subset and a long tail, and the eviction behaviour from Chapter 6 becomes visible.

Authentication belongs in the mix as well. A test that runs entirely as a guest skips session reads, permission checks and per-user cache keys, which are exactly the parts that do not share work between users. Log in a pool of distinct accounts rather than reusing one, because one account produces one cache key and hides the whole problem.

Thresholds turn a test into a pass or a fail

A load test that produces a chart produces a conversation. A load test that produces an exit code produces a decision, and k6's thresholds are the mechanism.

The k6 documentation gives the form as an aggregation, an operator and a value, and its own examples are shaped like an average below a stated figure or a p(90) below one. Trend metrics support avg, min, max, med and p(N) for any percentile between 0 and 100, and thresholds commonly apply to built-in metrics including http_req_duration, http_req_failed, group_duration and checks. When a threshold fails, k6 exits non-zero, and a passing run exits zero.

That non-zero exit is the entire reason this belongs in a book about holding performance. It is what lets a pipeline refuse a change, which is Chapter 13's subject.

Two options change how a failing test behaves. abortOnFail stops the run as soon as the threshold evaluates false, and delayAbortEval postpones evaluation for a stated duration so metrics can accumulate first. The second exists because of the cold-cache problem above: without it, a warm-up phase fails your threshold and aborts a test that would have passed.

Write the thresholds as percentiles, never as averages. Chapter 1 explained why a mean has no customers in it.

Find saturation on purpose, then stop

The most valuable single run is not a pass. It is a ramping-arrival-rate test that keeps increasing until something bends, because that gives you a number you can plan against.

What you are watching for is the shape Chapter 2 predicted. Throughput rises linearly with arrival rate, then flattens while the tail percentile climbs sharply, then errors appear. The arrival rate at which the tail starts climbing is your usable capacity, and it is always lower than the rate at which throughput flattens.

Instrument the pools while you do it, not just the responses. Busy FPM children against pm.max_children, open database connections against the server limit, Redis latency, and queue backlog age from Chapter 7. The point of the run is to learn which pool bends first.

Then stop. A test pushed far past saturation tells you about failure cascades rather than capacity, and Google's SRE book chapter on those is more useful for that than a chart is.

Record the number with the date and the commit. Capacity is a moving figure and an undated one is folklore within a quarter.

The objection: staging is smaller than production

This is the strongest objection in the chapter, and it is largely correct. A test on a smaller machine with a smaller dataset cannot tell you production's absolute capacity.

What it can tell you is three things, and they are the things you actually need. Which pool saturates first, since that ordering is usually a property of the architecture rather than the hardware. Whether a change moved the tail percentile relative to the previous run on the same environment. And whether a plan degraded, since the query plans from Chapter 3 depend on data volume and statistics rather than on core count.

The third is the one that needs care. A staging dataset an order of magnitude smaller than production will choose different plans, so plan-sensitive tests need a realistically sized dataset even when the hardware is small. Size the data before you size the machine.

So the honest framing is comparative rather than absolute. Staging answers whether this change made things worse, on identical conditions, which is exactly what a regression gate needs. Production answers what your capacity is, and Chapter 1's instrumentation is how you read it there.

Chapter summary

Most Laravel load tests are built to produce a reassuring answer, because virtual-user executors close the loop between latency and load: Grafana's k6 documentation states that a stressed system responding more slowly makes the test wait, giving longer iterations and a tapering arrival rate, which it names coordinated omission, so the instrument protects the system from the load it was meant to apply. Open-model executors, constant-arrival-rate and ramping-arrival-rate, start iterations on a schedule, and the documentation's own statement is that response times no longer influence load, which is the property required to find saturation. Virtual users is not a unit of load, since Chapter 2's utilisation formula contains arrival rate and service time and nothing else, so translate expected sessions and requests per session into arrivals per second before writing anything. Google's SRE book, chapter 22, names cold caches and slow start as their own failure mode, which makes an unwarmed first run a measurement of your cache-fill path, compounded by an empty buffer pool and an uncompiled OPcache; warm with a low-rate phase, discard it, and vary parameters so the hit ratio is realistic rather than flattering. A single-endpoint test cannot find lock contention, buffer pool eviction or cache eviction, so build the mix from a real peak hour's access log including writes and component traffic. k6 thresholds, expressed as an aggregation, an operator and a value with p(N) percentiles on trend metrics, make the run exit non-zero, with abortOnFail and delayAbortEval controlling when that happens. And the run worth having is a ramp to saturation with the pools instrumented, recorded with its date and commit.

A number that nobody re-runs decays into folklore, and every improvement in this book is one refactor away from being undone. Chapter 13 is Staying Fast, which turns all of it into a gate.