Staying Fast: Budgets and Regression Gates

Staying Fast: Budgets and Regression Gates is the chapter that decides whether any of the previous twelve mattered in six months. Performance work decays by default, because every feature added after yours is written by somebody who does not know what you measured or why.

Key takeaways

• A budget is a number with a consequence. Grafana's k6 documentation supplies the consequence mechanically, since a failed threshold makes k6 exit non-zero and a pipeline can act on that.
• Express budgets as percentiles for a named route, never as an application-wide mean, because a mean can absorb a serious regression on a minority of requests without moving.
• The cheapest gates are structural rather than temporal: query count per request, and Model::preventLazyLoading turning a lazy load into a test failure, both run in a normal test suite in seconds.
• Pulse is the production half of the same idea, with per-recorder slow thresholds defaulting to 1,000 milliseconds, per-pattern overrides, sampling that marks values with a tilde, and silent failure if capture throws.
• Google's Core Web Vitals page, last updated 31 October 2024, sets LCP at 2.5 seconds, INP at 200 milliseconds and CLS at 0.1, each assessed at the 75th percentile.

Read this after Chapter 12, which produced the numbers this chapter turns into gates, and after Chapter 1, which explained why a percentile rather than a mean. The general case for a gate rather than a review is the evidence-before-shipping argument.

A team spends a quarter on the query layer. The listing routes get demonstrably better and everybody is pleased.

Eight months later the same routes are slow again. Nobody did anything wrong. Fourteen features shipped, each adding a relationship touch, a prop, a poll, and none of them individually visible.

Nobody removed the improvement. It was diluted, one pull request at a time.

A budget is a number with a consequence

A performance budget is not a target, an aspiration or a dashboard. It is a stated value, for a named thing, with something that happens automatically when the value is exceeded.

The consequence is the whole of it. A budget with no consequence is a number in a document, and documents lose to deadlines every time. Every single time. A budget that fails a build wins, because the person who broke it finds out while they still remember what they changed.

Three properties make one usable. It names a specific route or operation rather than the application. It uses a statistic that cannot hide a minority, meaning a percentile. And it fires without a human deciding to look.

Everything else in this chapter is an instance of those three.

Express it as a percentile, for a named route

Chapter 1 argued that an average is the wrong summary of a latency distribution, and a budget inherits that argument entirely.

An application-wide mean can absorb a serious regression. Make one route in twenty much slower and leave the rest alone: the mean barely moves, while the ninety-ninth percentile moves a great deal, and the customers in that percentile are disproportionately the ones completing purchases. A mean budget therefore permits exactly the failure you care most about.

So write budgets per route, at a percentile, with the route named. The checkout submission at the ninety-fifth percentile. The product listing at the ninety-ninth. Different routes deserve different numbers, because their tolerance genuinely differs.

Include the structural budgets alongside the temporal ones. Queries per request for a route is a better regression signal than milliseconds, because it is deterministic, it does not vary with machine load, and it is the metric that catches an N+1 the day it arrives.

The gate is the exit code

Everything else about a gate is decoration. What makes it real is a process that ends with a non-zero status and a pipeline configured to care.

Grafana's k6 documentation gives you that directly. Thresholds are written as an aggregation, an operator and a value. They support p(N) percentile expressions on trend metrics. They apply to built-in metrics including http_req_duration and http_req_failed. And when one fails, k6 exits non-zero, while a passing run exits zero.

Two settings shape the failure. abortOnFail stops the run as soon as a threshold evaluates false, which saves pipeline minutes on an obviously broken build. delayAbortEval postpones evaluation for a stated duration, which is what you use to stop Chapter 12's warm-up phase from aborting a test that would have passed.

Keep the gated test small and keep it warm. A ten minute load test in a pull-request pipeline gets disabled within a month, and a disabled gate is worse than none, because it produces the false belief that something is watching.

The cheap gates run on every commit

Load-testing gates are valuable and slow. There is a second tier that costs almost nothing and catches most regressions before they reach a load test at all.

Turn lazy loading into a test failure. Chapter 3's Model::preventLazyLoading, guarded so it is active outside production, throws when Eloquent lazily loads a relationship, which means the N+1 introduced this morning fails the suite this morning.

Assert query counts on the routes that matter. A feature test that hits a listing route and asserts the query count is below a stated number is a few lines using the DB::listen instrumentation from Chapter 1, and it fails deterministically rather than flakily.

Assert payload width where you have an API contract. A test that fails when a resource starts returning fourteen fields instead of six catches the over-fetching from Chapter 4 at the point it is introduced.

None of these three measure time, which is exactly why they belong in CI. They are invariants rather than measurements. They do not flake on a shared runner. That is the entire appeal.

Pulse is the production half of the same gate

A CI gate protects you from your own changes. It does not protect you from traffic growth, data growth or a dependency degrading, and for those you need the same thresholds applied to production.

Laravel's Pulse documentation gives you the machinery. Its recorders carry default slow thresholds of 1,000 milliseconds with per-pattern overrides, so a route you know is legitimately slow can carry its own number instead of polluting the signal. That per-pattern override is the feature that makes Pulse usable as a budget rather than a noise generator.

Three documented behaviours change how you read it, and all three are in the same documentation. Pulse samples and scales sampled values, prefixing them with a tilde, so a Pulse figure is a population estimate. Its Redis ingest requires Redis 6.2 or greater on a connection separate from the queue. And Pulse fails silently if capture throws, which means a flat line is not proof of a quiet system.

So treat Pulse as the alarm and Chapter 1's in-request instrumentation as the evidence. The alarm tells you to look. The instrumentation tells you what happened. Keep both.

Core Web Vitals are a budget somebody else already set

For anything user-facing there is an externally defined budget you can adopt rather than invent, and adopting it saves an argument.

Google's web.dev Web Vitals page, last updated 31 October 2024, sets Largest Contentful Paint at 2.5 seconds, Interaction to Next Paint at 200 milliseconds and Cumulative Layout Shift at 0.1, each assessed at the 75th percentile of page loads and segmented across mobile and desktop. That page is older than eighteen months at the time of writing, so check it before you commit a number to a contract.

Two of those three are partly yours even though they are measured in a browser. Largest Contentful Paint includes your server response time, so every chapter of this book feeds it. Interaction to Next Paint includes the round trip from Chapter 10, because in a server-driven interface an interaction is frequently a request.

Note the percentile choice, because it is the same argument this chapter has made twice. Google assesses at the 75th percentile rather than the mean, for the same reason your budgets should.

The last row is deliberately not a gate. Field data arrives too late to block a merge, and treating it as a gate produces a queue of blocked releases waiting for data that has not been collected yet.

Loosening a budget is a decision, and it gets written down

Every budget will eventually be wrong, and pretending otherwise is how gates get deleted rather than adjusted.

A feature may legitimately make a route slower. A new report may genuinely need more queries. When that happens there are two honest paths and one dishonest one. Change the code to fit the budget, or change the budget and record why.

The dishonest path is deleting the assertion in the same commit as the feature, with a message about updating the test. It is invisible in review, and it converts a gate into a formality.

So make loosening cost a paragraph. The new value, the reason, the name of whoever agreed, the date, and the condition under which it should be tightened again. Put it next to the budget in the repository rather than in a ticket system, so the next person reading the number also reads the argument.

That record is also how you detect drift you would otherwise never see. Four loosenings in a year on the same route is a finding, and it is the finding that sends you back to Chapter 1.

The objection: a gate will block every release

The fear is reasonable and the failure it describes is real. A flaky performance gate is worse than no gate, because it trains a team to bypass it, and once bypassing is normal the gate is gone.

The answer is to choose gate metrics by their variance rather than their importance. Query count does not vary with machine load. A lazy-loading exception does not vary at all. A latency percentile on a shared CI runner varies a great deal, which is why it belongs on a dedicated environment with warmed caches and a threshold set from observed variance rather than from a hope.

Measure the gate itself, which almost nobody does. Run it twenty times against an unchanged commit and look at the spread of results. If it fails once in twenty with no code change, it will be disabled within a month, and you have learned that before spending the team's trust rather than after.

Then give failure a route that is not deletion. A failing budget opens a decision: fix, or loosen with a record. Both are acceptable. Silently removing the check is not.

Chapter summary

A performance budget is a stated value for a named thing with an automatic consequence, and the consequence is the entire mechanism, since a number in a document loses to a deadline every time. Express budgets as percentiles for named routes rather than application-wide means, because a mean absorbs a severe regression on a minority of requests, and add structural budgets like queries per request, which are deterministic where timings are not. Grafana's k6 documentation supplies the gate mechanically: thresholds are an aggregation, an operator and a value, they support p(N) on trend metrics, they apply to metrics including http_req_duration and http_req_failed, and a failure exits non-zero, with abortOnFail stopping the run early and delayAbortEval protecting a warm-up phase. Below that sits a tier of cheap gates that run in a normal test suite and measure no time at all: Model::preventLazyLoading turning an N+1 into a failing test, an asserted query count per route built on DB::listen, and an asserted payload width for an API resource. Pulse is the production counterpart, with per-recorder slow thresholds defaulting to 1,000 milliseconds and per-pattern overrides, sampling that marks estimated values with a tilde, a Redis ingest needing 6.2 or greater on a connection separate from the queue, and silent failure if capture throws. Google web.dev's Web Vitals page, last updated 31 October 2024, gives externally defined budgets of 2.5 seconds for LCP, 200 milliseconds for INP and 0.1 for CLS, each at the 75th percentile. Loosening a budget is legitimate and must be recorded with value, reason, owner and date. And choose gate metrics by variance, measuring the gate's own flakiness before trusting it.

That is the book. Its argument in one line: Laravel applications that slowed down as they succeeded are slow for a small number of findable reasons, so measure before touching anything, understand that every bottleneck is a queue in front of a fixed pool whose utilisation is arrival rate times service time over pool size, then search in a fixed order, being the query layer, then reading at volume, then write contention, then cache invalidation, then queue saturation, then the process model, then the bytes on the wire, then the traffic your own interface generates, and only then the runtime, and finally hold the result with load tests you can believe and budgets that fail a build. The measure of whether it worked is not a percentage you can put in a deck. There is no such number here, deliberately. It is that the next traffic increase becomes a capacity calculation somebody performs in an afternoon, rather than an incident nobody saw coming.