Heavy Middleware

Why the cost of middleware is multiplicative — and what belongs there versus what should be resolved on demand.

What Middleware Is For

Middleware runs on every request before the handler is reached. Its cost is multiplicative: a 50ms operation in middleware adds 50ms to the latency of every request the service processes, regardless of what that request is doing.

This makes middleware appropriate for a narrow class of work:

Stateless verification — parsing and cryptographically verifying a JWT; confirming a required header is present
Request enrichment — attaching a request ID, recording the start timestamp
Cross-cutting headers — CORS, security headers, content negotiation
Unconditional routing — protocol upgrades, canonical redirects

What these have in common: they are fast, they do not require I/O, and they apply meaningfully to every request.

What Ends Up There

In practice, middleware accumulates responsibilities beyond this boundary.

Database lookups per request. Loading a full user profile, checking feature flags against a database, querying organisation settings to attach to the request context. This feels convenient — the data is available on every subsequent handler. The cost is a database round-trip on every request, including requests that never use the loaded data.

Synchronous external service calls. Validating a token against a remote auth service, fetching permission sets from an external policy engine. A single network call in middleware can add hundreds of milliseconds to every request at its tail.

Eager context loading. Loading the full user object, team memberships, organisation configuration, and permission sets for every request — even for endpoints that need only a subset. The data feels "free" because it is already on the context object when the handler runs. The cost is paid regardless.

Business logic decisions. Redirecting users based on account state, applying pricing tier limits, modifying request content based on business rules. These belong in the application layer.

The Multiplicative Cost

A concrete example. A service handles 200 requests per second. Middleware performs a single database query to load the user's profile — average 30ms.

	Without the middleware query	With the middleware query
Per-request overhead	0ms	30ms
At 200 req/s	0ms wasted	6,000ms CPU per second
Endpoint that needs the profile	Query happens in handler	No extra cost
Endpoint that doesn't need the profile	No query	30ms wasted
Health check endpoint	No overhead	30ms overhead

The overhead is invisible under low load. Under sustained traffic it consumes headroom. Under spikes it becomes a bottleneck.

Tail latency is affected disproportionately. A slow database query that takes 200ms at its 99th percentile adds that 200ms to the 99th percentile of every endpoint — including those with no business need for the data.

The Lazy Load Pattern

The alternative to eager loading in middleware is on-demand resolution in the handler or application layer. Dependencies needed by a specific endpoint are resolved when that endpoint is reached.

# Eager — runs for every request, including those that don't need it
async def load_user_context_middleware(request: Request, call_next):
    user = await user_repository.find_by_token(request.headers["Authorization"])
    request.state.user = user  # always loaded, sometimes unused
    return await call_next(request)
 
 
# On-demand — resolved only when the handler declares the dependency
async def get_current_user(
    token: str = Depends(extract_token),
    user_repo: IUserRepository = Depends(get_user_repo),
) -> User:
    return await user_repo.find_by_token(token)
 
@router.get("/profile")
async def get_profile(user: User = Depends(get_current_user)):
    # user loaded here, only for this endpoint
    ...
 
@router.get("/health")
async def health_check():
    # no user loaded
    return {"status": "ok"}

The health check no longer pays the cost of a user lookup. Endpoints that need the user still get it. The cost is proportional to actual use.

The Diagnostic

If middleware is slow, every endpoint is slow. Profiling is straightforward: measure the latency of a request that invokes only the middleware chain, with a minimal handler. Any latency present before the handler runs is middleware cost.

Watch for: