Skip to main content

Choosing a Backend for Operational State

Everything Abblix OIDC Server keeps between requests (authorization grants, pushed authorization requests, CIBA and device-flow records, token status marks, replay-prevention markers) lives as expiring entries in whatever IDistributedCache implementation the host registers. The library deliberately registers none of its own. There is no default and no fallback, and the first protocol request fails fast if the host forgot to supply one, caught immediately rather than in production. AddDistributedMemoryCache() gets a sample running; this guide is about choosing what runs in production, because the choice carries one functional requirement and one security requirement that are easy to miss.

What actually lives in the cache​

Each entry kind arrives with its own TTL, driven by the matching option or token lifetime, which is why this store never needs a cleanup job; it expires everything itself:

  • Authorization grants: the state behind an issued authorization code, kept for the code's lifetime (per-client, 1 minute by default).
  • Pushed authorization requests (PAR): consumed atomically at the authorize endpoint.
  • Token status marks: the Revoked/Used verdicts written by the revocation endpoint, refresh-token rotation, and authorization-code reuse detection; each mark lives exactly as long as the token it condemns.
  • CIBA and device-flow requests: polled state, plus the device flow's user-code map and its brute-force rate-limit counters.
  • Replay-prevention markers: used jti marks for DPoP proofs, JWT-bearer assertions, and one-time client-authentication assertions, and the rolling HMAC secrets behind DPoP nonces.

Entries are serialized with protobuf (a JSON fallback covers host-supplied types the protobuf whitelist does not know). Encrypting them at rest is its own topic with its own pitfalls; the Production Hardening Checklist owns it.

Requirement one: shared, or multi-instance breaks​

The protocol itself hops between requests, and under a load balancer those requests land on different instances. With a per-instance cache (a plain memory cache on each pod), every one of these flows breaks:

  • Authorization code exchange: the code is minted on the node that served /authorize and redeemed on whichever node serves /token; if the grant is not in a shared store, redemption returns invalid_grant, intermittently, depending on the load balancer. The shared experiment below surfaces it deterministically.
  • PAR: the request_uri is pushed to one node and consumed on another.
  • CIBA and device flow: the client polls the token endpoint repeatedly, and each poll may hit a different instance; the user approves on yet another.
  • DPoP nonces: the rolling HMAC secret must be readable by every instance, or nonces issued by one node fail validation on the next and force clients into retry loops.
  • Rate limiting: per-instance counters multiply the rate-limit budget by the pod count, and collapse back to one budget once the counter lives in the shared cache.

So the first requirement is simply shared. This is also why the in-memory cache is a development tool, not a deployment option, even before durability enters the picture.

Requirement two: persistent, or a wipe un-revokes tokens​

The less obvious requirement is durability, and it comes from the token status and replay-prevention marks. Revocation in Abblix's stateless model works by exception, the standard stateless-JWT tradeoff: tokens carry their own validity, and the cache holds only the marks that condemn them, checked on every validation of a token that carries a jti. Lose the cache, and the marks are gone:

  • a revoked refresh or access token validates again, until its own natural expiry;
  • a rotated-out refresh token is accepted again;
  • used jti replay marks vanish, reopening the replay window for DPoP proofs, JWT-bearer assertions, and one-time client-authentication assertions within their validity period.

A wipe also drops in-flight authorization codes, PAR, and device flows, but those are short-lived and their loss is a brief availability blip clients retry through. The status marks are what make this a durability requirement rather than a self-healing one: they must outlive the tokens they condemn.

Nothing in the deployment fails visibly, which is what makes it a security requirement rather than an operational one; the persistence check in the last section makes that invisible failure visible on demand. Treat the cache as a security store and run it with persistence enabled, on encrypted disk, never as an ephemeral instance that is free to restart empty. For a store holding token status marks, prefer AOF with appendfsync everysec. That still loses up to about a second of writes on a hard crash, so a token revoked in that final second can come back: acceptable for most, and far better than RDB snapshots alone, which leave a minutes-to-hours window in which a crash silently loses everything marked since the last snapshot. appendfsync always closes the second-wide gap by fsyncing every write, but that lands on the token path and caps write throughput at the disk's fsync rate, so reach for it only if a sub-second window is unacceptable. AOF rewrites and snapshots also fork the process; the dataset here is small and short-lived, so the copy-on-write rarely spikes latency, but watch latencystats if you tune persistence aggressively. The Production Hardening Checklist makes the same point from the defense side; this is the storage decision that implements it.

Latency sits on the token path​

The backend's round-trip time lands directly in request latency:

  • Every validation of a jti-bearing token does one cache read for the status mark: introspection, userinfo, every flow that validates a token server-side.
  • Every DPoP-protected request adds a get-and-set pair for the replay mark, plus a nonce-secret read where DPoP nonces are enabled.
  • Consuming a PAR request, or redeeming a CIBA or device-code request, runs the get-and-remove protocol: half a dozen sequential cache round-trips, the price of resolving a redemption race to a single consumer on top of the plain IDistributedCache contract. A poll that finds the request still pending is a single read; the expensive protocol runs once, at redemption. On a 1 ms backend that is invisible; on a 20 ms cross-zone hop it is over a hundred milliseconds added to whichever endpoint runs the consume: the authorize endpoint for PAR, the token endpoint for CIBA and device-code.

Keep the cache topologically close to the server instances, and prefer backends where a warm, pooled round trip is measured in fractions of a millisecond (the first call after a reconnect also pays TLS setup, and Redis Cluster can add a redirect hop, so pool connections and keep them warm).

The candidates​

  • Redis is the canonical choice and what Abblix's own identity provider runs: shared, sub-millisecond, durable with the right persistence mode, and available on every cloud. If you take one recommendation from this page, it is a private, persistent, TLS-protected Redis close to the pods. Two settings are not optional for a security store. Set maxmemory-policy noeviction and size maxmemory with headroom: every entry here has a TTL, so any volatile-* or allkeys-* policy evicts token status and replay marks first under memory pressure, silently un-revoking tokens with no restart involved; Azure Cache for Redis and AWS ElastiCache both default to volatile-lru, so check and override it (noeviction fails writes closed, a visible, alertable error, instead of dropping security state silently). And know that persistence protects a crash-and-restart, not a failover: Redis replication is asynchronous, so a promoted replica can be missing the last writes, including a just-written revocation mark, whatever the primary's appendfsync. A single durable instance has no failover window; an HA pair trades some of the guarantee for uptime unless you force replica acknowledgement (WAIT/WAITAOF) at a latency cost.
  • SQL Server distributed cache (or an equivalent relational-backed IDistributedCache) inverts the trade: durable by nature and already inside many enterprises' operational comfort zone, at the cost of higher per-read latency (workable for modest traffic, measurable at scale).
  • In-memory (AddDistributedMemoryCache): development and single-instance evaluation only; it fails both requirements above by construction.
  • Anything else implementing IDistributedCache works functionally; the library never sees past the interface. But the exactly-once guarantees above assume linearizable single-key operations, so judge a candidate by that too, alongside the two requirements and the latency profile, not by its name.

One honesty note for the strictest deployments: IDistributedCache offers no atomic compare-and-set. Both exactly-once mechanisms here, the get-and-remove protocol behind PAR, CIBA, and device-code redemption and the get-then-set jti replay mark, are emulated on top of the interface and are only as strong as the backend's per-key linearizability. The get-and-remove uses a lock-token last-write-wins check; the replay mark is a plain get-then-set, a probabilistic defense under extreme concurrent replay, as the source itself documents. Redis (single-threaded per shard) and relational-backed caches (row locks) provide that linearizability, so on those the guarantees hold in practice; a backend that serves stale reads from replicas can let two consumers each believe they won, double-redeeming a code. A host that needs a hard guarantee can replace the affected seam with a backend-native atomic (Redis SET NX EX, INSERT ... ON CONFLICT DO NOTHING); one more interface, same pattern as every other seam in this cookbook.

Verify the deployment, not the config​

Two experiments prove the two requirements. Run them in staging with production-identical persistence, never against live traffic: in staging this proves nothing is lost, but against live traffic with persistence off (the very thing the second experiment tests), the restart drops all in-flight operational state and un-revokes every previously revoked token system-wide, not just your test one:

  • Shared: run two instances behind the balancer, complete an authorization-code flow with sticky sessions disabled, and repeat until both nodes have served each leg; no invalid_grant may appear.
  • Persistent: revoke a refresh token through the revocation endpoint, restart the cache backend, and present the revoked token again; it must still be rejected. If it validates, persistence is not actually on.

Where this sits in the bigger picture​

This page covers operational state, the short-lived entries the server itself expires. Clients are a different store with different durability rules (A Durable Client Store), and a queryable inventory of issued tokens is a build-out on separate seams (Token Inventory and Incident Revocation).