Cache invalidation without selling a stale promise

Define what a cached answer is allowed to mean

Cache the public schedule for one observatory and calendar day: instrument labels, slot start times, closure state, and advisory availability. Do not put member names, email addresses, access instructions, or personalized reservation details into this shared representation.

A free slot in this response means “free when this snapshot was read.” It does not mean “reserved for the person looking at it.” The booking endpoint checks current slot state and inserts ownership transactionally. If another member booked in the meantime, an ordinary conflict is the correct result. The storage and consistency lesson develops that write protocol.

Return a snapshot timestamp or age indicator with the public data. A browser must stop presenting the data as current once its allowed age expires, even if the tab stays open. Cache freshness at the server does not automatically refresh a browser that fetched the page ten minutes ago.

Choose a key that describes one representation

Use a key such as availability:v1:observatory-12:2030-04-18:UTC. This is an example application key, not a Unograph model identifier. The version prevents an older response shape from being read as a new one. The observatory and date bound the payload. The date is interpreted in a documented timezone; in the real product, use the observatory's agreed calendar rules consistently.

Do not add a member ID to rescue accidental personalization in a public endpoint. Split public and private responses first. An account reservation response uses a separate authenticated route and the example policy Cache-Control: no-store. Authorization must still be enforced: a cache directive is not access control.

RFC 9111's response directives distinguish no-store, which prohibits storage by compliant HTTP caches, from no-cache, which permits storage but requires validation before reuse. An application cache also needs an explicit policy; HTTP headers alone do not configure the server's key-value cache.

Start with a bounded cache-aside read

The API checks the shared availability cache. On a usable hit, it returns the representation and its original age. On a miss, it reads a fresh snapshot from the primary database, stores it with a bounded lifetime, and returns it. Begin with one application caching layer; additional browser or CDN reuse needs its own age accounting.

For the exercise, choose a four-second cache validity window anchored to the source snapshot, leaving one second of the five-second target for clock uncertainty, transport, and rendering. Require a bounded source query, a bounded response path, and monitored clocks. These allocations must be demonstrated together under the chosen workload.

Record a conservative sourceReadAt no later than the acquisition of the database snapshot, and an absolute expiresAt = sourceReadAt + 4 seconds. A slow fill receives only its remaining lifetime. Do not start a new four-second window when a delayed result is inserted. Hits do not renew the timestamp or expiry.

If a source query exceeds its deadline or returns after the candidate has expired, discard the candidate. Use the same source timestamp when serializing a response so a browser can enforce the remaining allowance. This is a design contract that must account for clock skew between the database, API, cache, and browser; absent that accounting, report freshness as best effort rather than a hard bound.

Walk through the delete-and-refill race

Consider this sequence. At 10:00:00.000, reader A misses the cache and begins reading a snapshot where T7 is free. At 10:00:00.080, writer B books T7 and commits. At 10:00:00.100, the invalidation worker deletes the day's cache key. At 10:00:00.180, reader A finishes and inserts its older snapshot.

The delete ran successfully, yet the cache now contains stale data. “Invalidate after writing” alone does not prevent an in-flight reader from refilling an old value. Reversing the order is also unsafe: deleting before commit lets another reader fetch the old database state while the transaction is still pending.

For NightLens discovery, we accept this race within the bounded advisory window. A's candidate expires at 10:00:04.000, not four seconds after its late insertion. The booking transaction remains correct throughout. If the product demanded immediate visibility after every change, this simple policy would not satisfy it.

A stronger option is version fencing. Advance a generation for each observatory-day in the same transaction as the source change. Cache fills carry the generation observed with their snapshot. A monotonic watermark rejects fills older than an already observed generation. Implementing the comparison and update atomically, handling worker restart, and detecting missing events add real complexity. Do not introduce it until the freshness requirement justifies the extra state.

Use invalidation to reduce lag, and expiry to limit forgotten work

Booking, cancellation, closure, and reopening all change public availability. Each transaction records an outbox event identifying the affected observatory-day key. A worker deletes that key after commit. Delivering an invalidation more than once is harmless; an older invalidation can cause an extra miss but must not recreate an older value.

The transactional outbox guidance describes committing event intent with the data change and handling duplicate downstream delivery. Here, the outbox reduces the chance that a process crash loses the invalidation request. Expiry remains necessary when delivery is delayed or a consumer is unavailable.

Do not claim the outbox makes the cache instantly consistent. The event may wait, a fill may race, or the cache may be unreachable. Monitor oldest invalidation age and observed snapshot age, and alert on sustained breaches. A high hit rate does not tell you whether the answers are fresh.

Design for a cold cache and a hot key

At the Monday release, hundreds of members may request the same day. If every miss triggers a database read, the cache can fail precisely when it is most needed. Coalesce concurrent fills for the same key so followers share one bounded result. A process-local coalescer helps within one replica; several replicas may still initiate separate fills.

A shared fill lease can reduce duplicate work further, but it needs expiry, a clear owner, and a follower deadline. It must never be reused as a booking lock. If the fill owner disappears, followers should stop waiting and follow the admission policy. An optimization must not become an indefinite queue.

Use short randomized expiry within the accepted budget, for example a source-anchored lifetime between three and four seconds. It can spread refresh work across many keys, but one extremely popular key still needs coalescing or refresh control. Never let jitter extend the maximum tolerated age.

During cache failure, cap database fallback concurrency rather than converting every failed cache call into an unrestricted database read. Serve a still-valid cached value only when its age can be established. Otherwise return an explicit temporary unavailability for availability data, while static instrument descriptions may remain visible. The next lesson explains how those bounds protect the write path.

Practice: test freshness rather than only cache hits

Take a single day key and script five scenarios: a booking, a cancellation, a closure, an invalidation-worker pause, and a slow reader that fills after invalidation. Observe the source timestamp and booking outcome in each case. Use the controlled-interleaving method to hold the old read open while the booking commits and invalidation runs; a test that performs them only one after another will miss the refill race.

1. Pause the worker for thirty seconds. Expiry should still prevent unlimited use of an old snapshot.
2. Delay a source read beyond its absolute expiry. Its result must not be cached or returned as fresh.
3. Keep a browser tab open without polling. Its freshness label must expire locally.
4. Drop the cache connection under peak traffic. Database fallback must remain bounded.

A correct result is not “zero conflicts.” Members can legitimately race for the same slot. A correct result is no duplicate ownership, no unlabelled use beyond the freshness contract, and a controlled fallback load. Record cache hit ratio, source age, fill duration, coalesced waits, invalidation lag, and fallback rejection counts.

Your next artifact is a cache policy for each representation: owner, key, source, age budget, invalidating operations, and failure response. Continue to Backpressure, timeouts, and retries during a reservation surge to decide which work the system should decline when those limits are reached.