How would you design an in-memory image cache?

2.7.2 · L3 · Interview

Predict first

Two list cells request the same uncached image URL at the same moment. How many network requests should a well-designed cache send?

// A: 2
// B: 1
// C: 0
  • A
  • B
  • C

Mental model. A good cache is a librarian who, when two people ask for the same book, fetches it once and hands both a copy.

Start from the access pattern. Cache decoded images at the size you display them (decoding is the expensive part), keyed by URL plus size, with a cost equal to the byte count so totalCostLimit reflects real memory. Put NSCache in front and let the system evict. Add a disk layer or URLCache beneath it for downloaded bytes.

The subtle part is concurrency. If two screens ask for the same image at once, a naive cache issues two network requests. Track in-flight requests in a dictionary protected by a lock or an actor and let the second caller await the first (request coalescing). Decide invalidation (TTL, versioned keys), and never assume a hit: the miss path is the real code.

One. The second caller joins the request already in flight, instead of starting its own.

NSCacheDictionary + lockHand-written LRUURLCache
Evictionsystem decidesneverleast recently usedsize limits and headers
Thread-safeyesyou add ityou add ityes
Survives relaunchnononooptional (disk)
Best fordecoded images, derived valuessmall bounded mapspredictable evictionHTTP responses

Common traps

  • Caching the downloaded bytes only. You still decode on every use. Cache the decoded image too.
  • A cache without in-flight tracking. A scrolling list can fire dozens of identical requests.

They might follow up with

  • Where does URLCache fit? It caches HTTP responses according to cache headers and size limits, on memory and disk. Use it for network bytes, not for decoded values.
  • When would you hand-roll an LRU? When eviction must be predictable, for example in tests or on a strict memory budget.

Related questions

  • 2.7.1 What is NSCache, and how is it different from a Dictionary?
  • 2.7.3 NSLock, NSRecursiveLock, os_unfair_lock, Mutex, serial queue: how do you choose?

← What is NSCache, and how is it different from a Dictionary?   NSLock, NSRecursiveLock, os_unfair_lock, Mutex, serial queue: how do… →