This module is available to use in your EdgeWorkers functions to keep frequently used data in memory between events. The lru-cache module provides an in-memory, least-recently-used (LRU) cache with optional time-to-live (TTL) expiration, size-based eviction, and request coalescing.

A cache is scoped to the EdgeWorker instance that created it. Entries stored while handling one event are available to later events handled by the same instance.

🚧

The cache is not shared between EdgeWorkers instances or edge servers, and it is not persistent. Entries can disappear at any time, for example when the instance is restarted. Use the cache as an optimization only, and always be ready to load a value that is missing.

Entries are visible to every subsequent event handled by the same instance, regardless of which end user triggered it. Do not cache user-specific or sensitive data under keys that other requests can produce.

📘

Cached values are held in the EdgeWorker's memory and count towards its memory limit. Use max, and optionally maxSize, or ttl to bound how much the cache can grow.

LRUCache Object

There are three steps to using the lru-cache.

1. Create a new LRUCache at module scope.
   Create the cache outside of your event handlers so that the same cache, and its entries, are reused across events. If you create a cache inside a handler it is discarded when the handler finishes.

2. Store values with set(), or configure a fetchMethod and let fetch() store them for you.

3. Read from the cache with get() or fetch().

Consider this EdgeWorkers function that caches origin responses for one minute.

import { LRUCache } from 'lru-cache';
import { httpRequest } from 'http-request';

// (1) Create the cache once, at module scope.
const cache = new LRUCache({ max: 100, ttl: 60_000 });

export async function onClientRequest(request) {
  // (2) Look the value up in the cache.
  let body = cache.get(request.path);

  if (body === undefined) {
    const orginResponse = await httpRequest(`/origin/${request.path}`);
    if (!orginResponse.ok) {
      return request.respondWith(502, {}, `Origin returned ${orginResponse.status}`);
    }
    body = await orginResponse.text();

    // (3) Store the value for subsequent events.
    cache.set(request.path, body);
  }

  return request.respondWith(200, {}, body);
}

The first request for a path loads the body from the origin. Later requests for the same path, handled by the same EdgeWorker instance within 60 seconds, are served from memory.

👍

Use fetch() with a fetchMethod to combine the lookup and the load into a single call. Concurrent requests for the same key then share a single origin request.

Constructor

new LRUCache(options)

Creates a new cache. All options other than max are optional.

import { LRUCache } from 'lru-cache';

const cache = new LRUCache({
  max: 500,
  ttl: 5 * 60_000,
  updateAgeOnGet: true,
});

The constructor throws a TypeError if options is not an object, if max is missing or 0, or if an option has the wrong type. It throws a RangeError if a size or TTL limit is exceeded. See Errors.

Limits

import { LRUCache, MAX_SIZE } from 'lru-cache';

const cache = new LRUCache({
  max: 1000,
  maxSize: Math.min(MAX_SIZE, 1024 * 1024),
  sizeCalculation: (value) => value.length,
});

Keys

Keys must be strings or numbers. Keys are converted to strings, so 1 and '1' refer to the same entry. Any other key type throws a TypeError.

Values

Any JavaScript value can be stored, except undefined. Calling set(key, undefined) deletes the entry.

Values are stored by reference. If you store an object and then modify it, later reads return the modified object.

Eviction

When adding an entry would exceed max or maxSize, the cache evicts the least recently used entries until the new entry fits.

get(), set(), and fetch() hits mark an entry as most recently used. has(), peek(), info(), and getRemainingTTL() do not change the order.

Time-to-live

Set a default ttl in the constructor, or a per-entry ttl in set() or fetch().

• Time is measured using the event time, which is the same value Date.now() returns in the event. Note that time does not advance within a single event.
• By default, an entry's TTL is measured from the time it was stored. Use the start option to measure it from a different point in time, in epoch milliseconds.
• An entry becomes stale when its TTL has elapsed. Stale entries are not returned unless allowStale is set.
• get() deletes a stale entry when it reads it, unless noDeleteOnStaleGet is set. Other stale entries stay in the cache until they are read, evicted, removed by purgeStale(), or removed by ttlAutopurge.

const cache = new LRUCache({ max: 100, ttl: 60_000 });

cache.set('short', 'value', { ttl: 5_000 });   // Expires after 5 seconds.
cache.set('default', 'value');                 // Expires after 60 seconds.
cache.set('forever', 'value', { ttl: 0 });     // Does not expire.

Size Tracking

A cache is size-constrained when maxSize or maxEntrySize is set. Every entry in a size-constrained cache needs a size. The cache uses the size option passed to set() or fetch() if present, otherwise it calls sizeCalculation. If neither is available, set() throws a TypeError.

Entries larger than maxEntrySize are not stored. If the key already had an entry, that entry is removed. Pass a status object to detect this case through status.maxEntrySizeExceeded.

const cache = new LRUCache({
  max: 1000,
  maxSize: 64 * 1024,
  maxEntrySize: 8 * 1024,
  sizeCalculation: (value, key) => key.length + value.length,
});

cache.set('greeting', 'hello');               // Size computed by sizeCalculation.
cache.set('blob', someString, { size: 4096 }); // Size provided explicitly.

Properties

All properties are read-only and reflect the options used to create the cache.

Methods

clear()

Removes all entries from the cache.

cache.clear();

delete()

Removes an entry. Returns true if an entry was removed, or false if the key was not in the cache.

If a fetch() is in flight for the key, its result is discarded and pending fetch() calls for the key reject with 'deleted'.

cache.set('a', 1);
cache.delete('a'); // true
cache.delete('a'); // false

fetch()

Returns a Promise that resolves with the cached value, or loads the value with the configured fetchMethod and stores it.

• If a fresh value is cached, the promise resolves with it and fetchMethod is not called.
• If no value is cached, or the value is stale, fetchMethod is called and the promise resolves with its result.
• If fetch() is already in flight for the key, the call waits for that load instead of starting another one.
• If no fetchMethod is configured, fetch() behaves like get().

import { LRUCache } from 'lru-cache';
import { httpRequest } from 'http-request';

const products = new LRUCache({
  max: 500,
  ttl: 5 * 60_000,
  allowStale: true,
  fetchMethod: async (key) => {
    const fromOrigin = await httpRequest(`/api/products/${key}`);
    if (!fromOrigin.ok) {
      throw new Error(`Origin returned ${fromOrigin.status}`);
    }
    return await fromOrigin.json();
  },
});

export async function onClientRequest(request) {
  try {
    const product = await products.fetch(id);
    request.respondWith(200, { 'Content-Type': ['application/json'] }, JSON.stringify(product));
  } catch (error) {
    request.respondWith(502, {}, 'Product unavailable');
  }
}

📘

When fetchMethod needs to be called, fetch() must run inside an event handler. Calling it from module scope, or from a promise that outlives the event that created it, rejects with a TypeError. The event that starts a load stays active until fetchMethod settles, even if it returned a stale value immediately.

forceFetch()

Same as fetch(), but the returned promise rejects if the result is undefined. Use it when a missing value should be treated as an error.

When the result is undefined, the promise rejects with the string 'fetch() returned undefined'.

try {
  const config = await cache.forceFetch('config');
  // config is never undefined here.
} catch (error) {
  // fetchMethod failed or returned undefined.
}

forceFetch() takes the same parameters as fetch(). When a status object is passed, status.op is set to 'fetch'.

get()

Returns the value for a key, or undefined if the key is not in the cache. Marks the entry as most recently used.

If the entry is stale, get() returns undefined and deletes the entry. Set allowStale to return the stale value instead, and noDeleteOnStaleGet to keep the entry.

const cache = new LRUCache({ max: 10 });
cache.set('color', 'blue');

cache.get('color');   // 'blue'
cache.get('missing'); // undefined

getRemainingTTL()

Returns the remaining TTL of an entry, in milliseconds.

const cache = new LRUCache({ max: 10, ttl: 60_000 });
cache.set('a', 1);

cache.getRemainingTTL('a');       // 60000
cache.getRemainingTTL('missing'); // 0

has()

Returns true if a non-stale entry exists for the key. Does not change the recency of the entry.

const cache = new LRUCache({ max: 10 });
cache.set('a', 1);

cache.has('a'); // true
cache.has('b'); // false

info()

Returns details about an entry, or undefined if the key is not in the cache. Does not change the recency or TTL of the entry.

const cache = new LRUCache({ max: 10, ttl: 60_000 });
cache.set('a', 'value');

cache.info('a');
// { value: 'value', ttl: 60000, start: 1792490924343 }

The returned object has the following properties.

Tip

The ttl, start, and size properties can be passed directly to set() to copy an entry into another cache with the same remaining lifetime.

peek()

Returns the value for a key without any side effects. peek() does not change the recency of the entry, renew its TTL, or delete it if it is stale.

If the entry is stale, peek() returns undefined unless allowStale is set.

cache.set('a', 1);
cache.set('b', 2);

cache.peek('a'); // 1, 'a' is still the least recently used entry.
cache.pop();     // 1

pop()

Removes the least recently used entry and returns its value. Returns undefined if the cache is empty.

cache.set('a', 1);
cache.set('b', 2);

cache.pop(); // 1
cache.pop(); // 2
cache.pop(); // undefined

purgeStale()

Removes all stale entries. Returns true if any entries were removed.

Use purgeStale() to reclaim memory from expired entries without waiting for them to be read or evicted. To do this automatically, set the ttlAutopurge option instead.

set()

Stores a value under a key and marks the entry as most recently used. If the key already exists, its value is replaced.

Passing undefined as the value deletes the entry.

const cache = new LRUCache({ max: 100, ttl: 60_000 });

cache.set('a', { id: 1 });
cache.set('b', 'short lived', { ttl: 1_000 });

If a fetch() is in flight for the key, the fetched value is discarded and pending fetch() calls for the key reject with 'replaced'.

fetchMethod

The fetchMethod option is a function that loads a value for fetch() and forceFetch(). It can return a value or a Promise. It is called with this set to undefined.

fetchMethod(key, staleValue, { options, context })

This example caches "not found" responses for a shorter time than successful ones.

import { httpRequest } from 'http-request';
import { LRUCache } from 'lru-cache';

const cache = new LRUCache({
  max: 1000,
  ttl: 10 * 60_000,
  fetchMethod: async (key, staleValue, { options }) => {
    const response = await httpRequest(`/api/items/${encodeURIComponent(key)}`);
    if (response.status === 404) {
      options.ttl = 30_000;
      return null;
    }
    if (!response.ok) {
      throw new Error(`Origin returned ${response.status}`);
    }
    return response.json();
  },
});

Concurrent Fetches

Calls to fetch() for the same key share a single fetchMethod call while it is in flight, including calls made by other events handled by the same instance. All callers receive the same result.

// fetchMethod is called once. Both promises resolve with the same value.
const [a, b] = await Promise.all([cache.fetch('key'), cache.fetch('key')]);

Fetch Failures

When fetchMethod throws or returns a rejected promise:

• By default, the stale entry is deleted and fetch() rejects with the error.
• With allowStaleOnFetchRejection, fetch() resolves with the stale value, or undefined if there is none.
• With noDeleteOnFetchRejection, the stale entry is kept in the cache.

If the entry is deleted, replaced, or evicted while the fetch is in flight, the result is discarded and status.fetchAborted is set. Pending fetch() calls for the key reject with 'deleted', 'replaced', or 'evicted'.

Status Object

Most methods accept a status option. When you pass an object, the method populates it with details about the call. This is useful for logging and debugging.

import { logger } from 'log';

const getStatus = {};
const value = cache.get('config', { status: getStatus });
logger.log(`get ${status.key}: ${status.get}`); // For example, 'get config: hit'

Properties are only set when they apply to the call.

Errors