crema

crema ☕️

A Go cache library with probabilistic revalidation and optional singleflight loading. It smooths refreshes near TTL expiry while deduplicating concurrent loads.

Features

• Smooth probabilistic revalidation near expiry
• Built-in singleflight loader (can be disabled)
• Zero external dependencies in the core module
• Pluggable storage (CacheProvider) and storage codecs (CacheStorageCodec)

Core functionality is covered by a high level of automated tests.

Revalidation Algorithm

Within the revalidation window, the cache reloads with probability

p(t)=1-e^{-k(w-t)}

where t is the remaining time and w the configured revalidation window. The steepness k is set so that $p(0)=0.999$, so the probability is $0$ when an entry enters the window and approaches $0.999$ as expiry nears. This avoids a fixed refresh point and helps smooth spikes near expiry.

How widely reloads actually spread depends on how often the key is requested: because the probability is drawn per request, frequently requested keys tend to reload earlier in the window than rarely requested ones.

This design is inspired by the following references:

• Cache Stampede: Avoiding Hot Spots in Distributed Caching Systems
• Sometimes I Cache | The Cloudflare Blog

Installation

go get github.com/abema/crema

Providers and codecs under ext/ are separate Go modules. Add only the modules your application uses. For example:

go get github.com/abema/crema/ext/golang-lru

Go 1.25 or newer is required.

Quick Start

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/abema/crema"
	golanglru "github.com/abema/crema/ext/golang-lru"
)

func main() {
	provider := golanglru.NewCacheProvider[crema.CacheObject[int]](128, time.Minute)
	cache := crema.NewCache(provider, crema.NoopCacheStorageCodec[int]{})

	value, err := cache.GetOrLoad(
		context.Background(),
		"answer",
		time.Minute,
		func(context.Context) (int, error) {
			// Database or computation logic here.
			return 42, nil
		},
	)
	if err != nil {
		panic(err)
	}

	fmt.Println(value)
}

Usage Notes

• CacheProvider: Responsible for persistence with TTL handling. Works with Redis/Memcached, files, or databases.
• CacheStorageCodec: Encodes/decodes cached objects. Swap in JSON, protobuf, or your own codec.
• CacheObject: A thin wrapper holding Value and absolute expiry (ExpireAtMillis).
• Revalidation fallback: Revalidation failures return a still-valid cached value by default. Missing or expired values still return the loader error.

Options

• WithRevalidationWindow(duration): Set how long before expiry probabilistic reloads may start (0 disables them)
• WithDirectLoader(): Disable singleflight and call loaders directly
• WithMaxLoadTimeout(duration): Set max duration for singleflight loaders and their synchronous cache writebacks (ignored with WithDirectLoader())
• WithRevalidationFallback(enabled): Enable or disable revalidation fallback (enabled by default)
• WithLogger(logger): Override warning logger for get/set failures
• WithMetricsProvider(metrics): Record cache and loader events
• WithNegativeCacheProvider(provider, ttl, isNegative): Cache absent loader results through a separate provider
• WithLoadErrorCacheProvider(provider, ttl, shouldCache): Cache selected load errors through a separate provider

The default singleflight load path runs in an internal goroutine. crema does not recover panics from that path, so application-provided loaders, callbacks, metrics providers, and cache providers must avoid panics or recover them at their own boundary. WithDirectLoader runs the loader in the caller's goroutine.

Negative Cache

Load results are not cached by default. Use WithNegativeCacheProvider when a loader result means that the value does not exist. This includes a successful empty result and errors such as sql.ErrNoRows or an application-specific ErrNotFound.

cache := crema.NewCache(
	provider,
	crema.NoopCacheStorageCodec[int]{},
	crema.WithNegativeCacheProvider(negativeProvider, 500*time.Millisecond, func(value int, err error) bool {
		return value == 0 && err == nil || errors.Is(err, ErrNotFound)
	}),
)

• Get does not use this cache. After a value-cache miss, GetOrLoad returns a live cached negative result without invoking the loader.
• The negative provider is a CacheProvider[CacheLoadResult[V]]; its TTL is independent of the value TTL passed to GetOrLoad.
• A still-valid value wins over a negative result during revalidation.
• Successful Set and Delete invalidate the negative result.
• An in-process provider such as Ristretto preserves error identity for errors.Is and errors.As.
• A NegativeCacheMetricsProvider can record negative-cache hits and stores.

WithLoadErrorCacheProvider is for errors that do not mean absence, such as a short-lived rate-limit or backend-unavailable error. Its provider is a CacheProvider[error]. It can be configured together with the negative cache; negative results take precedence when both predicates match.

Implementations

CacheProvider

CacheStorageCodec

NewBinaryCompressionCodec does not limit decompressed size by default. Use WithMaxDecompressedBytes when compressed values may come from an untrusted or shared backend.

MetricsProvider

MetricsProvider records cache operations and loader starts, reasons (miss, expired, revalidation, or get_error), errors, and concurrency. Direct loads always report a concurrency of 1. Embed BaseMetricsProvider and override only the callbacks you need.

Concurrency

Cache is goroutine-safe as long as its CacheProvider and CacheStorageCodec implementations are goroutine-safe.

Development

go generate
go test ./... $(find ./ext/ -type d -mindepth 1 -maxdepth 1 | sed 's|$|/...|') ./ext/madvfree/bench/...

Examples that use Redis or Valkey require a server on 127.0.0.1:6379; CI runs them separately with the required service.

Tools

• cmd/plot-revalidation: SVG plot generator for revalidation curves

Why "crema"?

Crema is the golden foam that forms on top of a freshly pulled espresso coffee shot. Like crema that gradually dissipates over time, this cache library probabilistically refreshes entries, ensuring your data stays fresh without the overhead of deterministic expiration checks.