icache

iCache - Generic Two-Layer Cache Package for Go

A high-performance, production-ready two-layer cache package for Go with local in-memory cache (L1) and pluggable distributed cache backend (L2). Use any storage backend - Redis, Memcached, DynamoDB, or implement your own!

📚 Documentation

• Quick Start Guide - Get started in 5 minutes
• Testing Guide - Comprehensive testing strategies
• Migration Guide - Migrating from Redis-only version
• Examples - Working code examples

Features

• Two-Layer Architecture: Fast local cache with optional distributed backend
• Backend Agnostic: Works with any storage backend through a simple interface
• Cache Invalidation: Automatic cache invalidation across multiple instances via pub/sub (if backend supports it)
• Cache Stampede Prevention: Built-in mutex protection to prevent concurrent requests for the same key
• Generic Support: Full Go generics support for type-safe operations
• Fetcher Functions: Automatic cache population on miss
• TTL Support: Flexible expiration times for cache entries
• ARC Eviction Policy: Adaptive Replacement Cache for optimal memory usage
• Local-Only Mode: Can be used without any backend for simple in-memory caching

Installation

go get github.com/bulutaysarac/icache

Quick Start

Local Cache Only (No Backend)

package main

import (
    "context"
    "time"
    
    "github.com/bulutaysarac/icache"
)

func main() {
    ctx := context.Background()
    
    // Initialize cache without backend
    config := icache.Config{
        LocalCacheSize: 10000,
        InstanceID:     "instance-1",
    }
    
    cache := icache.New(config)
    defer cache.Close()
    
    // Set a value
    cache.Set(ctx, "user:1", map[string]string{"name": "John"}, 5*time.Minute)
    
    // Get a value
    if user, ok, err := icache.Get[map[string]string](cache, ctx, "user:1"); err == nil && ok {
        println("User:", user["name"])
    }
}

With Custom Backend

package main

import (
    "context"
    "time"
    
    "github.com/bulutaysarac/icache"
)

func main() {
    ctx := context.Background()
    
    // Initialize cache with your custom backend
    backend := NewMyCustomBackend() // Implement icache.Backend interface
    
    config := icache.Config{
        Backend:        backend,
        LocalCacheSize: 10000,
        InstanceID:     "instance-1",
    }
    
    cache := icache.New(config)
    defer cache.Close()
    
    // Use the cache
    cache.Set(ctx, "key", "value", 5*time.Minute)
}

Implementing a Custom Backend

To use icache with any storage system, implement the Backend interface:

type Backend interface {
    // Get retrieves a value from the backend
    Get(ctx context.Context, key string) (string, bool, error)
    
    // Set stores a value in the backend with optional expiration
    Set(ctx context.Context, key string, value string, expiration time.Duration) error
    
    // Del deletes a value from the backend
    Del(ctx context.Context, key string) error
    
    // TTL returns the time-to-live for a key
    // Returns -1 if the key exists but has no expiration
    // Returns -2 if the key does not exist
    TTL(ctx context.Context, key string) (time.Duration, error)
    
    // Publish publishes a message to a channel for cache invalidation
    // Can be a no-op if you don't need multi-instance cache invalidation
    Publish(ctx context.Context, channel string, message string) error
    
    // Subscribe subscribes to a channel for cache invalidation messages
    // Should return a channel that receives messages
    // Can return nil if you don't need multi-instance cache invalidation
    Subscribe(ctx context.Context, channel string) (<-chan Message, error)
    
    // Close closes any connections to the backend
    Close() error
}

Example: Redis Backend

package main

import (
    "context"
    "time"
    
    "github.com/bulutaysarac/icache"
    "github.com/redis/go-redis/v9"
)

type RedisBackend struct {
    client *redis.Client
}

func NewRedisBackend(addr, password string, db int) *RedisBackend {
    return &RedisBackend{
        client: redis.NewClient(&redis.Options{
            Addr:     addr,
            Password: password,
            DB:       db,
        }),
    }
}

func (r *RedisBackend) Get(ctx context.Context, key string) (string, bool, error) {
    val, err := r.client.Get(ctx, key).Result()
    if err == redis.Nil {
        return "", false, nil
    }
    if err != nil {
        return "", false, err
    }
    return val, true, nil
}

func (r *RedisBackend) Set(ctx context.Context, key, value string, expiration time.Duration) error {
    return r.client.Set(ctx, key, value, expiration).Err()
}

func (r *RedisBackend) Del(ctx context.Context, key string) error {
    return r.client.Del(ctx, key).Err()
}

func (r *RedisBackend) TTL(ctx context.Context, key string) (time.Duration, error) {
    return r.client.TTL(ctx, key).Result()
}

func (r *RedisBackend) Publish(ctx context.Context, channel, message string) error {
    return r.client.Publish(ctx, channel, message).Err()
}

func (r *RedisBackend) Subscribe(ctx context.Context, channel string) (<-chan icache.Message, error) {
    pubsub := r.client.Subscribe(ctx, channel)
    redisCh := pubsub.Channel()
    
    // Convert redis.Message to icache.Message
    ch := make(chan icache.Message)
    go func() {
        defer close(ch)
        for msg := range redisCh {
            ch <- icache.Message{
                Channel: msg.Channel,
                Payload: msg.Payload,
            }
        }
    }()
    
    return ch, nil
}

func (r *RedisBackend) Close() error {
    return r.client.Close()
}

// Usage
func main() {
    backend := NewRedisBackend("localhost:6379", "", 0)
    cache := icache.New(icache.Config{
        Backend:        backend,
        LocalCacheSize: 10000,
        InstanceID:     "instance-1",
    })
    defer cache.Close()
    
    // Use the cache...
}

Example: Simple Map Backend (for testing)

package main

import (
    "context"
    "sync"
    "time"
    
    "github.com/bulutaysarac/icache"
)

type MapBackend struct {
    data sync.Map
}

func NewMapBackend() *MapBackend {
    return &MapBackend{}
}

func (m *MapBackend) Get(ctx context.Context, key string) (string, bool, error) {
    val, ok := m.data.Load(key)
    if !ok {
        return "", false, nil
    }
    return val.(string), true, nil
}

func (m *MapBackend) Set(ctx context.Context, key, value string, expiration time.Duration) error {
    m.data.Store(key, value)
    // Note: This simple implementation doesn't handle expiration
    // In a real implementation, you'd need to handle TTL
    return nil
}

func (m *MapBackend) Del(ctx context.Context, key string) error {
    m.data.Delete(key)
    return nil
}

func (m *MapBackend) TTL(ctx context.Context, key string) (time.Duration, error) {
    _, ok := m.data.Load(key)
    if !ok {
        return -2, nil
    }
    return -1, nil // No expiration
}

func (m *MapBackend) Publish(ctx context.Context, channel, message string) error {
    return nil // No-op for simple backend
}

func (m *MapBackend) Subscribe(ctx context.Context, channel string) (<-chan icache.Message, error) {
    return nil, nil // No pub/sub support
}

func (m *MapBackend) Close() error {
    return nil
}

Architecture

Layer 1 (L1) - Local Cache

• In-memory cache using gcache with ARC (Adaptive Replacement Cache) policy
• Extremely fast reads (nanoseconds)
• Automatic eviction when capacity is reached
• Synchronized across instances via backend pub/sub (if supported)

Layer 2 (L2) - Backend Cache

• Optional distributed cache for persistence
• Shared across multiple instances
• TTL support
• Pub/sub for cache invalidation (if backend supports it)

Cache Flow

1. Read Operation:

   • Check L1 cache → if hit, return immediately
   • Check L2 (backend) → if hit, populate L1 and return
   • Execute fetcher function (if provided) → populate both layers

2. Write Operation:

   • Write to both L1 and L2
   • Publish invalidation message to other instances (if backend supports pub/sub)
   • Other instances remove the key from their L1 cache

API Reference

Creating a Cache Instance

// Without backend (local only)
config := icache.Config{
    LocalCacheSize: 10000,
    InstanceID:     "unique-instance-id",
}

// With backend
config := icache.Config{
    Backend:        myBackend, // implements icache.Backend
    LocalCacheSize: 10000,
    InstanceID:     "unique-instance-id",
}

cache := icache.New(config)
defer cache.Close()

Get

Retrieve a value from cache with optional fetcher function.

// Simple get
value, exists, err := icache.Get[string](cache, ctx, "key")

// Get with fetcher (no expiration)
value, exists, err := icache.Get(cache, ctx, "key", func() (string, error) {
    return fetchFromDB()
})

// Get with fetcher and expiration
value, exists, err := icache.Get(cache, ctx, "key", func() (string, time.Duration, error) {
    return fetchFromDB(), 5*time.Minute, nil
})

Set

Store a value in cache with optional expiration.

// Set without expiration
err := cache.Set(ctx, "key", "value")

// Set with expiration
err := cache.Set(ctx, "key", "value", 5*time.Minute)

LocalSet

Store a value only in local cache (L1) without backend synchronization.

err := cache.LocalSet(ctx, "key", "value", 5*time.Minute)

Delete

Remove a value from cache.

err := cache.Del(ctx, "key")

GetLocalKeys

Get all keys from local cache (useful for debugging).

keys := cache.GetLocalKeys(ctx)

Configuration

Best Practices

1. Instance ID: Use a unique instance ID for each application instance to properly handle cache invalidation
2. Cache Size: Set LocalCacheSize based on your memory constraints
3. TTL: Always set appropriate TTL for cache entries to prevent stale data
4. Error Handling: Always check errors returned by cache operations
5. Context: Use context with timeout for better control
6. Fetcher Functions: Use fetcher functions to automatically populate cache on miss
7. Backend Selection: Choose a backend that fits your needs - simple map for testing, Redis for production, etc.

Examples

Check out the /examples directory for more detailed examples:

• Basic usage
• Custom backend implementations
• Multi-instance setup with cache invalidation
• Fetcher functions

Performance

• L1 Cache: ~100ns per operation (in-memory)
• L2 Cache: Depends on backend (typically ~1ms for Redis)
• Cache Hit Ratio: Typically 90%+ with proper TTL configuration

Use Cases

• Web Applications: Cache API responses, database queries, session data
• Microservices: Share cache across service instances
• Data Processing: Cache expensive computations
• Rate Limiting: Store rate limit counters
• Configuration: Cache configuration values

License

MIT License

Contributing

Contributions are welcome! Please open an issue or submit a pull request.