Skip to content

Scale out

Gerasimos (Makis) Maropoulos edited this page May 28, 2026 · 3 revisions

When to use this page

  • You're running more than one neffos server instance behind a load balancer.
  • You need a broadcast from instance A to reach a client connected to instance B.
  • You need Server.Ask to find a responder on a different instance.

The problem scale-out solves

A single neffos server keeps every connection in memory. When you scale horizontally — two or more processes behind a load balancer — each process only sees its own connections. A Broadcast from instance A reaches only A's clients; the client that just connected to instance B never gets the message.

Scale-out solves this by relaying every broadcast and every ask through a shared message broker. The broker fans out to every neffos instance, each of which delivers to its local clients.

Architecture

flowchart LR
    subgraph Broker
        R[(Redis or NATS)]
    end
    subgraph Instance A
        SA[neffos Server]
        CA1[Client A1]
        CA2[Client A2]
    end
    subgraph Instance B
        SB[neffos Server]
        CB1[Client B1]
        CB2[Client B2]
    end
    CA1 <--> SA
    CA2 <--> SA
    CB1 <--> SB
    CB2 <--> SB
    SA <-->|publish/subscribe| R
    SB <-->|publish/subscribe| R
Loading

What you give up

  • Order is per-namespace, not global. Two messages broadcast in close succession may arrive at instance B in either order.
  • Sticky sessions are still recommended for reconnection. A client that reconnects to a different instance keeps working, but loses its previous Conn.ID — broadcasts targeted by ID won't find them under the old ID.
  • An extra dependency. Redis or NATS must be up for cross-instance broadcasts to work. Each instance still serves its local clients if the broker is down, but cross-instance traffic stops.
  • A latency hop. Every cross-instance broadcast and ask pays one broker round-trip.

What you keep

  • At-most-once. Same as the single-instance default.
  • Server.Ask works across instances. The reply finds you wherever you're running.
  • All event handlers and namespaces are unchanged. Scale-out is transparent to your handlers.

Choosing a backend

Property Redis NATS
Operational familiarity Most teams already run Redis. Smaller community, simpler ops.
Memory footprint Heavier (general-purpose store). Lightweight (single binary, <20MB).
Persistence Optional (RDB/AOF), not used by neffos. Optional (JetStream), not used by neffos.
Pub/sub model Pattern subscriptions (PSUBSCRIBE). Subject subscriptions (*/> wildcards).
Auth/TLS Built-in (requirepass, TLS). Built-in (NKey, JWT, mTLS).
Clustering Yes (cluster mode supported). Yes (gossip, no extra config needed).

Start with Redis if your stack already has it. Switch to NATS if you want a smaller operational footprint or you're already using NATS for other services. Throughput is broker-bound — benchmark with your message size before committing.

Three-line setup

  1. Import github.com/kataras/neffos/stackexchange/redis (or .../nats).
  2. Construct the StackExchange.
  3. Pass it to server.UseStackExchange.

Redis:

import "github.com/kataras/neffos/stackexchange/redis"

exc, err := redis.NewStackExchange(redis.Config{Addr: "127.0.0.1:6379"}, "neffos-app")
if err != nil { log.Fatal(err) }

server := neffos.New(gorilla.DefaultUpgrader, handler)
if err := server.UseStackExchange(exc); err != nil { log.Fatal(err) }

NATS:

import "github.com/kataras/neffos/stackexchange/nats"

exc, err := nats.NewStackExchange("nats://localhost:4222")
if err != nil { log.Fatal(err) }

server := neffos.New(gorilla.DefaultUpgrader, handler)
if err := server.UseStackExchange(exc); err != nil { log.Fatal(err) }

How it works under the hood

  1. On OnConnect, each connection gets a dedicated subscriber registered with the backend on a self-channel keyed by Conn.ID (for To:-targeted messages).
  2. On OnNamespaceConnected, the subscriber adds a per-namespace channel/subject.
  3. Server.Broadcast calls the backend's Publish, which fans out to all interested subscribers (both local and on other instances).
  4. Server.Ask calls the backend's Ask, which publishes the request and waits on a unique reply channel.
  5. On OnDisconnect, the subscriber tears down.

If Close is supported on the StackExchange (both Redis and NATS implementations do), Server.Close() calls it to release the background goroutine and broker connection.

Multiple StackExchanges

You can register more than one — for example, to bridge two brokers temporarily during a migration:

server.UseStackExchange(redisExc)
server.UseStackExchange(natsExc) // wraps redisExc; both receive every publish

Publish returns false if either underlying exchange fails. Ask falls through to the next exchange if the first errors.

Next steps

See also

Clone this wiki locally