This blog is divided into 7 parts:

1. Why This Project
2. Understanding BitTorrent
3. GoTorrent Architecture
4. System Workflow: How GoTorrent Operates
5. The P2P Engine (Under the Hood)
6. Key Features and Big Challenges
7. Conclusion and Learnings

1. Why This Project

Before diving into the technical details, here is why I chose this project.

When I was around 12–13 years old, I used to download games and movies using applications like uTorrent. Those early experiences made me curious about how files could be shared among many users without relying on a single central server.

That curiosity eventually led me to build GoTorrent as my final-year project, both to understand the BitTorrent protocol deeply and to challenge myself with a real-world networking system.

2. Understanding BitTorrent

Before explaining GoTorrent, it helps to understand how BitTorrent works.

BitTorrent is a peer-to-peer (P2P) protocol where each user (peer) can both download and upload. Instead of downloading an entire file from one server, the file is split into pieces and shared across peers.

2.1 Torrent File and Tracker

A .torrent file contains metadata such as:

• File name and size
• Piece length and SHA-1 hashes
• Tracker URLs

When a client loads this file, it contacts a tracker to get a list of peers sharing that torrent.

2.2 Peer-to-Peer Communication

After peers are discovered, they connect directly over TCP and exchange protocol messages such as:

• handshake for connection setup
• interested / not interested
• choke / unchoke
• request, piece, have

2.3 Piece Verification

Each piece is verified using SHA-1 hashes from torrent metadata. If a hash check fails, that piece is downloaded again.

2.4 Seeding and Swarming

Once a peer completes the file, it becomes a seeder and uploads pieces to other peers. This cooperative model is what makes BitTorrent scalable.

In short, BitTorrent is a distributed exchange of pieces among peers, not a one-way client-server download.

3. GoTorrent Architecture

GoTorrent follows a daemon-based architecture built in Go.

3.1 Daemon (gotorrentd): The Engine

The daemon handles core responsibilities:

• pkg/daemon: orchestration, lifecycle control, and coordination
• pkg/torrent: .torrent parsing and tracker communication (HTTP/UDP)
• DHT integration in daemon for additional peer discovery
• pkg/p2p, pkg/client, pkg/message: peer protocol engine and piece transfer
• pkg/daemon/store.go + SQLite: persistence of state, progress, and bitfield

Note: Magnet link support is planned, but current implementation primarily handles .torrent input.

3.2 Client Layer

Users interact with the daemon through:

• CLI client via RPC
• REST endpoints for programmatic control

3.3 Why Go

Go was a strong fit because of:

• Goroutines and channels for concurrency
• Reliable networking primitives for TCP/UDP/HTTP
• Good runtime performance for high-throughput transfers

This architecture keeps core download logic isolated and long-running, while control surfaces remain lightweight.

4. System Workflow: How GoTorrent Operates

4.1 Parsing The .torrent file via bencode-go parser

type bencodeInfo struct {
 Pieces      string `bencode:"pieces"`
 PieceLength int    `bencode:"piece length"`
 Length      int    `bencode:"length"`
 Name        string `bencode:"name"`
}

type bencodeTorrent struct {
 Announce     string      `bencode:"announce"`
 AnnounceList [][]string  `bencode:"announce-list"`
 Info         bencodeInfo `bencode:"info"`
}

func (bto *bencodeTorrent) toTorrentFile() (TorrentFile, error) {}

This Function parses the data and splits the hashes and fill up the trackers array with the trackers.

4.2 Contacting the Tracker and Peer Discovery

After parsing, we get a TorrentFile model:

type TorrentFile struct {
    Announce     string
    InfoHash     [20]byte
    PieceHashes  [][20]byte
    AnnounceList []string
    PieceLength  int
    Length       int
    Name         string
}

The tracker request starts in pkg/torrent/tracker.go. GoTorrent builds a standard tracker URL with query params like:

• info_hash
• peer_id
• port
• uploaded, downloaded, left
• compact=1

Core builder:

func buildTrackerURL(tf *TorrentFile, peerID [20]byte, port uint16) (string, error)

The main entry point is:

func RequestPeers(tf *TorrentFile, peerID [20]byte) ([]peers.Peer, error)

This function checks tracker scheme and routes to:

• HTTP/HTTPS tracker flow: requestPeersHTTP(...)
• UDP tracker flow: RequestPeersUDPWithFallback(...)

HTTP tracker response model

type bencodeTrackerResp struct {
    Interval       int    `bencode:"interval"`
    Peers          string `bencode:"peers"`
    FailureReason  string `bencode:"failure reason,omitempty"`
    WarningMessage string `bencode:"warning message,omitempty"`
    Complete       int    `bencode:"complete,omitempty"`
    Incomplete     int    `bencode:"incomplete,omitempty"`
}

The response is decoded and compact peers are converted using:

func UnmarshalPeers(peersBin []byte) ([]Peer, error)

where each peer is parsed from 6 bytes (4 for IP + 2 for port).

UDP tracker flow and resilience

pkg/torrent/udp.go implements the UDP tracker protocol in two phases:

1. Connect (actionConnect)
2. Announce (actionAnnounce)

Key functions:

func requestPeersUDP(u *url.URL, tf *TorrentFile, peerID [20]byte) ([]peers.Peer, error)
func buildConnectRequest(transactionID uint32) []byte
func parseConnectResponse(data []byte, expectedTransactionID uint32) (uint64, error)
func buildAnnounceRequest(connectionID uint64, transactionID uint32, tf *TorrentFile, peerID [20]byte, port uint16) []byte
func parseAnnounceResponse(data []byte, expectedTransactionID uint32) ([]peers.Peer, error)

To improve reliability, sendAndReceive(...) uses retry + exponential backoff, and RequestPeersUDPWithFallback(...) tries known fallback UDP trackers if the primary one fails.

4.3 Hybrid Peer Discovery Inside the Daemon

Tracker contact alone is not enough for all torrents, so GoTorrent adds DHT discovery in the daemon (pkg/daemon/server.go).

In Server.AddTorrent(...), peer discovery happens in this order:

1. Iterate tracker URLs from AnnounceList
2. Query trackers in order via torrent.RequestPeers(...) until one returns peers
3. If DHT is enabled, query DHT using s.dht.Announce(tf.InfoHash, 0, true)
4. Merge and deduplicate peers using IP:Port keys

This gives a hybrid discovery strategy:

• Trackers provide fast bootstrap peers
• DHT improves availability when trackers are down or limited

4.4 End-to-End Workflow (Execution Pipeline)

Once peers are found, the daemon creates the P2P job:

torrentJob := &p2p.Torrent{
    Peers:        initialPeers,
    PeerID:       ourPeerID,
    InfoHash:     tf.InfoHash,
    PieceHashes:  tf.PieceHashes,
    PieceLength:  tf.PieceLength,
    Length:       tf.Length,
    Name:         tf.Name,
    Announce:     tf.Announce,
    DownloadPath: s.downloadPath,
}

Then the workflow is:

1. Persist torrent metadata in SQLite (store.AddTorrent)
2. Add job to in-memory map (s.torrents)
3. Start async download worker (go torrentJob.DownloadTheFile(s.store))
4. Periodically update progress (store.UpdateTorrentProgress)
5. Serve status through RPC + REST APIs

So the full data path is:

.torrent -> metadata parse -> peer discovery (Tracker + DHT) -> worker swarm -> piece validation -> disk write -> DB update -> status APIs

5. The P2P Engine (Under the Hood)

This is the core download logic in pkg/p2p/p2p.go and related protocol packages.

5.1 Torrent Runtime Model

type Torrent struct {
    Peers           []peers.Peer
    PeerID          [20]byte
    InfoHash        [20]byte
    PieceHashes     [][20]byte
    PieceLength     int
    Length          int
    Name            string
    Announce        string
    BytesDownloaded int64
    Status          string
    ActivePeers     int
    State           uint8
    OurBitfield     bitfield.Bitfield
    DownloadPath    string
}

This struct tracks both static metadata (hashes, piece size, total length) and mutable runtime state (status, speed, active peers, bitfield).

5.2 Peer Connection, Handshake, and Bitfield

Each worker connects through client.New(...):

• TCP connect to peer IP:Port
• Perform BitTorrent handshake
• Receive initial bitfield

Handshake package (pkg/handshake/handshake.go) defines:

type Handshake struct {
    Pstr     string
    InfoHash [20]byte
    PeerID   [20]byte
}

completeHandshake(...) verifies returned InfoHash matches the target torrent, preventing cross-torrent contamination.

5.3 Piece Scheduling and Parallel Download

The engine uses a producer-consumer model:

• workQueue contains pending pieces (pieceWork)
• Each peer goroutine runs startDownloadWorker(...)
• Successful pieces are sent through results

Per-piece download uses pipelining with backlog:

const MaxBlockSize = 16384
const MaxBacklog = 5

attemptDownloadPiece(...) keeps requesting blocks while unchoked and reads protocol messages (Unchoke, Have, Piece, etc.) via readMessage().

5.4 Integrity and Safe Write Path

Before writing piece data, GoTorrent validates SHA-1:

func checkIntegrity(pw *pieceWork, buf []byte) error

If hash check fails, piece is re-queued. If it passes, piece is written at exact file offset with WriteAt(...), and local bitfield is updated.

5.5 Pause, Resume, Stop, and Speed Tracking

Torrent control uses a state channel:

func (t *Torrent) Pause()
func (t *Torrent) Resume()
func (t *Torrent) Stop()

Download speed is calculated every 2 seconds from byte deltas. Progress is persisted periodically so restarts can continue from saved bitfield state.

6. Key Features and Big Challenges

6.1 Key Features Implemented (Go Backend)

1. Hybrid peer discovery
2. HTTP and UDP tracker support
3. Multi-peer concurrent piece download using goroutines
4. Piece-level SHA-1 integrity validation
5. Persistent progress with SQLite (including bitfield)
6. Pause/resume/remove lifecycle management
7. Daemon architecture with RPC and REST interfaces
8. Real-time status broadcasting pipeline

6.2 Big Technical Challenges

1. Unreliable tracker ecosystem Many trackers fail, timeout, or return empty peer lists. I solved this using multi-tracker attempts, UDP retries, and fallback tracker logic.
2. Peer instability and choking behavior Not all peers cooperate. Some choke, disconnect, or have partial data. I handled this using per-worker timeouts, piece re-queueing, and concurrent workers.
3. Correct piece boundary math Last piece is often smaller than normal piece length. I added calculateBoundsForPiece(...) and calculatePieceSize(...) to avoid overflow and corruption.
4. Data integrity under concurrency Concurrent downloads can cause races if state updates are unsafe. I used mutexes for shared state (BytesDownloaded, bitfield, status) and only mark piece complete after successful hash + disk write.
5. Crash/restart resilience Download clients should not lose progress after restart. I persisted status + bitfield in SQLite and load managed torrents from DB during daemon startup.
6. Lifecycle transitions are tricky Pausing, resuming, stopping, and restarting workers can create invalid states. I made explicit state transitions (StatePaused, StateDownloading, StateSeeding, etc.) and guarded resume behavior based on current status.

7. Conclusion and Learnings

Building GoTorrent as my final year project taught me far more than just “how torrents work”. It gave me hands-on experience with distributed systems, binary protocols, concurrency, fault tolerance, and long-running backend service design in Go.

The biggest learning was this: A BitTorrent client is not just about downloading files. It is a coordination problem across unreliable networks, inconsistent peers, and asynchronous state updates.

From parsing bencoded metadata to tracker communication, DHT lookup, handshake negotiation, piece scheduling, integrity verification, and persistence, every stage has failure modes that must be handled explicitly.

Go’s goroutines, channels, and standard library made it practical to build this as a real production-style system, not just a classroom prototype.

You can explore the complete source code and implementation details here: https://github.com/Yashh56/GoTorrent

If you found the project interesting or helpful, feel free to leave a star on the repository ⭐

“In multiplayer online games, matchmaking is the core system that pairs players together in real-time battles, ranked matches, or casual games. Without an efficient matchmaking engine, players might experience long wait times, unfair matchups, and poor game experiences.”

This blog walks through the architecture and implementation of a simple but scalable matchmaking engine, covering its major services and logic.

What This Blog Will Teach You

• Matchmaking logic based on MMR, ping, and region.
• Real-time WebSocket notifications to clients when a match is found.
• Using Redis Queue for managing players in matchmaking.
• Implementing the Pub/Sub pattern for scalable async messaging.
• Clean and modular architecture using internal/ and pkg/ folders.

What This Blog Will Not Teach You

• The Game Orchestrator Service that dynamically creates Kubernetes pods.
• Generating the WebSocket URL for players to connect to the game pod.
• Automatic cleanup of game pods after a timeout (e.g., 5 minutes).
• Deploying the full system on Kubernetes with auto-scaling.
• Advanced matchmaking techniques like ML-based MMR adjustment.

These topics are more advanced and would make the blog too lengthy and complex for a single article. Since the complete source code is available on GitHub, you can explore the orchestrator and deployment details directly from there, or I’ll cover them in a follow-up blog.

Prerequisites

Before we dive into building the matchmaking engine, make sure you’re familiar with the following:

• Go: A working knowledge of Go (Golang) and how to structure a basic Go project.
• Redis: Understanding of Redis, as we’ll use it for queuing players and publishing match events.
• Kubernetes (Basic): Some basic understanding of Kubernetes concepts (although this blog won’t cover Kubernetes in detail, it’s part of the larger system architecture).
• HTTP & WebSockets: Familiarity with how HTTP APIs and WebSocket connections work.

Low-Level System Design :

Before Jumping into the code. Let’s look at how our system’s components interact.

Core Components

1. Player Service (🟧 Player Related):

• Manages player data and matchmaking requests.
• Places matchmaking requests into a Redis Queue for asynchronous processing.
• Acts as the primary entry point for the matchmaking process.

2. Redis Queue

• Queue system that temporarily stores matchmaking requests.
• Enables asynchronous decoupling between the Player Service and the Matchmaking Server.

3. Matchmaking Server (🟪 Core-Matchmaking Logic)

• Consumes matchmaking requests from the Redis Queue.
• Implements the core matchmaking algorithm (e.g., MMR-based pairing, latency checks).
• Publishes matchmaking results to a Pub/Sub system.

4. Pub/Sub System

• Decouples communication between the Matchmaking Server, Game Orchestrator, and Notification Services.
• Facilitates asynchronous event distribution, such as when a match is found or a game server is created.

5. Notification Services (🟦 Notification/WebSockets)

• Listens to Pub/Sub for real-time game/match events.
• Sends real-time updates to connected clients via WebSockets, such as match found, game starting, or server errors.

⚠️ Additional Components (Out of Scope for This Blog):

6. Game Orchestrator Service (🟫 Infrastructure Controller)

• Listens to matchmaking events via Pub/Sub.
• Responsible for creating or managing game server instances, usually on-demand.
• Likely interacts with Kubernetes (K8s) for managing the game server lifecycle.

6. Kubernetes API Server (K8s Control Plane)

• Exposes the Kubernetes API for managing infrastructure.
• Allows the Game Orchestrator to spin up or terminate game server pods on-demand.

Important Note :

The Game Orchestrator and Kubernetes integration are part of the larger system architecture but are outside the scope of this blog. This article focuses on the matchmaking engine itself. These infrastructure components will be covered in a future blog post.

Setup

Since we are going to build this matchmaking engine in Go, we assume you have some experience with Go and have it installed and configured.

We’ll also install a few essential Go packages using go get:

go get github.com/gorilla/mux

Gorilla/mux is a popular router for writing HTTP handlers in Go.

go get github.com/gin-gonic/gin

Gin is a fast and lightweight HTTP web framework for Go. It features a clean API and much better performance compared to older frameworks like Martini.

go get github.com/joho/godotenv

godotenv is used for loading environment variables from .env files.

go get github.com/redis/go-redis/v9

go-redis is the official Redis client library for Go, used to interact with Redis.

Initializing the Redis Client

We’ll create a utility package to initialize and access the Redis client throughout the project.

//utils/redis.go

package utils

import (
 "fmt"

 "github.com/redis/go-redis/v9"
)

var client *redis.Client

func SetClient(redisClient *redis.Client) {
 fmt.Println("Setting Redis client")

 client = redisClient
 fmt.Println("Redis client has been initialized")
}

func GetRedisClient() *redis.Client {
 if client == nil {
  fmt.Println("WARNING: Redis client is nil")
 }
 return client
}

This utility allows us to initialize the Redis connection once and access it from anywhere in the codebase.

Player Service

Let’s start by creating the Player Service, which is responsible for handling player matchmaking requests.

This service exposes a POST API endpoint where players submit their:

• MMR (Matchmaking Rating)
• Region
• Ping
• and other game-related preferences.

Once the player submits this data, the service places the matchmaking request into a Redis queue. This data will later be consumed by the Matchmaking Service, which uses it to group players into fair matches based on similar skill levels and network conditions.

Data Model

Let’s define a struct called Player that contains various fields. Don’t worry, we’ll explain all of these in a minute.

// internal/player/player.go

package player

type Player struct {
 Player_id string `json:"player_id"`
 MMR       int    `json:"mmr"`
 Region    string `json:"region"`
 Ping      int    `json:"ping"`
 JoinedAt  int64  `json:"joined_at"`
}

Player_id: A unique identifier for each player. This helps track them across sessions.

MMR : The Matchmaking Rating (MMR) that represents the player’s skill level. The matchmaking system uses this to form balanced teams.

Region: The geographic region where the player is located (e.g., Asia, Europe). Players from the same region are grouped to minimize latency.

Ping: The player’s current network latency in milliseconds. This is used to ensure a fair gameplay experience.

Joined_at : The timestamp when the player joined the matchmaking queue. This can be used for a timeout or prioritizing players waiting longer.

Imports

Here are the imports we’ll need, along with our package declaration. Let’s write these to handler.go

// internal/player/handler.go 
package player

import (
 "context"
 "encoding/json"
 "fmt"
 "net/http"

 "github.com/Yashh56/matchmakingEngine/utils"
 "github.com/gin-gonic/gin"
 "github.com/redis/go-redis/v9"
)

Now, Let’s Write the Join_queue function which handles the Player's Input and Inserts the data in the Redis Queue based on their region.

var ct = context.Background()

func Join_queue(ctx *gin.Context) {
 var player Player

 // Bind JSON input
 if err := ctx.ShouldBindJSON(&player); err != nil {
  ctx.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
  return
 }

 // Get Redis client
 redisClient := utils.GetRedisClient()
 if redisClient == nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Redis not initialized"})
  return
 }

 // Serialize player struct to JSON string
 playerBytes, err := json.Marshal(player)
 if err != nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to serialize player"})
  return
 }
 playerString := string(playerBytes)

 // 1. Save player in Redis as key-value pair for future retrieval
 err = redisClient.Set(ct, player.Player_id, playerString, 0).Err()
 if err != nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to store player in Redis"})
  return
 }

 // 2. Add player to matchmaking queue (sorted set)
 queueKey := fmt.Sprintf("queue:%s:%s", "solo", player.Region)

 err = redisClient.ZAdd(ct, queueKey, redis.Z{
  Score:  float64(player.JoinedAt),
  Member: playerString,
 }).Err()

 if err != nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to add to queue"})
  return
 }

 // 3. Retrieve stored player (optional check)
 val, err := redisClient.Get(ct, player.Player_id).Result()
 if err != nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to retrieve player from Redis"})
  return
 }

 // Deserialize JSON back to struct
 var retrievedPlayer Player
 err = json.Unmarshal([]byte(val), &retrievedPlayer)
 if err != nil {
  ctx.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to decode player from Redis"})
  return
 }

 // Success response
 ctx.JSON(http.StatusOK, gin.H{
  "status": "Waiting in Queue",
  "player": retrievedPlayer,
 })
}

Matchmaking Service

Now, let’s move on to the Matchmaking Service, which is responsible for pairing players into matches.

This service consumes player data from the Redis queue and applies matchmaking logic based on factors like MMR (Matchmaking Rating) and region.
Its goal is to group players into fair and balanced matches while keeping the matchmaking process fast and scalable.

Imports

import (
 "context"
 "encoding/json"
 "fmt"
 "math"
 "time"

 "github.com/Yashh56/matchmakingEngine/internal/player"
 "github.com/google/uuid"
 "github.com/redis/go-redis/v9"
)

Data Model

Let’s define a struct called Match that contains various fields. Don’t worry, we’ll explain all of these in a minute.

// internal/matchmaking/elo.go
type Match struct {
 Id      string          `json:"id"`
 Players []player.Player `json:"players"`
 Region  string          `json:"region"`
}

Id : A unique identifier Assigned to each match (match Id).

Players : An Array of Player who has a Match based on their Data.

Region : The region in which the match will take place (used to reduce latency).

Matchmaking Logic

Matching Players: Can They Be Matched?

Before we run the matchmaking worker, we need a function to check whether two players can be fairly matched. This check considers their region, MMR gap, and ping.

// internal/matchmaking/elo.go

func CanMatch(p1, p2 player.Player) bool {
// Skip if both players have different regions
 if p1.Region != p2.Region {
  fmt.Printf("[Skip] Region mismatch: %s vs %s\n", p1.Region, p2.Region)
  return false
 }

 mmrGap := math.Abs(float64(p1.MMR - p2.MMR))
 waitTime := float64(time.Now().Unix() - p1.JoinedAt)
 maxGap := 100 + (waitTime/30)*10
// skip if there is too much Gap between Players MMR
 if mmrGap > maxGap {
  fmt.Printf("[Skip] MMR gap too high (%f > %f) between %s and %s\n", mmrGap, maxGap, p1.Player_id, p2.Player_id)
  return false
 }
// Skip if there are high ping of any player
 if p1.Ping > 100 || p2.Ping > 100 {
  fmt.Printf("[Skip] Ping too high: %s (%d ms), %s (%d ms)\n", p1.Player_id, p1.Ping, p2.Player_id, p2.Ping)
  return false
 }

 fmt.Printf("[Match ✅] %s vs %s | MMR Gap: %.0f | Region: %s\n", p1.Player_id, p2.Player_id, mmrGap, p1.Region)
 return true
}

The CanMatch function returns true if the players meet the criteria for matchmaking, otherwise false.

Once two players can be matched, we publish the match to a Redis Pub/Sub topic so other services (like Notification Service or Game Orchestrator) can react to it.

// internal/matchmaking/elo.go
func FormMatch(p1, p2 player.Player, redisClient *redis.Client) {
 matchId := uuid.New().String()
 ctx := context.Background()

 match := Match{
  Id:      matchId,
  Players: []player.Player{p1, p2},
  Region:  p1.Region,
 }

 jsonData, err := json.Marshal(match)
 if err != nil {
  fmt.Printf("[❌ ERROR] Failed to marshal match: %v\n", err)
  return
 }
 redisClient.Set(ctx, "match:"+matchId, jsonData, 0)

 redisClient.Publish(ctx, "matchmaking:events", jsonData)
 redisClient.SAdd(ctx, "match_players:"+matchId, p1.Player_id, p2.Player_id)

 err = redisClient.Publish(ctx, "matchmaking:events", jsonData).Err()
 if err != nil {
  fmt.Printf("[❌ ERROR] Failed to publish match: %v\n", err)
  return
 }

 fmt.Printf("[📣 MATCH PUBLISHED] Match ID: %s | Players: [%s, %s] | Region: %s\n",
  matchId, p1.Player_id, p2.Player_id, p1.Region)
}

This publishes the match data to a Redis topic called matchmaking:events, so downstream services can process it.

We also need a function that removes the Player from the queue if they get a Match based on it Region, Ping and MMR.

// internal/matchmaking/matchmaking.go

func RemoveFromQueue(ctx context.Context, redisClient *redis.Client, p1, p2 playerService.Player) {
 queueKey := fmt.Sprintf("queue:%s:%s", p1.GameMode, p1.Region)

 p1Json, err1 := json.Marshal(p1)
 P2Json, err2 := json.Marshal(p2)

 if err1 != nil || err2 != nil {
  fmt.Println("[❌ ERROR] Failed to marshal players during ZREM")
  return
 }
 removed, err := redisClient.ZRem(ctx, queueKey, string(p1Json), string(P2Json)).Result()

 err = redisClient.Del(ctx, p1.Player_id).Err()
 err = redisClient.Del(ctx, p2.Player_id).Err()

 if err != nil {
  fmt.Printf("[❌ ERROR] Failed to remove players from queue: %v\n", err)
  return
 }

 fmt.Printf("[🗑️ Removed] %d player(s) removed from queue %s\n", removed, queueKey)

}

RemoveFromQueue function removes the players from Queue based on their matchmaking.

Now, let’s implement the RunMatchmaking function. This is the main worker function that continuously monitors the player queue and runs the matchmaking algorithm.
It works in a loop, constantly checking for players and forming fair matches based on their MMR, region, and ping.

// internal/matchmaking/matchmaking.go

func RunMatchmaking(ctx context.Context, redisClient redis.Client) {
    // Define supported regions and game modes
    regions := []string{"asia", "europe", "na", "sa"}
    modes := []string{"solo"}

    for {
        // Iterate over each mode and region combination
        for _, mode := range modes {
            for _, region := range regions {

                // Construct the Redis sorted set key for the queue
                queueKey := fmt.Sprintf("queue:%s:%s", mode, region)

                // Retrieve all players currently waiting in the queue
                matchCandidates, err := redisClient.ZRange(ctx, queueKey, 0, -1).Result()
                if err != nil {
                    log.Printf("❌ Error reading from queue %s: %v", queueKey, err)
                    continue
                }

                // If no players are waiting, skip to the next queue
                if len(matchCandidates) == 0 {
                    log.Printf("ℹ️ No candidates in queue %s", queueKey)
                    continue
                }

                // Create a set to track players who have already been matched in this iteration
                matchedSet := make(map[string]bool)

                // Loop through all players to find potential matches
                for i := 0; i < len(matchCandidates); i++ {
                    var player playerService.Player
                    // Deserialize the player data and skip if failed or already matched
                    if err := json.Unmarshal([]byte(matchCandidates[i]), &player); err != nil || matchedSet[player.Player_id] {
                        continue
                    }

                    // Compare the current player with all subsequent players in the queue
                    for j := i + 1; j < len(matchCandidates); j++ {
                        var candidate playerService.Player
                        // Deserialize candidate and skip if failed or already matched
                        if err := json.Unmarshal([]byte(matchCandidates[j]), &candidate); err != nil || matchedSet[candidate.Player_id] {
                            continue
                        }

                        // If the player and candidate can be matched based on defined criteria
                        if CanMatch(player, candidate) {
                            log.Printf("✅ Match found: %s vs %s in %s (%s)", player.Player_id, candidate.Player_id, region, mode)

                            // Mark both players as matched to avoid duplicate matches
                            matchedSet[player.Player_id] = true
                            matchedSet[candidate.Player_id] = true

                            // Form a match (create match data, store it somewhere, notify players, etc.)
                            FormMatch(player, candidate, &redisClient)

                            // Remove both players from the matchmaking queue
                            RemoveFromQueue(ctx, &redisClient, player, candidate)

                            // Break inner loop since the player has been matched
                            break
                        }
                    }
                }

            }
        }

        // Sleep for a short interval before checking the queues again
        time.Sleep(3 * time.Second)
    }
}

Notification Service

The primary role of the Notification Service is to notify players about important events from the matchmaking engine, such as when a match has been successfully found.

This service uses WebSocket connections to communicate with the players in real time. Each player establishes a WebSocket connection using their unique player_id, allowing the system to push notifications directly to the correct player.

In the background, the Notification Service subscribes to a Redis Pub/Sub topic called "game:allocated", which publishes player match events from the Matchmaking Service. Whenever a new match is created, the Notification Service receives the player data from this topic and sends a WebSocket message to notify the players involved.

So Let’s Start with ListenForMatches function which subscribes game:allocated topic and gets the complete match details based on the player's info.

//internal/ws/pubSub.go

func ListenForMatches(ctx context.Context, rdb *redis.Client, manager *clients.Manager) {
 sub := rdb.Subscribe(ctx, "game:allocated")
 ch := sub.Channel()

 for msg := range ch {
  var payload gameorchestrator.GameSession

  if err := json.Unmarshal([]byte(msg.Payload), &payload); err != nil {
   log.Println("❌ Failed to parse session payload:", err)
   continue
  }

  matchId := payload.MatchId
  address := fmt.Sprintf("%s:%d", payload.Address, payload.Port)

  // 🧠 Get full match data
  matchJSON, err := rdb.Get(ctx, "match:"+matchId).Result()
  if err != nil {
   log.Printf("❌ Could not fetch match data for match %s: %v", matchId, err)
   continue
  }

  var matchData map[string]interface{}
  if err := json.Unmarshal([]byte(matchJSON), &matchData); err != nil {
   log.Printf("❌ Failed to unmarshal match data for match %s: %v", matchId, err)
   continue
  }

  // 🧩 Enrich match data with address and message
  matchData["address"] = address
  matchData["message"] = fmt.Sprintf("🎯 You’ve been matched!\n✅ Pod for match %s created at %s", matchId, address)
  matchData["type"] = "match_found"

  // 🎯 Get matched player IDs
  playerIds, err := rdb.SMembers(ctx, "match_players:"+matchId).Result()
  if err != nil {
   log.Printf("❌ Could not get players for match %s: %v", matchId, err)
   continue
  }

  // 📨 Send to matched players only
  for _, playerId := range playerIds {
   conn := manager.Get(playerId)
   if conn == nil {
    log.Printf("⚠️ No active WebSocket connection for player %s", playerId)
    continue
   }

   if err := conn.WriteJSON(matchData); err != nil {
    log.Printf("❌ Failed to send match to player %s: %v", playerId, err)
   } else {
    log.Printf("📨 Sent full match info to player %s", playerId)
   }
  }
 }
}

Handling Player Connections

Now that we have a handler where players can establish a WebSocket connection, let’s break down how we keep track of these active players.

Whenever a player connects, we extract them player_id from the query params and upgrade the connection to WebSocket. Then, we store this connection for future use (for example, to notify them when a match is found).

package ws

import (
 "log"
 "net/http"

 "github.com/Yashh56/matchmakingEngine/pkg/clients"
 "github.com/gorilla/websocket"
)

var upgrader = websocket.Upgrader{}

func HandleWebSocket(manager *clients.Manager) http.HandlerFunc {
 return func(w http.ResponseWriter, r *http.Request) {
  playerID := r.URL.Query().Get("player_id")
  if playerID == "" {
   http.Error(w, "player_id required", http.StatusBadRequest)
   return
  }

  conn, err := upgrader.Upgrade(w, r, nil)
  if err != nil {
   log.Println("WebSocket upgrade error:", err)
   return
  }

  manager.Add(playerID, conn)
  log.Printf("🟢 Player connected: %s\n", playerID)
 }
}

Managing Connected Players

But where exactly are we storing these player connections? For that, we use a simple Connection Manager, which acts as an in-memory registry of all connected WebSocket clients.

This manager allows us to:

• Add a new player’s connection when they join.
• Retrieve a player’s connection by their ID when sending them a message.
• Remove a player’s connection when they disconnect.
• List all connected players (useful for broadcasting or debugging).

// package/clients/manager.go
package clients

import (
 "sync"

 "github.com/gorilla/websocket"
)

type Manager struct {
 mu      sync.RWMutex
 clients map[string]*websocket.Conn
}

func NewManager() *Manager {
 return &Manager{
  clients: make(map[string]*websocket.Conn),
 }
}

func (m *Manager) Add(id string, conn *websocket.Conn) {
 m.mu.Lock()
 defer m.mu.Unlock()
 m.clients[id] = conn
}

func (m *Manager) Get(id string) *websocket.Conn {
 m.mu.RLock()
 defer m.mu.RUnlock()
 return m.clients[id]
}

func (m *Manager) GetAllClients() map[string]*websocket.Conn {
 m.mu.RLock()
 defer m.mu.RUnlock()

 // Return a copy to avoid race conditions
 copied := make(map[string]*websocket.Conn)
 for id, conn := range m.clients {
  copied[id] = conn
 }
 return copied
}

func (m *Manager) Remove(id string) {
 m.mu.Lock()
 defer m.mu.Unlock()
 delete(m.clients, id)
}

🎉 Hooray! We’re Done

And that’s it — our simple yet functional matchmaking engine is ready to go!

Of course, this is just a basic implementation to demonstrate the matchmaking process. In a real-world scenario, you could make it much more robust, adding:

• Advanced matchmaking algorithms
• Scalable microservices
• Dynamic game server orchestration
• Fault tolerance and retries
• Distributed queue handling

Here are the Full Code:

GitHub - Yashh56/matchmakingEngine: A scalable real-time matchmaking engine built using Go, Redis, WebSockets, and Kubernetes (via kind or minikube) to simulate game pod creation and player assignment in a distributed environment. This system mimics real-world multiplayer games' server orchestration using custom services and Kubernetes pods.

🚀 What’s Next?

You can extend this project yourself, or just try running the complete version from the GitHub repo.

Once everything is running, you’ll start seeing real-time match events like this:

📨 [14:30:25] New Message Received
────────────────────────────────────────────────────────────
🎯 MATCH FOUND!

📋 Match ID: bf93e3f1-5517-4d8f-a162-44400a40fb30
🗺️  Region: ASIA
👥 Players:
   Player 1:
     🆔 ID: 1
     ⭐ MMR: 1000
     📶 Ping: 30ms
     🎮 Mode: solo
     🌍 Region: asia
     ⏰ Joined: 14:30:20

   Player 2:
     🆔 ID: 3
     ⭐ MMR: 1000
     📶 Ping: 30ms
     🎮 Mode: solo
     🌍 Region: asia
     ⏰ Joined: 14:30:24

💬 Message: 🎯 You've been matched!
────────────────────────────────────────────────────────────

Thanks for following along. Hope this gave you a clear understanding of how matchmaking works in online games and how you can build scalable systems using Go and Redis.

Feel free to fork the repo, build upon this, and maybe even deploy your matchmaking system for your game projects.