Record Management

Relevant source files

• ingress.go
• legacy_session_store.go
• metrics.go
• record.go
• store.go

This document describes the Record structure and its management within the relay system. A Record represents an active session's metadata, associating a unique key with a tunnel session, HTTP transport, client information, and filtering tags. Records are the primary data structure stored in the Store's registry and enable session lookup, routing, and metadata tracking.

For information about the Store implementation that manages the registry of Records, see Store and Session Management. For details on how Records are used during HTTP request processing, see Ingress Handler and HTTP Processing.

---

Record Structure

The Record type is defined in record.go13-22 and contains all metadata associated with an active session:

Field	Type	JSON Tag	Purpose
Key	string	"key"	Unique identifier (onion ID or TCP port)
Session	tunnel.Session	"-"	Underlying tunnel session (not serialized)
RoundTripper	http.RoundTripper	"metrics"	HTTP transport for proxying requests
Header	tags.Tags	"header"	HTTP headers from the connection upgrade request
Tags	tags.Tags	"tags"	Query parameters for filtering and metadata
Since	time.Time	"since"	Session establishment timestamp
IP	string	"ip"	Client's real IP address
Path	string	"path"	Original connection path

The Session field is excluded from JSON serialization (marked with json:"-"), while the RoundTripper is serialized under the key "metrics", allowing metrics from a MetricsTransport wrapper to be exposed.

Record Field Relationships

Sources: record.go13-22

---

Record Creation During Session Upsert

Records are created by the Store.Upsert method in store.go174-212 when a new edge connection is established or an existing session is updated. The creation process captures metadata from the edge.Edge structure:

Record Creation Flow

Sources: store.go174-212

The Upsert method performs the following steps:

1. Timestamp Capture: Records the current time as Since store.go175
2. Header Conversion: Converts edge.Header to tags.Tags store.go176
3. Tags Conversion: Converts edge.Values to tags.Tags store.go177
4. Record Construction: Creates a new Record instance store.go178-186
5. RoundTripper Assignment: For HTTP protocol edges, creates a RoundTripper from the session store.go188-190
6. Map Storage: Inserts or updates the record in RecordMap store.go193-196
7. Logging: Logs "insert" or "update" action store.go198-204
8. Goroutines: Launches Ping (if enabled) and Scan goroutines store.go206-209
9. Metrics: Increments expvars.WebteleportRelaySessionsAccepted store.go211

Sources: store.go174-212

---

Record Storage in RecordMap

The Store maintains a RecordMap field of type map[string]*Record defined in store.go35 This map serves as the primary registry of active sessions, with keys being either:

• Onion IDs: For HTTP protocol connections, derived using deriveOnionID(edge.Path)
• TCP Ports: For TCP protocol connections, formatted as ":port" after allocation via freeport.GetFreePort()

The map is protected by a sync.RWMutex in store.go32 ensuring thread-safe concurrent access. All mutations to RecordMap occur within the Store.Mut method store.go70-75 which acquires an exclusive lock, executes a mutation function, releases the lock, and triggers an optional OnUpdate callback.

Store RecordMap Structure

Sources: store.go29-37 store.go59-68

---

Record Lookup Methods

The Store provides two primary methods for retrieving Records:

GetRecord Method

The GetRecord method in store.go140-149 implements host-based lookup with internationalized domain name (IDN) support:

Sources: store.go140-149

The method:

1. Strips the port from the hostname store.go141
2. Converts IDN (internationalized domain names) to ASCII using idna.ToASCII store.go142
3. Extracts the subdomain (first component before .) store.go143
4. Calls LookupRecord to perform the actual lookup store.go144

LookupRecord Method

The LookupRecord method in store.go114-125 provides key-based lookup with alias resolution:

LookupRecord with Alias Resolution

Sources: store.go114-125

The lookup algorithm:

1. Acquires a read lock store.go115
2. Attempts direct lookup in RecordMap store.go116
3. If not found, checks AliasMap for an alias store.go117-118
4. If alias exists, performs another lookup using the aliased key store.go119-120
5. Releases the read lock store.go123
6. Returns the record and success status store.go124

Sources: store.go114-125

---

Record Filtering with Matches

The Matches method in record.go24-39 enables filtering Records based on query parameters. This is used by the RecordsHandler in ingress.go53-68 to return only Records matching specific tag criteria.

Matching Algorithm

Sources: record.go24-39

The matching logic implements a subset check:

1. For each key in the query parameters record.go25
2. Verify the Record's Tags contains that key record.go27-29
3. For each value in the query parameter's value list record.go32
4. Verify the Record's tag values contain that value record.go33-35
5. Return true only if all keys and values match record.go38

Example usage in RecordsHandler ingress.go56-60:

for _, rec := range all {
    if rec.Matches(r.URL.Query()) {
        filtered = append(filtered, rec)
    }
}

Sources: record.go24-39 ingress.go53-68

---

Record Lifecycle

A Record transitions through three primary states: creation, active use, and removal.

Record State Diagram

Sources: store.go127-138 store.go174-212 store.go214-238

Creation Phase

Records are created during the subscription flow when an upgrader produces a new edge.Edge:

1. Allocation: store.go151-172

   • TCP protocol: Allocates a free port store.go161-166
   • HTTP protocol: Derives an onion ID from the path store.go169-171

2. Upsertion: store.go174-212

   • Creates Record instance with metadata
   • Assigns RoundTripper for HTTP sessions
   • Inserts into RecordMap
   • Launches monitoring goroutines

Active Phase

During active operation, Records serve two concurrent purposes:

1. Session Monitoring: store.go214-238

   • Ping: Sends periodic keepalives store.go214-223
   • Scan: Reads control commands ("PONG", "CLOSE") store.go225-238

2. Request Handling: ingress.go45-51

   • Lookup via GetRecord or LookupRecord
   • Extract RoundTripper for HTTP proxying
   • Process requests through reverse proxy

Termination Phase

Records are removed when sessions terminate:

1. Trigger: Write error in Ping, "CLOSE" command in Scan, or scanner error
2. Removal: store.go127-138
   • Mutex-protected iteration through RecordMap
   • Match by tunnel.Session identity store.go130
   • Delete from map store.go131
   • Log removal store.go132
3. Metrics: Increment expvars.WebteleportRelaySessionsClosed store.go137

Sources: store.go127-138 store.go151-172 store.go174-212 store.go214-238

---

Integration with HTTP Processing

Records integrate with the HTTP processing layer through their RoundTripper field, which is set during creation for HTTP protocol edges store.go188-190:

if edgeProtocol(r) == "http" {
    rec.RoundTripper = RoundTripper(r.Session)
}

The RoundTripper function (defined in the transport layer) wraps the tunnel.Session in a MetricsTransport to collect statistics. This enables the relay to:

1. Route Requests: Extract the RoundTripper via GetRoundTripper ingress.go45-51
2. Collect Metrics: Automatically track bytes sent/received, request counts, and durations
3. Serialize Stats: Expose metrics through JSON serialization record.go16

The IngressHandler.Dispatch method demonstrates this integration ingress.go118-134:

rt, ok := i.GetRoundTripper(r.Host)
if !ok {
    return utils.HostNotFoundHandler()
}
rp := utils.LoggedReverseProxy(rt)

For details on how the RoundTripper adapts tunnel sessions to HTTP transports, see HTTP Tunneling with RoundTripper. For information on metrics collection, see Metrics Transport.

Sources: store.go188-190 ingress.go45-51 ingress.go118-134 record.go16

---

Records Endpoint

The relay exposes a RESTful API for retrieving Records at the /records endpoint, implemented by IngressHandler.RecordsHandler in ingress.go53-68 This endpoint:

1. Sets JSON content type ingress.go54
2. Retrieves all Records from storage ingress.go55
3. Filters Records using the Matches method ingress.go56-61
4. Serializes filtered Records to JSON ingress.go62-67

Query parameters enable filtering by tags. For example:

• GET /records?protocol=http returns only HTTP protocol sessions
• GET /records?protocol=http&key=value returns sessions matching both criteria

The JSON response includes all Record fields except Session (which is excluded via json:"-" tag). The RoundTripper field is serialized as "metrics", exposing the MetricsTransport statistics if present.

Sources: ingress.go53-68 record.go13-22