zap

package module
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2026 License: BSD-3-Clause Imports: 4 Imported by: 0

README

zap-proto/go

Canonical Go runtime for the ZAP wire format. See github.com/zap-proto/zap-spec for the format specification.

Documentation

Overview

Package zap implements the Zero-copy Application Protocol (ZAP) runtime.

ZAP is a binary serialization format designed for high-performance inter-process and network communication. It uses a fixed-offset layout so reads are zero-copy: every field is accessed directly from the underlying byte buffer at a compile-time offset, with no parse pass and no allocation.

This package provides the canonical Go runtime for the ZAP wire format. The specification lives at github.com/zap-proto/zap-spec; code generation from ZAP schemas is handled by github.com/zap-proto/go/cmd/zapgen (not yet released in this skeleton).

Wire Format: 

┌─────────────────────────────────────────────────┐
│ Header (16 bytes)                               │
│  ├─ Magic (4 bytes): "ZAP\x00"                  │
│  ├─ Version (2 bytes): 1 or 2                   │
│  ├─ Flags (2 bytes): compression, etc.          │
│  ├─ Root Offset (4 bytes): offset to root       │
│  └─ Size (4 bytes): total message size          │
├─────────────────────────────────────────────────┤
│ Data Segment (variable)                         │
│  └─ Structs, lists, text, bytes...              │
└─────────────────────────────────────────────────┘

All multi-byte integers are little-endian. Offsets are relative to the position of the offset field itself.

Index

Constants

View Source 
const (
	// HeaderSize is the size of the ZAP message header
	HeaderSize = 16

	// Magic bytes identifying a ZAP message
	Magic = "ZAP\x00"

	// ZAP wire versions. Two schema generations share the same data-segment
	// encoding and differ only in the meaning of the leading struct byte:
	//
	//   Version1 — original layout (e.g. legacy platformvm v2 schema, where
	//              byte 0 of the root struct is a payload field).
	//   Version2 — adds a one-byte discriminator at struct byte 0 (the v3
	//              platformvm TxKind tag); every later field shifts by +1.
	//
	// The reader ACCEPTS both versions (forward-compatible parse); consumers
	// that require a specific schema generation gate on Message.Version after
	// Parse. The data segment past the 6-byte magic+version prefix is byte-
	// identical regardless of version — the version byte is the only header
	// difference between a v1 and a v2 buffer carrying the same payload.
	//
	// Version (the bare constant) is the version this runtime's Builder emits
	// by default. zap-proto/go is the pure-stdlib baseline runtime and emits
	// Version1; the hardened downstream runtime (github.com/luxfi/zap) emits
	// Version2 by default. Both readers accept both — see the cross-wire
	// conformance test (zap_crosswire_test.go).
	Version1 uint16 = 1
	Version2 uint16 = 2
	Version  uint16 = Version1

	// DefaultPort is the canonical TCP port for ZAP transport. Like 80
	// means HTTP and 443 means HTTPS, 9999 means ZAP — services that
	// host ZAP endpoints bind this port by convention; the DNS name
	// disambiguates which service is on the other end.
	DefaultPort = 9999

	// Alignment for data segments
	Alignment = 8
)
View Source 
const (
	FlagNone       uint16 = 0
	FlagCompressed uint16 = 1 << 0
	FlagEncrypted  uint16 = 1 << 1
	FlagSigned     uint16 = 1 << 2
)

Flags for message header

Variables

View Source 
var (
	ErrInvalidMagic   = errors.New("zap: invalid magic bytes")
	ErrInvalidVersion = errors.New("zap: unsupported version")
	ErrBufferTooSmall = errors.New("zap: buffer too small")
	ErrOutOfBounds    = errors.New("zap: offset out of bounds")
	ErrInvalidOffset  = errors.New("zap: invalid offset")
)

Functions

func TypeSize 

func TypeSize(t Type) int

TypeSize returns the size of a type in bytes.

Types

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder constructs ZAP messages.

func NewBuilder 

func NewBuilder(capacity int) *Builder

NewBuilder creates a new builder with the given initial capacity. It emits a Version1 header (this runtime's baseline); the data segment is identical across versions, so the choice only sets header byte 4.

func NewBuilderV2 added in v1.3.0 

func NewBuilderV2(capacity int) *Builder

NewBuilderV2 creates a builder that emits a Version2 header. Version2 is the header the ZAP transport framing (and github.com/luxfi/zap) uses by default; the data segment is byte-identical to Version1. Used by the rpc call envelope so its bytes match the other language runtimes' transport envelopes exactly.

func (*Builder) Finish 

func (b *Builder) Finish() []byte

Finish finalizes the message and returns the bytes.

func (*Builder) FinishWithFlags 

func (b *Builder) FinishWithFlags(flags uint16) []byte

FinishWithFlags finalizes with specific flags.

func (*Builder) Reset 

func (b *Builder) Reset()

Reset resets the builder for reuse.

func (*Builder) StartList 

func (b *Builder) StartList(elemSize int) *ListBuilder

StartList starts building a list.

func (*Builder) StartObject 

func (b *Builder) StartObject(dataSize int) *ObjectBuilder

StartObject starts building an object with the given data size.

The buffer is pre-extended to startPos+dataSize and zero-filled, so the caller can SetX into any field offset within the fixed section in any order, and any subsequent StartList / StartObject / WriteBytes call lays its tail data AFTER the fixed section without colliding with fixed-section fields that haven't been written yet. Without this pre-extension, a list whose pointer field is at a lower offset than a fixed bytes_fixed field would have its element data overwritten by the later SetBytesFixed call.

func (*Builder) WriteBytes 

func (b *Builder) WriteBytes(data []byte) int

WriteBytes writes raw bytes and returns the offset.

func (*Builder) WriteText 

func (b *Builder) WriteText(s string) int

WriteText writes a string and returns the offset.

type Enum 

type Enum struct {
	Name   string
	Type   Type // Underlying type (Uint8, Uint16, etc.)
	Values map[string]uint64
}

Enum describes a ZAP enum.

type Field 

type Field struct {
	Name       string
	Type       Type
	Offset     int    // Byte offset within struct
	ListElem   Type   // Element type if Type == TypeList
	StructName string // Struct name if Type == TypeStruct
	Default    any    // Default value
}

Field describes a struct field.

type List 

type List struct {
	// contains filtered or unexported fields
}

List is a zero-copy view into a ZAP list.

func (List) Bytes 

func (l List) Bytes() []byte

Bytes returns the raw bytes of the list (for byte lists).

func (List) BytesAt 

func (l List) BytesAt(i int) []byte

BytesAt returns the i-th raw-bytes element of a variable-element list. Element layout matches ListBuilder.AddBytes for elemSize=0: a 4-byte length prefix followed by the entry bytes.

func (List) Each added in v1.1.1 

func (l List) Each(fn func(i int, elem Object) bool)

Each walks a variable-element list once, parsing each entry's bytes into an Object and calling fn with the index and that Object, in order. It is the single-pass counterpart to calling ObjectAt(0)…ObjectAt(n-1): identical elements, but O(N) total instead of O(N²).

An entry whose bytes do not parse as a ZAP message yields the zero Object (Object{}, IsNull() == true) — the same value ObjectAt returns for an unparseable element — while the cursor still advances by the entry's wire length, so iteration position stays correct. fn returns true to continue, false to stop early; returning false on IsNull reproduces ObjectAt's "stop at the first absent/garbage element" semantics. The walk also stops when the element count is exhausted or the next entry would read past the buffer.

func (List) EachBytes added in v1.1.1 

func (l List) EachBytes(fn func(i int, b []byte) bool)

EachBytes walks a variable-element list once, calling fn with the index and the zero-copy bytes of each entry, in order. It is the single-pass counterpart to calling BytesAt(0)…BytesAt(n-1): identical elements, but O(N) total instead of O(N²).

fn returns true to continue, false to stop early (the caller's break). The walk also stops when the declared element count is exhausted or the next entry would read past the buffer — the latter is the same "stream exhausted" signal BytesAt conveys by returning nil, so a short or over-claiming stream yields only its real entries and never fabricates one.

An empty-payload entry (4-byte zero-length prefix) yields a non-nil zero-length slice, matching BytesAt — a genuine "" survives. The returned slices alias the underlying message data; the caller MUST NOT mutate them.

func (List) IsNull 

func (l List) IsNull() bool

IsNull returns true if the list is null.

func (List) Len 

func (l List) Len() int

Len returns the number of elements.

func (List) Length 

func (l List) Length() int

Length returns the list's wire-encoded element count. Spec-named alias for Len().

func (List) Object 

func (l List) Object(i int, elemSize int) Object

Object returns an object list element.

func (List) ObjectAt 

func (l List) ObjectAt(i int) Object

ObjectAt returns the i-th element of a variable-element list as an Object. Each element on the wire is a self-contained ZAP buffer preceded by a 4-byte little-endian length prefix (the shape emitted by ListBuilder.AddBytes for elemSize=0).

Returns Object{} if the index is out of range or the sub-buffer fails to parse.

func (List) Uint8 

func (l List) Uint8(i int) uint8

Uint8 returns a uint8 list element.

func (List) Uint32 

func (l List) Uint32(i int) uint32

Uint32 returns a uint32 list element.

func (List) Uint32At 

func (l List) Uint32At(i int) uint32

Uint32At returns the i-th uint32 element. Spec-named alias for Uint32.

func (List) Uint64 

func (l List) Uint64(i int) uint64

Uint64 returns a uint64 list element.

func (List) Uint64At 

func (l List) Uint64At(i int) uint64

Uint64At returns the i-th uint64 element. Spec-named alias for Uint64.

type ListBuilder 

type ListBuilder struct {
	// contains filtered or unexported fields
}

ListBuilder builds a ZAP list.

func (*ListBuilder) AddBytes 

func (lb *ListBuilder) AddBytes(data []byte)

AddBytes adds raw bytes (for byte lists).

func (*ListBuilder) AddObjectBytes 

func (lb *ListBuilder) AddObjectBytes(data []byte)

AddObjectBytes appends a single variable-length entry to a list: 4-byte little-endian length prefix followed by data. Increments the element count by 1 (in contrast with AddBytes, which appends raw bytes to a flat byte-stream list and increments count by len(data)).

Used by codegen-emitted builders for `list<T>` fields where T is a nested struct or a variable-width bytes/text payload. The matching reader is List.ObjectAt / List.BytesAt.

func (*ListBuilder) AddUint8 

func (lb *ListBuilder) AddUint8(v uint8)

AddUint8 adds a uint8 element.

func (*ListBuilder) AddUint32 

func (lb *ListBuilder) AddUint32(v uint32)

AddUint32 adds a uint32 element.

func (*ListBuilder) AddUint64 

func (lb *ListBuilder) AddUint64(v uint64)

AddUint64 adds a uint64 element.

func (*ListBuilder) Finish 

func (lb *ListBuilder) Finish() (offset int, length int)

Finish returns the list offset and length.

func (*ListBuilder) FinishOffset 

func (lb *ListBuilder) FinishOffset() int

FinishOffset returns just the list's start offset. Used by codegen- emitted builders that track the element count externally (the count is then passed to ObjectBuilder.SetList alongside the offset).

The parallel runtime's primary Finish() returns (offset, length), suited for in-flight count tracking. FinishOffset is the single-value counterpart for the codegen pattern.

type Message 

type Message struct {
	// contains filtered or unexported fields
}

Message is a ZAP message that can be read zero-copy.

func Parse 

func Parse(data []byte) (*Message, error)

Parse parses a ZAP message from bytes without copying.

Accepts both Version1 and Version2 wire headers (forward-compatible read). Callers that require a specific schema generation gate on Message.Version after Parse.

The declared size field must be at least HeaderSize: a size=0 buffer would otherwise pass Parse and then panic on subsequent Root()/Flags() reads against the empty slice. It is rejected at the wire boundary.

func (*Message) Bytes 

func (m *Message) Bytes() []byte

Bytes returns the underlying byte slice.

func (*Message) Flags 

func (m *Message) Flags() uint16

Flags returns the message flags.

func (*Message) Root 

func (m *Message) Root() Object

Root returns the root object of the message.

func (*Message) Size 

func (m *Message) Size() int

Size returns the total message size.

func (*Message) Version added in v1.1.1 

func (m *Message) Version() uint16

Version returns the wire version of the message (Version1 or Version2).

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object is a zero-copy view into a ZAP struct.

func (Object) Bool 

func (o Object) Bool(fieldOffset int) bool

Bool reads a bool at the given field offset.

func (Object) Bytes 

func (o Object) Bytes(fieldOffset int) []byte

Bytes reads a byte slice at the given field offset (zero-copy).

Wire-format rule: relOffset is an UNSIGNED forward pointer from the field position into the variable section. Negative bit-patterns (high bit set) flow through uint32→int conversion as large positive values and are rejected by the absPos+length > len(data) bounds check. This closes the pointer-escape malleability surface where a signed cast would let a crafted relOffset alias bytes back inside the fixed section or the wire header — a Bytes target can never legitimately live in offsets 0..HeaderSize-1.

func (Object) BytesFixed 

func (o Object) BytesFixed(fieldOffset, n int) []byte

BytesFixed returns a zero-copy slice of n inline bytes at fieldOffset within the object's fixed payload. Used for ids, signatures, public keys — anything declared as bytes_fixed[N] in a .zap schema.

Returns nil if the requested span falls outside the buffer. The returned slice aliases the underlying message data; the caller MUST NOT mutate it.

func (Object) Float32 

func (o Object) Float32(fieldOffset int) float32

Float32 reads a float32 at the given field offset.

func (Object) Float64 

func (o Object) Float64(fieldOffset int) float64

Float64 reads a float64 at the given field offset.

func (Object) Int8 

func (o Object) Int8(fieldOffset int) int8

Int8 reads an int8 at the given field offset.

func (Object) Int16 

func (o Object) Int16(fieldOffset int) int16

Int16 reads an int16 at the given field offset.

func (Object) Int32 

func (o Object) Int32(fieldOffset int) int32

Int32 reads an int32 at the given field offset.

func (Object) Int64 

func (o Object) Int64(fieldOffset int) int64

Int64 reads an int64 at the given field offset.

func (Object) IsNull 

func (o Object) IsNull() bool

IsNull returns true if the object is null.

func (Object) List 

func (o Object) List(fieldOffset int) List

List reads a list at the given field offset.

Wire-format rule: relOffset is SIGNED (see Object()). Any absOffset < HeaderSize is rejected (lists cannot start inside the wire header). The length field is bounded by the total message size — an attacker-set length=0xFFFFFFFF would otherwise let a downstream `for i := 0; i < Len()` loop iterate 4G times even though every per-element accessor returns 0.

func (Object) Object 

func (o Object) Object(fieldOffset int) Object

Object reads a nested object at the given field offset.

Wire-format rule: relOffset is SIGNED. The builder may finalize a nested object BEFORE its parent (in which case the nested payload lives EARLIER in the variable section than the parent's pointer cell, and the relOffset is negative). The bounds check rejects any absOffset outside the message; for the Bytes-malleability rule see Bytes().

An attacker can use a backward relOffset to alias the WIRE HEADER (offsets 0..HeaderSize-1). The header carries Magic/Version/Flags/RootOffset/Size — none of which is a legitimate object payload. Any absOffset < HeaderSize is rejected. The signed cast still lets honest builders point backward to nested objects they finalized first (which live at offset >= HeaderSize).

func (Object) Text 

func (o Object) Text(fieldOffset int) string

Text reads a string at the given field offset (zero-copy).

func (Object) Uint8 

func (o Object) Uint8(fieldOffset int) uint8

Uint8 reads a uint8 at the given field offset.

func (Object) Uint16 

func (o Object) Uint16(fieldOffset int) uint16

Uint16 reads a uint16 at the given field offset.

func (Object) Uint32 

func (o Object) Uint32(fieldOffset int) uint32

Uint32 reads a uint32 at the given field offset.

func (Object) Uint64 

func (o Object) Uint64(fieldOffset int) uint64

Uint64 reads a uint64 at the given field offset.

type ObjectBuilder 

type ObjectBuilder struct {
	// contains filtered or unexported fields
}

ObjectBuilder builds a ZAP object (struct).

func (*ObjectBuilder) Finish 

func (ob *ObjectBuilder) Finish() int

Finish finalizes the object and returns its offset. Writes deferred text/bytes data after the object's fixed section and patches relative offsets.

func (*ObjectBuilder) FinishAsRoot 

func (ob *ObjectBuilder) FinishAsRoot() int

FinishAsRoot finalizes and sets as the message root.

func (*ObjectBuilder) SetBool 

func (ob *ObjectBuilder) SetBool(fieldOffset int, v bool)

SetBool sets a bool field.

func (*ObjectBuilder) SetBytes 

func (ob *ObjectBuilder) SetBytes(fieldOffset int, v []byte)

SetBytes sets a bytes field. The data is written after the object's fixed section during Finish().

func (*ObjectBuilder) SetBytesFixed 

func (ob *ObjectBuilder) SetBytesFixed(fieldOffset int, v []byte)

SetBytesFixed copies len(v) bytes inline at fieldOffset within the object's fixed payload. Used for ids, signatures, public keys — anything declared as bytes_fixed[N] in a .zap schema. Symmetric to Object.BytesFixed on the read side.

A zero-length argument is a no-op (the slot retains the zero value).

func (*ObjectBuilder) SetFloat32 

func (ob *ObjectBuilder) SetFloat32(fieldOffset int, v float32)

SetFloat32 sets a float32 field.

func (*ObjectBuilder) SetFloat64 

func (ob *ObjectBuilder) SetFloat64(fieldOffset int, v float64)

SetFloat64 sets a float64 field.

func (*ObjectBuilder) SetInt8 

func (ob *ObjectBuilder) SetInt8(fieldOffset int, v int8)

SetInt8 sets an int8 field.

func (*ObjectBuilder) SetInt16 

func (ob *ObjectBuilder) SetInt16(fieldOffset int, v int16)

SetInt16 sets an int16 field.

func (*ObjectBuilder) SetInt32 

func (ob *ObjectBuilder) SetInt32(fieldOffset int, v int32)

SetInt32 sets an int32 field.

func (*ObjectBuilder) SetInt64 

func (ob *ObjectBuilder) SetInt64(fieldOffset int, v int64)

SetInt64 sets an int64 field.

func (*ObjectBuilder) SetList 

func (ob *ObjectBuilder) SetList(fieldOffset int, listOffset int, length int)

SetList sets a list field.

func (*ObjectBuilder) SetObject 

func (ob *ObjectBuilder) SetObject(fieldOffset int, objOffset int)

SetObject sets a nested object field (by offset).

func (*ObjectBuilder) SetText 

func (ob *ObjectBuilder) SetText(fieldOffset int, v string)

SetText sets a text (string) field.

func (*ObjectBuilder) SetUint8 

func (ob *ObjectBuilder) SetUint8(fieldOffset int, v uint8)

SetUint8 sets a uint8 field.

func (*ObjectBuilder) SetUint16 

func (ob *ObjectBuilder) SetUint16(fieldOffset int, v uint16)

SetUint16 sets a uint16 field.

func (*ObjectBuilder) SetUint32 

func (ob *ObjectBuilder) SetUint32(fieldOffset int, v uint32)

SetUint32 sets a uint32 field.

func (*ObjectBuilder) SetUint64 

func (ob *ObjectBuilder) SetUint64(fieldOffset int, v uint64)

SetUint64 sets a uint64 field.

type Schema 

type Schema struct {
	Name    string
	Structs map[string]*Struct
	Enums   map[string]*Enum
}

Schema describes a complete ZAP schema.

func NewSchema 

func NewSchema(name string) *Schema

NewSchema creates a new empty schema.

func (*Schema) AddEnum 

func (s *Schema) AddEnum(e *Enum)

AddEnum adds an enum to the schema.

func (*Schema) AddStruct 

func (s *Schema) AddStruct(st *Struct)

AddStruct adds a struct to the schema.

type Struct 

type Struct struct {
	Name   string
	Size   int // Total size in bytes
	Fields []Field
}

Struct describes a ZAP struct.

type StructBuilder 

type StructBuilder struct {
	// contains filtered or unexported fields
}

StructBuilder helps build struct definitions.

func NewStructBuilder 

func NewStructBuilder(name string) *StructBuilder

NewStructBuilder creates a struct builder.

func (*StructBuilder) Bool 

func (sb *StructBuilder) Bool(name string) *StructBuilder

Bool adds a bool field.

func (*StructBuilder) Build 

func (sb *StructBuilder) Build() *Struct

Build finalizes and returns the struct.

func (*StructBuilder) Bytes 

func (sb *StructBuilder) Bytes(name string) *StructBuilder

Bytes adds a bytes field.

func (*StructBuilder) Float64 

func (sb *StructBuilder) Float64(name string) *StructBuilder

Float64 adds a float64 field.

func (*StructBuilder) Int32 

func (sb *StructBuilder) Int32(name string) *StructBuilder

Int32 adds an int32 field.

func (*StructBuilder) Int64 

func (sb *StructBuilder) Int64(name string) *StructBuilder

Int64 adds an int64 field.

func (*StructBuilder) List 

func (sb *StructBuilder) List(name string, elemType Type) *StructBuilder

List adds a list field.

func (*StructBuilder) Struct 

func (sb *StructBuilder) Struct(name string, structName string) *StructBuilder

Struct adds a nested struct field.

func (*StructBuilder) Text 

func (sb *StructBuilder) Text(name string) *StructBuilder

Text adds a text field.

func (*StructBuilder) Uint32 

func (sb *StructBuilder) Uint32(name string) *StructBuilder

Uint32 adds a uint32 field.

func (*StructBuilder) Uint64 

func (sb *StructBuilder) Uint64(name string) *StructBuilder

Uint64 adds a uint64 field.

type Type 

type Type uint8

Type represents a ZAP type. 

const (
	TypeVoid Type = iota
	TypeBool
	TypeInt8
	TypeInt16
	TypeInt32
	TypeInt64
	TypeUint8
	TypeUint16
	TypeUint32
	TypeUint64
	TypeFloat32
	TypeFloat64
	TypeText
	TypeBytes
	TypeList
	TypeStruct
	TypeEnum
	TypeUnion
)

Source Files

View all Source files

Directories

Path Synopsis
Package cap implements the ZAP capability runtime.
 Package cap implements the ZAP capability runtime.
cmd
zapgen command
zapgen reads a .zap schema file and emits per-struct Go accessor + builder code that calls the github.com/zap-proto/go runtime.
 zapgen reads a .zap schema file and emits per-struct Go accessor + builder code that calls the github.com/zap-proto/go runtime.
examples
agents command
Multi-Agent LLM Consensus via ZAP Protocol
 Multi-Agent LLM Consensus via ZAP Protocol
echo
Package echo is a self-contained demo of zapgen interface (RPC) codegen.
 Package echo is a self-contained demo of zapgen interface (RPC) codegen.
Package rpc is the ZAP call envelope: the msgType + method + capability framing that carries an interface method call and its response over a ZAP transport.
 Package rpc is the ZAP call envelope: the msgType + method + capability framing that carries an interface method call and its response over a ZAP transport.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.