sandbox

package
v0.1.14 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 5, 2026 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package sandbox — images.go manages base images for sandboxes.

Images are stored as rootfs.ext4 + vmlinux kernel pairs. Each image has a name, and sandboxes reference images by name.

Directory layout: 

{ImagesDir}/
  ├── default/
  │   ├── rootfs.ext4
  │   └── vmlinux
  ├── python312/
  │   ├── rootfs.ext4
  │   └── vmlinux
  └── node22/
      ├── rootfs.ext4
      └── vmlinux

Package sandbox manages Firecracker microVM lifecycles.

Architecture: 

Manager
  ├── CreateSandbox()  → spawn jailer + firecracker, wait for vsock agent
  ├── ExecInSandbox()  → connect vsock, send ExecRequest, return ExecResponse
  ├── DestroySandbox() → SIGKILL process, cleanup state dir + tap + DB
  ├── Reconcile()      → startup recovery of orphaned VMs
  └── cidAllocator     → atomic counter for unique vsock CIDs

Package sandbox — network.go implements per-sandbox network isolation.

Default policy: DENY ALL. Each sandbox gets a tap device on a bridge, but iptables rules block all traffic by default.

Available policies:

  • "none" — no network access (default)
  • "outbound" — outbound internet via NAT, no inbound, no inter-sandbox
  • "full" — full network access (dangerous, for trusted workloads)

Implementation: 

Host iptables chain: FCLK-{sandbox-id}
  ├── ACCEPT established,related (stateful return traffic)
  ├── DROP inter-sandbox (block tap-to-tap)
  └── policy-specific rules

NAT (outbound policy):
  POSTROUTING -s {vm-ip}/32 -o {wan-iface} -j MASQUERADE

Package sandbox — pool.go implements pre-warmed snapshot pools.

The pool maintains N ready-to-use VM snapshots per image. When a sandbox is requested, it restores from snapshot (~50ms) instead of cold-booting (~1-3s). The pool auto-replenishes in the background.

Pool lifecycle: 

┌───────────┐   boot+snapshot   ┌──────────────┐   restore   ┌─────────┐
│ WARMING   │──────────────────▶│ POOL (ready)  │────────────▶│ CLAIMED │
│ (cold boot│                   │ N snapshots   │             │ (in use)│
│  + snap)  │                   │ per image     │             │         │
└───────────┘                   └──────────────┘             └─────────┘
      ▲                               │ count < target
      └───────────────────────────────┘ (auto-replenish)

Index

Constants

This section is empty.

Variables

View Source 
var ErrAtCapacity = errors.New("at capacity")

ErrAtCapacity is returned when the sandbox manager can't create more VMs.

Functions

func ApplyNetworkPolicy 

func ApplyNetworkPolicy(tapDevice, vmIP string, policy NetworkPolicy, cfg NetworkConfig) error

ApplyNetworkPolicy configures iptables rules for a sandbox.

func RemoveNetworkPolicy 

func RemoveNetworkPolicy(tapDevice, vmIP string, cfg NetworkConfig)

RemoveNetworkPolicy cleans up iptables rules for a sandbox.

Types

type Config 

type Config struct {
	// StateDir is the base directory for VM state (sockets, metadata).
	// Each VM gets a subdirectory: {StateDir}/{sandbox-id}/
	StateDir string

	// FirecrackerBin is the path to the firecracker binary.
	FirecrackerBin string

	// JailerBin is the path to the jailer binary.
	JailerBin string

	// KernelPath is the path to the guest kernel image.
	KernelPath string

	// DefaultRootfs is the path to the default rootfs ext4 image.
	DefaultRootfs string

	// ImagesDir is the directory containing image subdirectories.
	// Each image has {ImagesDir}/{name}/rootfs.ext4.
	ImagesDir string

	// BridgeName is the network bridge for VM tap devices.
	BridgeName string

	// VsockAgentPort is the port the in-VM agent listens on inside vsock.
	VsockAgentPort uint32

	// ExecTimeout is the default timeout for command execution.
	ExecTimeout time.Duration

	// MaxSandboxes is the maximum number of concurrent sandboxes.
	MaxSandboxes int

	// MaxVCPU is the max vCPUs per sandbox (0 = no limit).
	MaxVCPU int

	// MaxMemMiB is the max memory per sandbox in MiB (0 = no limit).
	MaxMemMiB int

	// DefaultVCPU is the default vCPU count when not specified.
	DefaultVCPU int

	// DefaultMemMiB is the default memory in MiB when not specified.
	DefaultMemMiB int

	// Metrics for recording phase timings (optional).
	Metrics interface {
		RecordCreatePhase(ctx context.Context, image, phase string, duration time.Duration)
	}
}

Config holds sandbox manager configuration.

type ImageConfig 

type ImageConfig struct {
	// ImagesDir is the base directory for all images.
	ImagesDir string

	// AgentBinaryPath is the path to the compiled pyro-agent binary.
	// It gets injected into new rootfs images at /usr/bin/pyro-agent.
	AgentBinaryPath string

	// MaxImageSizeMB caps the estimated decompressed rootfs size for
	// registry pulls. Σ(layer_size) × 1.3 must not exceed this. Zero
	// applies defaultMaxImageSizeMB; explicit -1 disables (operators
	// who want to opt out can set MaxImageSizeMB=-1).
	MaxImageSizeMB int
}

ImageConfig configures image management.

type ImageInfo 

type ImageInfo struct {
	Name       string            `json:"name"`
	Status     imagestate.Status `json:"status,omitzero"`
	Source     string            `json:"source,omitzero"`
	Digest     string            `json:"digest,omitzero"`
	Error      string            `json:"error,omitzero"`
	RootfsPath string            `json:"rootfs_path,omitzero"`
	KernelPath string            `json:"kernel_path,omitzero"`
	Size       int64             `json:"size,omitzero"` // rootfs size in bytes
	Labels     map[string]string `json:"labels,omitzero"`
	CreatedAt  time.Time         `json:"created_at,omitzero"`
}

ImageInfo describes a stored base image.

`omitzero` JSON tags keep the response payload terse for in-flight or disk-only views (e.g. a pulling entry has no rootfs path; a ready disk image without a meta sidecar has no digest).

type ImageManager 

type ImageManager struct {
	// contains filtered or unexported fields
}

ImageManager handles base image lifecycle.

func NewImageManager 

func NewImageManager(cfg ImageConfig, log *slog.Logger) (*ImageManager, error)

NewImageManager creates an image manager.

func (*ImageManager) CreateFromDockerfile 

func (im *ImageManager) CreateFromDockerfile(ctx context.Context, name, dockerfilePath string) (*ImageInfo, error)

CreateFromDockerfile builds a rootfs from a Dockerfile. It uses docker/buildah to build the image, then exports the filesystem to an ext4 image with the pyro-agent binary injected.

func (*ImageManager) CreateFromRegistry added in v0.1.11 

func (im *ImageManager) CreateFromRegistry(ctx context.Context, name, source string, force bool, puller *registry.Puller) (*ImageInfo, error)

CreateFromRegistry registers an image pull from a remote OCI registry.

Behavior depends on force and existing state:

  • name absent → start async pull, return pulling info (handler 202).
  • name present + ready on disk + !force → idempotent no-op, return existing ImageInfo with Status=ready (handler 200). No pull, no ledger touch.
  • name present + force → invalidate warm snapshots, start a fresh pull, atomically swap the on-disk rootfs when extraction completes (handler 202).
  • in-flight pull (pulling/extracting) + force → ErrForceDuringPull (handler 409). User must wait for the in-flight pull to settle.

The async pull drives the ledger through pulling → extracting → ready (or failed) in a background goroutine. Callers poll ImageManager.Status(name) or subscribe to the SSE stream.

On failure: non-force removes the partial image directory; force removes only the staged .new files so the prior ready image stays intact. The ledger records the error and the failed entry stays queryable for ~1h.

puller may be nil; in that case a default Puller is created.

func (*ImageManager) Get 

func (im *ImageManager) Get(name string) (*ImageInfo, error)

Get returns info for a specific image.

func (*ImageManager) Ledger added in v0.1.11 

func (im *ImageManager) Ledger() *imagestate.Ledger

Ledger exposes the in-memory pull ledger so the API layer can surface in-flight and recently-failed pull state.

func (*ImageManager) List 

func (im *ImageManager) List() ([]*ImageInfo, error)

List returns all available images.

func (*ImageManager) ListKernels 

func (im *ImageManager) ListKernels() ([]*KernelInfo, error)

ListKernels returns all vmlinux-* kernels in the images dir, sorted by version descending.

func (*ImageManager) ResolveKernel 

func (im *ImageManager) ResolveKernel(version string) (string, error)

ResolveKernel returns the path for a kernel version, or the latest if version is empty.

func (*ImageManager) SetEmitter added in v0.1.11 

func (im *ImageManager) SetEmitter(e imagestate.Emitter)

SetEmitter wires an event emitter for image lifecycle SSE events. The same emitter is installed on the underlying ledger.

func (*ImageManager) SetInvalidator added in v0.1.11 

func (im *ImageManager) SetInvalidator(p PoolInvalidator)

SetInvalidator installs the snapshot-pool invalidator used by force-replace pulls. nil disables invalidation (force pulls still swap the rootfs but warm snapshots survive — only safe in tests).

func (*ImageManager) Status added in v0.1.11 

func (im *ImageManager) Status(name string) *ImageInfo

Status returns the live view of an image: ledger entry if one exists (in-flight or recently-failed), otherwise the disk record. Returns nil if the image is unknown to both the ledger and the disk.

type KernelInfo 

type KernelInfo struct {
	Version string `json:"version"`
	Path    string `json:"path"`
	Size    int64  `json:"size"`
}

KernelInfo describes an available guest kernel.

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager handles Firecracker VM lifecycle operations.

func New 

func New(cfg Config, st *store.Store, log *slog.Logger) (*Manager, error)

New creates a sandbox Manager.

func (*Manager) ActiveCount 

func (m *Manager) ActiveCount() int

ActiveCount returns the number of running sandboxes.

func (*Manager) CreateSandbox 

func (m *Manager) CreateSandbox(ctx context.Context, apiKeyID, image string, ttl time.Duration, res VMResources) (*store.Sandbox, error)

CreateSandbox provisions a new Firecracker microVM.

func (*Manager) DestroySandbox 

func (m *Manager) DestroySandbox(ctx context.Context, id string) error

DestroySandbox kills a VM and cleans up all resources.

func (*Manager) ExecInSandbox 

func (m *Manager) ExecInSandbox(ctx context.Context, id string, req *protocol.ExecRequest) (*protocol.ExecResponse, error)

ExecInSandbox runs a command inside a sandbox via vsock.

func (*Manager) ReadFileFromSandbox 

func (m *Manager) ReadFileFromSandbox(ctx context.Context, id string, req *protocol.FileReadRequest) (*protocol.FileReadResponse, error)

ReadFileFromSandbox reads a file from the in-VM agent.

func (*Manager) Reconcile 

func (m *Manager) Reconcile(ctx context.Context) error

Reconcile recovers orphaned VMs after a server restart.

func (*Manager) SetPool 

func (m *Manager) SetPool(p *Pool)

SetPool attaches a snapshot pool to the manager.

func (*Manager) Shutdown 

func (m *Manager) Shutdown(ctx context.Context)

Shutdown gracefully stops all sandboxes.

func (*Manager) WaitForAgentAt 

func (m *Manager) WaitForAgentAt(stateDir string, timeout time.Duration) error

WaitForAgentAt polls the vsock agent at a given UDS path until it responds.

func (*Manager) WriteFileInSandbox 

func (m *Manager) WriteFileInSandbox(ctx context.Context, id string, req *protocol.FileWriteRequest) (*protocol.FileWriteResponse, error)

WriteFileInSandbox sends a file to the in-VM agent for writing.

type NetworkConfig 

type NetworkConfig struct {
	// WANInterface is the host's outbound network interface (e.g., "eth0").
	WANInterface string

	// SubnetCIDR is the subnet for sandbox VMs (e.g., "172.16.0.0/24").
	SubnetCIDR string
}

NetworkConfig holds network isolation settings.

type NetworkPolicy 

type NetworkPolicy string

NetworkPolicy defines what network access a sandbox has. 

const (
	NetworkNone     NetworkPolicy = "none"
	NetworkOutbound NetworkPolicy = "outbound"
	NetworkFull     NetworkPolicy = "full"
)

type Pool 

type Pool struct {
	// contains filtered or unexported fields
}

Pool manages pre-warmed Firecracker snapshots for fast sandbox creation.

func NewPool 

func NewPool(cfg PoolConfig, manager *Manager, log *slog.Logger) (*Pool, error)

NewPool creates a snapshot pool.

func (*Pool) Available 

func (p *Pool) Available(image string) int

Available returns the number of ready snapshots for an image.

func (*Pool) Claim 

func (p *Pool) Claim(image, kernelPath string) *snapshot

Claim takes a ready snapshot from the pool matching image and kernel. Returns nil if none available (caller should fall back to cold boot).

func (*Pool) Invalidate added in v0.1.11 

func (p *Pool) Invalidate(image string)

Invalidate drains the ready slice for image and removes the corresponding snapshot dirs from disk. The replenishment loop refills at its own cadence. Idempotent — safe for unknown image names and repeat calls.

Used by force-replace pulls (slice 06): warm snapshots reference the old rootfs's blocks, so reusing them after the rootfs is swapped would corrupt new sandboxes. Running sandboxes already hold their own copy from manager.CreateSandbox and are unaffected.

func (*Pool) Run 

func (p *Pool) Run(ctx context.Context)

Run starts the pool replenishment loop. Blocks until ctx is cancelled.

func (*Pool) Stats 

func (p *Pool) Stats() map[string]int

Stats returns pool statistics.

type PoolConfig 

type PoolConfig struct {
	// TargetSize is the number of warm snapshots to maintain per image.
	TargetSize int

	// SnapshotDir is where snapshot files are stored.
	SnapshotDir string

	// ReplenishInterval is how often to check and refill the pool.
	ReplenishInterval time.Duration

	// Images is the list of images to maintain warm pools for.
	// If empty, discovers from ImagesDir.
	Images []string
}

PoolConfig configures the snapshot pool.

type PoolInvalidator added in v0.1.11 

type PoolInvalidator interface {
	Invalidate(image string)
}

PoolInvalidator drops warm snapshots tied to an image. The snapshot pool implements this. Decoupled via interface so the image manager can be tested without spinning up a real pool.

type Reaper 

type Reaper struct {
	// contains filtered or unexported fields
}

Reaper periodically destroys sandboxes that have exceeded their TTL. 

┌──────────┐    tick     ┌──────────────┐    expired?   ┌───────────┐
│  Sleep   │───────────▶│ Query expired │──── yes ─────▶│ Destroy   │
│ interval │            │  sandboxes    │               │  sandbox  │
└──────────┘            └──────────────┘               └───────────┘
     ▲                        │ no                           │
     └────────────────────────┘                              │
     └───────────────────────────────────────────────────────┘

func NewReaper 

func NewReaper(manager *Manager, interval time.Duration, log *slog.Logger) *Reaper

NewReaper creates a TTL reaper.

func (*Reaper) Run 

func (r *Reaper) Run(ctx context.Context)

Run starts the reaper loop. Blocks until ctx is cancelled.

type VMResources 

type VMResources struct {
	VCPU           int    // 0 = use default
	MemMiB         int    // 0 = use default
	KernelPath     string // empty = use server default
	ScratchSizeMiB int    // 0 = no scratch disk, >0 = ephemeral /dev/vdb
}

VMResources holds configurable VM resources.

Source Files

View all Source files

Directories

Path Synopsis
Package imageconfig defines the OCI image runtime config shared between host-side image registration (registry pull / Dockerfile build) and the in-VM agent's exec handler.
 Package imageconfig defines the OCI image runtime config shared between host-side image registration (registry pull / Dockerfile build) and the in-VM agent's exec handler.
Package imageops builds rootfs ext4 filesystems from OCI image layers.
 Package imageops builds rootfs ext4 filesystems from OCI image layers.
Package imagestate owns the in-memory ledger of in-flight and recently failed image pulls.
 Package imagestate owns the in-memory ledger of in-flight and recently failed image pulls.
Package registry pulls OCI/Docker images from a remote registry without a Docker daemon.
 Package registry pulls OCI/Docker images from a remote registry without a Docker daemon.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.