ocijoin

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 3, 2026 License: MIT Imports: 20 Imported by: 0

README

oci-join

Composable primitives for reading, merging, filtering, and exporting OCI image layouts in Go.

Overview

ocijoin operates on a simple Layout interface that pairs an OCI index with containerd's content.Provider for blob access:

type Layout interface {
    content.Provider
    Index(ctx context.Context) (*ocispec.Index, error)
}

Because Layout embeds content.Provider, every layout is directly usable with containerd's push/pull infrastructure.

Primitives

Function Description
NewLocalLayout(path) Read an OCI layout directory from disk
Join(layouts...) Merge multiple layouts into a single deduplicated view
Filter(layout, fn) Keep only index descriptors matching a predicate
Unwrap(layout) Resolve a nested index (outer index → inner index)
Wrap(layout, annotations) Nest a layout's index inside a new outer index
Store(layout) Adapt a layout into a containerd content.Store (read-only)
IsIndex(desc) Predicate: is this an image index / manifest list?
IsAttestation(desc) Predicate: is this a Docker attestation manifest?
Export sub-packages
Package Description
direxport Write a layout as an OCI image layout directory
tarexport Write a layout as an OCI image layout tar archive

Usage

A typical multiplatform merge — combining per-platform layouts produced by docker buildx into a single layout suitable for oras cp:

layouts := make([]ocijoin.Layout, len(paths))
for i, p := range paths {
    l, _ := ocijoin.NewLocalLayout(p)
    layouts[i] = ocijoin.Unwrap(l) // resolve nested outer→inner index
}

merged := ocijoin.Join(layouts...)
wrapped := ocijoin.Wrap(merged, map[string]string{
    "org.opencontainers.image.ref.name": tag,
})

// Write to disk for oras or other tools.
direxport.Export(ctx, wrapped, outputDir)

// Or write as a tar archive.
tarexport.Export(ctx, wrapped, w)
Why Unwrap/Wrap?

Many build tools (Docker buildx, BuildKit) produce OCI layouts with a nested index structure: the top-level index.json contains a single descriptor pointing to an inner ImageIndex blob, which in turn lists the actual platform manifests and attestation manifests.

Unwrap resolves this nesting so you can work with the platform manifests directly. After merging, Wrap re-creates the nested structure with the annotations that tools like oras expect (e.g. org.opencontainers.image.ref.name for tag resolution).

If a layout is not nested, Unwrap returns it unchanged.

Filtering
// Remove attestation manifests.
clean := ocijoin.Filter(layout, func(desc ocispec.Descriptor) bool {
    return !ocijoin.IsAttestation(desc)
})

Filtering only affects the index — all blobs remain accessible through the filtered layout's ReaderAt.

Documentation

Overview

Package ocijoin provides composable primitives for reading, merging, filtering, wrapping, and exporting OCI image layouts.

The central type is Layout, an interface that combines an OCI index with containerd's content.Provider for blob access. Layouts are assumed to be immutable.

Primitives can be composed freely:

  • NewLocalLayout reads an OCI layout directory from disk.
  • Join merges multiple layouts into a single deduplicated view.
  • Filter keeps only index descriptors matching a predicate.
  • Unwrap resolves a nested index (outer index pointing to inner index).
  • Wrap nests a layout's index inside a new outer index with annotations.
  • Store adapts a layout into a containerd content.Store (read-only).
  • IsIndex and IsAttestation are predicate helpers for common filtering.

Because Layout embeds content.Provider, all layouts are directly usable with containerd's image push/pull infrastructure.

Sub-packages [tarexport] and [direxport] write layouts to tar archives and filesystem directories respectively.

A typical multiplatform merge workflow:

// Load per-platform layouts produced by docker build.
for _, path := range layoutPaths {
    l, _ := ocijoin.NewLocalLayout(path)
    layouts = append(layouts, ocijoin.Unwrap(l))
}

// Merge, re-wrap with tag annotations, and export.
merged := ocijoin.Join(layouts...)
wrapped := ocijoin.Wrap(merged, map[string]string{
    "org.opencontainers.image.ref.name": tag,
})
direxport.Export(ctx, wrapped, outputDir)
Example (MultiplatformMerge)

This example shows a complete multiplatform merge workflow: load per-platform layouts, unwrap nested indexes, merge, re-wrap with tag annotations, and export to a directory.

package main

import (
	"bytes"
	"context"

	"github.com/cpuguy83/ocijoin"
	"github.com/cpuguy83/ocijoin/direxport"
	"github.com/cpuguy83/ocijoin/tarexport"
)

func main() {
	layoutPaths := []string{
		"/path/to/linux-amd64",
		"/path/to/linux-arm64",
	}

	layouts := make([]ocijoin.Layout, len(layoutPaths))
	for i, p := range layoutPaths {
		l, _ := ocijoin.NewLocalLayout(p)
		layouts[i] = ocijoin.Unwrap(l)
	}

	merged := ocijoin.Join(layouts...)
	wrapped := ocijoin.Wrap(merged, map[string]string{
		"org.opencontainers.image.ref.name": "v1.0.0",
	})

	// Export as a directory for use with oras or other tools.
	_ = direxport.Export(context.Background(), wrapped, "/path/to/output")

	// Or export as a tar archive.
	var buf bytes.Buffer
	_ = tarexport.Export(context.Background(), wrapped, &buf)
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsAttestation

func IsAttestation(desc ocispec.Descriptor) bool

IsAttestation reports whether desc is an attestation manifest, identified by the "vnd.docker.reference.type" annotation.

func IsIndex

func IsIndex(desc ocispec.Descriptor) bool

IsIndex reports whether desc has an image index media type. This covers both OCI image indexes and Docker manifest lists.

func NewFS added in v0.2.0

func NewFS(ctx context.Context, l Layout) fs.FS

NewFS returns an fs.FS that presents the layout as an OCI image layout directory tree. The tree contains oci-layout, index.json, and all blobs referenced by the index's manifest tree under blobs/<algorithm>/<encoded>.

The returned FS implements fs.StatFS, fs.ReadFileFS, and fs.ReadDirFS. The context is used for index access and blob reads.

func Store added in v0.3.0

func Store(l Layout) content.Store

Store adapts a Layout into a containerd content.Store.

A Layout is already a content.Provider (it serves blobs by digest via ReaderAt). Store fills in the one additional method consumers of an immutable, read-only store actually exercise — Info, which reports a blob's digest and size — and stubs the mutating half of the content.Store interface (Writer/Update/Delete/Status/ListStatuses/Abort) with errdefs.ErrNotImplemented.

The motivating use case is BuildKit's oci-layout source, whose SolveOpt.OCIStores field is typed map[string]content.Store. On the read path BuildKit only calls Info (once, to size the root descriptor) and ReaderAt, so a Layout can be handed to it directly via Store without first being copied into a real, writable content store on disk.

Walk is implemented as a no-op that returns nil: the adapter exposes no "active" content for filtered enumeration. The mutating methods return errdefs.ErrNotImplemented because a Layout is immutable.

Types

type Layout

type Layout interface {
	content.Provider

	// Index returns the OCI index for this layout.
	Index(ctx context.Context) (*ocispec.Index, error)
}

Layout provides read-only access to an immutable OCI image layout. It embeds content.Provider, making it directly usable with containerd's push/pull infrastructure.

func Filter

func Filter(l Layout, fn func(ocispec.Descriptor) bool) Layout

Filter returns a Layout whose Index contains only the descriptors from l's Index for which fn returns true. Blob access via ReaderAt is unaffected; all blobs remain accessible regardless of filtering.

Example
package main

import (
	"context"
	"fmt"

	"github.com/cpuguy83/ocijoin"

	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)

func main() {
	// Filter keeps only index descriptors matching a predicate.
	// Blob access is unaffected -- all blobs remain accessible.
	layout, _ := ocijoin.NewLocalLayout("/path/to/oci-layout")

	// Keep only non-attestation manifests.
	filtered := ocijoin.Filter(layout, func(desc ocispec.Descriptor) bool {
		return !ocijoin.IsAttestation(desc)
	})

	idx, _ := filtered.Index(context.Background())
	fmt.Println("manifests after filtering:", len(idx.Manifests))
}

func Join

func Join(layouts ...Layout) Layout

Join returns a Layout that presents a merged view of the given layouts. The merged index contains deduplicated descriptors from all layouts. Blob lookups search the underlying layouts in order.

Example
package main

import (
	"context"
	"fmt"

	"github.com/cpuguy83/ocijoin"
)

func main() {
	// Join merges multiple layouts into a single view.
	// Duplicate descriptors (by JSON equality) are deduplicated.
	amd64, _ := ocijoin.NewLocalLayout("/path/to/linux-amd64")
	arm64, _ := ocijoin.NewLocalLayout("/path/to/linux-arm64")

	merged := ocijoin.Join(amd64, arm64)

	idx, _ := merged.Index(context.Background())
	for _, desc := range idx.Manifests {
		fmt.Println(desc.Digest)
	}
}

func Unwrap

func Unwrap(l Layout) Layout

Unwrap returns a Layout whose Index is the first nested image index found in l's Index. If l's Index contains no image index descriptors, l is returned unchanged.

This is equivalent to UnwrapWithFilter(l, nil).

Example
package main

import (
	"context"
	"fmt"

	"github.com/cpuguy83/ocijoin"
)

func main() {
	// Unwrap resolves a nested index. Many build tools (e.g. docker buildx)
	// produce layouts where index.json contains a single descriptor pointing
	// to an inner ImageIndex blob. Unwrap reads that blob and returns a
	// layout whose Index is the inner index.
	//
	// If the layout is not nested, it is returned unchanged.
	layout, _ := ocijoin.NewLocalLayout("/path/to/oci-layout")

	unwrapped := ocijoin.Unwrap(layout)

	idx, _ := unwrapped.Index(context.Background())
	for _, desc := range idx.Manifests {
		fmt.Println(desc.Platform.Architecture, desc.Digest)
	}
}

func UnwrapWithFilter

func UnwrapWithFilter(l Layout, fn func(ocispec.Descriptor) bool) Layout

UnwrapWithFilter returns a Layout whose Index is the first nested image index found in l's Index for which fn returns true. The IsIndex check is always applied; fn provides additional filtering on top of that. If fn is nil, any image index descriptor matches.

If no matching descriptor is found, l is returned unchanged. The returned Layout shares the same blob store as l.

func Wrap

func Wrap(l Layout, annotations map[string]string) Layout

Wrap returns a Layout that nests l's index inside a new outer index. The inner index is serialized as a blob and referenced by a single descriptor in the outer index. The descriptor carries the given annotations and has media type application/vnd.oci.image.index.v1+json.

This is the inverse of Unwrap: Unwrap(Wrap(l, ann)) produces a layout whose Index is equivalent to l's Index.

The returned Layout serves both the synthetic inner-index blob and all blobs from l.

Example
package main

import (
	"context"
	"fmt"

	"github.com/cpuguy83/ocijoin"
)

func main() {
	// Wrap nests a layout's index inside a new outer index with annotations.
	// This is the inverse of Unwrap and is useful for producing layouts
	// compatible with tools like oras that expect a tagged outer index.
	layout, _ := ocijoin.NewLocalLayout("/path/to/oci-layout")

	wrapped := ocijoin.Wrap(layout, map[string]string{
		"org.opencontainers.image.ref.name": "v1.0.0",
		"io.containerd.image.name":          "registry.example.com/myimage:v1.0.0",
	})

	idx, _ := wrapped.Index(context.Background())
	fmt.Println("outer descriptors:", len(idx.Manifests))
	fmt.Println("tag:", idx.Manifests[0].Annotations["org.opencontainers.image.ref.name"])
}

type LocalLayout

type LocalLayout struct {
	// contains filtered or unexported fields
}

LocalLayout is a Layout backed by an OCI image layout directory on the local filesystem. The index is eagerly loaded at construction time and cached.

func NewLocalLayout

func NewLocalLayout(path string) (*LocalLayout, error)

NewLocalLayout creates a new Layout backed by the OCI layout directory at the given path. The index.json is parsed eagerly.

Example
package main

import (
	"context"
	"fmt"

	"github.com/cpuguy83/ocijoin"
)

func main() {
	// NewLocalLayout reads an OCI layout directory from disk.
	// The index.json is parsed eagerly at construction time.
	layout, err := ocijoin.NewLocalLayout("/path/to/oci-layout")
	if err != nil {
		fmt.Println("error:", err)
		return
	}

	idx, _ := layout.Index(context.Background())
	fmt.Println("manifests:", len(idx.Manifests))
}

func (*LocalLayout) Index

func (l *LocalLayout) Index(_ context.Context) (*ocispec.Index, error)

Index returns the cached OCI index for this layout.

func (*LocalLayout) ReaderAt

ReaderAt returns a content.ReaderAt for the blob identified by the descriptor's digest.

Directories

Path Synopsis
Package direxport writes OCI image layouts as directories on the local filesystem.
Package direxport writes OCI image layouts as directories on the local filesystem.
Package tarexport writes OCI image layouts as tar archives.
Package tarexport writes OCI image layouts as tar archives.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL