rockyardkv

package module
v0.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 2, 2026 License: Apache-2.0 Imports: 37 Imported by: 0

README

RockyardKV

A pure Go implementation of RocksDB with bit-compatible on-disk formats.

Go Reference Go Report Card Ask DeepWiki


Overview

RockyardKV is a pure Go implementation of RocksDB. It reads and writes RocksDB databases without CGo or C++ dependencies.

The project targets bit-compatible file formats with RocksDB. Files created by RockyardKV can be read by C++ RocksDB, and vice versa.

Built with a lot of respect for RocksDB - the foundational storage engine that inspired this work. No affiliation or endorsement implied.

[!NOTE] Project status: Core storage operations work with verified format compatibility. Durability semantics are under active verification. Refer to docs/status/ for compatibility details and known limitations.

Features

  • Storage: LSM-tree with leveled, universal, and FIFO compaction; column families; snapshots; Bloom filters; Snappy, LZ4, Zstd, and Zlib compression
  • Write path: Merge operators, range deletions, SST ingestion, write stall control
  • Transactions: Optimistic and pessimistic modes with deadlock detection
  • Operations: Backup engine, checkpoints, compaction filters, TTL, rate limiting
  • Deployment: Read-only mode, secondary instances, Direct I/O

Installation

go get github.com/aalhour/rockyardkv

Requires Go 1.25 or later.

Quick start

Open a database and perform basic operations:

package main

import (
    "log"

    "github.com/aalhour/rockyardkv"
)

func main() {
    opts := rockyardkv.DefaultOptions()
    opts.CreateIfMissing = true

    database, err := rockyardkv.Open("/tmp/mydb", opts)
    if err != nil {
        log.Fatal(err)
    }
    defer database.Close()

    // Write a key-value pair
    err = database.Put(rockyardkv.DefaultWriteOptions(), []byte("key"), []byte("value"))
    if err != nil {
        log.Fatal(err)
    }

    // Read the value back
    value, err := database.Get(rockyardkv.DefaultReadOptions(), []byte("key"))
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("value: %s", value)
}

Documentation

Refer to the docs directory for detailed guides:

Command-line tools

The cmd/ directory contains utilities for database inspection and testing. Refer to cmd/README.md for details.

Compatibility

RockyardKV targets RocksDB v10.7.5 (commit 812b12b). Run make test-e2e-golden to verify format compatibility using C++ oracle tools.

Refer to docs/status for the compatibility matrix and verification details.

Benchmarks

Refer to docs/benchmarks.md for performance measurements.

Contributing

Refer to CONTRIBUTING.md for development setup and guidelines.

License

Apache 2.0. Refer to LICENSE for the full text.

Documentation

Overview

Package rockyardkv provides a pure-Go, RocksDB-compatible embedded durable key/value store.

RockyardKV targets on-disk format compatibility with RocksDB v10.7.5 for SST files, WAL, and MANIFEST. It provides an LSM-tree based storage engine suitable for high-write workloads. It provides APIs and features that are semantically compatible with RocksDB as well. Full API and feature parity is underway.

Usage

For runnable examples, see the repository's examples directory. The examples are written against the public API and are kept up-to-date as the API evolves.

Concurrency

A DB instance is safe for concurrent use by multiple goroutines. Individual Iterator instances are not safe for concurrent use; each goroutine should use its own iterator.

Compatibility

SST files created by RockyardKV are intended to be readable by C++ RocksDB v10.7.5 and vice versa.

Reference: RocksDB v10.7.5 include/rocksdb/db.h

Index

Examples

Constants

View Source
const (
	// Memtable properties
	PropertyNumImmutableMemTable        = "rocksdb.num-immutable-mem-table"
	PropertyNumImmutableMemTableFlushed = "rocksdb.num-immutable-mem-table-flushed"
	PropertyMemTableFlushPending        = "rocksdb.mem-table-flush-pending"
	PropertyCurSizeActiveMemTable       = "rocksdb.cur-size-active-mem-table"
	PropertyCurSizeAllMemTables         = "rocksdb.cur-size-all-mem-tables"
	PropertyNumEntriesActiveMemTable    = "rocksdb.num-entries-active-mem-table"
	PropertyNumDeletesActiveMemTable    = "rocksdb.num-deletes-active-mem-table"

	// Compaction properties
	PropertyCompactionPending     = "rocksdb.compaction-pending"
	PropertyNumRunningFlushes     = "rocksdb.num-running-flushes"
	PropertyNumRunningCompactions = "rocksdb.num-running-compactions"

	// Level properties (use PropertyNumFilesAtLevelPrefix + "N")
	PropertyNumFilesAtLevelPrefix = "rocksdb.num-files-at-level"
	PropertyLevelStats            = "rocksdb.levelstats"

	// Snapshot properties
	PropertyNumSnapshots       = "rocksdb.num-snapshots"
	PropertyOldestSnapshotTime = "rocksdb.oldest-snapshot-time"

	// Key estimates
	PropertyEstimateNumKeys = "rocksdb.estimate-num-keys"

	// Live data size
	PropertyEstimateLiveDataSize = "rocksdb.estimate-live-data-size"
	PropertyTotalSstFilesSize    = "rocksdb.total-sst-files-size"
	PropertyLiveSstFilesSize     = "rocksdb.live-sst-files-size"

	// Background errors
	PropertyBackgroundErrors = "rocksdb.background-errors"

	// CF and version info
	PropertyNumLiveVersions           = "rocksdb.num-live-versions"
	PropertyCurrentSuperVersionNumber = "rocksdb.current-super-version-number"
	PropertyNumColumnFamilies         = "rocksdb.num-column-families"
)

Property name constants for GetProperty. Reference: RocksDB include/rocksdb/db.h

View Source
const (
	LockTypeShared    = txn.LockTypeShared
	LockTypeExclusive = txn.LockTypeExclusive
)

Lock type constants.

View Source
const (
	CompressionNone   = compression.NoCompression
	CompressionSnappy = compression.SnappyCompression
	CompressionZstd   = compression.ZstdCompression
	CompressionLZ4    = compression.LZ4Compression
)

Compression type constants.

View Source
const (
	NoCompression     = compression.NoCompression
	SnappyCompression = compression.SnappyCompression
	ZlibCompression   = compression.ZlibCompression
	LZ4Compression    = compression.LZ4Compression
	LZ4HCCompression  = compression.LZ4HCCompression
	ZstdCompression   = compression.ZstdCompression
)

Compression type constants

View Source
const (
	ChecksumTypeNoChecksum = checksum.TypeNoChecksum
	ChecksumTypeCRC32C     = checksum.TypeCRC32C
	ChecksumTypeXXHash     = checksum.TypeXXHash
	ChecksumTypeXXHash64   = checksum.TypeXXHash64
	ChecksumTypeXXH3       = checksum.TypeXXH3
)

Checksum type constants

View Source
const (
	// OptionsFileVersion is the current options file format version
	OptionsFileVersion = 1

	// OptionsFilePrefix is the prefix for options file names
	OptionsFilePrefix = "OPTIONS-"
)
View Source
const DefaultColumnFamilyID uint32 = 0

DefaultColumnFamilyID is the ID of the default column family.

View Source
const DefaultColumnFamilyName = "default"

DefaultColumnFamilyName is the name of the default column family.

View Source
const (
	// TTLTimestampSize is the size of the timestamp suffix in bytes.
	TTLTimestampSize = 8 // int64 Unix timestamp
)
View Source
const TimestampSize = 8

TimestampSize is the size of a uint64 timestamp in bytes.

Variables

View Source
var (
	// ErrColumnFamilyNotFound is returned when a column family is not found.
	ErrColumnFamilyNotFound = errors.New("db: column family not found")

	// ErrColumnFamilyExists is returned when a column family already exists.
	ErrColumnFamilyExists = errors.New("db: column family already exists")

	// ErrInvalidColumnFamilyHandle is returned when a column family handle is invalid.
	ErrInvalidColumnFamilyHandle = errors.New("db: invalid column family handle")

	// ErrCannotDropDefaultCF is returned when trying to drop the default column family.
	ErrCannotDropDefaultCF = errors.New("db: cannot drop default column family")
)
View Source
var (
	ErrDBClosed            = errors.New("db: database is closed")
	ErrNotFound            = errors.New("db: key not found")
	ErrMergeOperatorNotSet = errors.New("db: merge operator not set in options")
	ErrDBExists            = errors.New("db: database already exists")
	ErrDBNotFound          = errors.New("db: database not found")
	ErrCorruption          = errors.New("db: corruption detected")
	ErrInvalidOptions      = errors.New("db: invalid options")
	ErrBackgroundError     = errors.New("db: unrecoverable background error")
	ErrFatal               = logging.ErrFatal // Re-export for convenience
)

Common errors returned by DB operations.

View Source
var (
	// ErrIngestOverlapMemtable is returned when ingesting files overlap with the memtable
	// and allow_blocking_flush is false.
	ErrIngestOverlapMemtable = errors.New("ingest: files overlap with memtable and blocking flush not allowed")

	// ErrIngestInvalidFile is returned when an ingested file is invalid or corrupted.
	ErrIngestInvalidFile = errors.New("ingest: invalid or corrupted SST file")

	// ErrIngestEmptyFile is returned when an ingested file has no entries.
	ErrIngestEmptyFile = errors.New("ingest: SST file has no entries")

	// ErrIngestFilesOverlap is returned when ingested files overlap and global seqno is disabled.
	ErrIngestFilesOverlap = errors.New("ingest: ingested files overlap and allow_global_seqno is false")

	// ErrIngestNotBottommostLevel is returned when fail_if_not_bottommost_level is set
	// but files cannot be placed in the bottommost level.
	ErrIngestNotBottommostLevel = errors.New("ingest: files cannot be placed in bottommost level")
)

Ingestion errors

View Source
var (
	// ErrLockTimeout is returned when a lock request times out.
	ErrLockTimeout = txn.ErrLockTimeout

	// ErrDeadlock is returned when a deadlock is detected.
	ErrDeadlock = txn.ErrDeadlock

	// ErrLockNotHeld is returned when trying to unlock a key not held by the transaction.
	ErrLockNotHeld = txn.ErrLockNotHeld
)

Lock Manager errors - re-exported from internal/txn for public API.

View Source
var (
	// ErrTransactionExpired is returned when a transaction has expired.
	ErrTransactionExpired = errors.New("db: transaction expired")

	// ErrNoSavePoint is returned when trying to rollback to a savepoint that doesn't exist.
	ErrNoSavePoint = errors.New("db: no savepoint to rollback to")

	// ErrTransactionReadOnly is returned when trying to write in a read-only transaction.
	ErrTransactionReadOnly = errors.New("db: transaction is read-only")

	// ErrWriteConflict is returned when a key was modified after the transaction's snapshot.
	ErrWriteConflict = errors.New("db: write conflict - key modified after snapshot")
)

Pessimistic transaction errors

View Source
var (
	// ErrSstWriterNotOpened is returned when trying to use a writer that hasn't been opened.
	ErrSstWriterNotOpened = errors.New("sst: file writer not opened")

	// ErrSstWriterAlreadyOpened is returned when trying to open an already opened writer.
	ErrSstWriterAlreadyOpened = errors.New("sst: file writer already opened")

	// ErrSstWriterAlreadyFinished is returned when trying to add to a finished writer.
	ErrSstWriterAlreadyFinished = errors.New("sst: file writer already finished")

	// ErrSstWriterKeyOutOfOrder is returned when keys are added out of order.
	ErrSstWriterKeyOutOfOrder = errors.New("sst: keys must be added in sorted order")

	// ErrSstWriterEmptyFile is returned when trying to finish a file with no entries.
	ErrSstWriterEmptyFile = errors.New("sst: cannot finish file with no entries")
)

SST File Writer errors

View Source
var (
	// ErrTimestampRequired is returned when a timestamp is required but not provided.
	ErrTimestampRequired = errors.New("db: timestamp required for timestamped database")

	// ErrTimestampNotSupported is returned when timestamps are not supported.
	ErrTimestampNotSupported = errors.New("db: timestamps not supported by comparator")

	// ErrInvalidTimestampSize is returned when the timestamp size is incorrect.
	ErrInvalidTimestampSize = errors.New("db: invalid timestamp size")
)
View Source
var (
	// ErrTransactionConflict is returned when a transaction commit fails due to conflicts.
	ErrTransactionConflict = errors.New("db: transaction conflict")

	// ErrTransactionNotFound is returned when a transaction is not found.
	ErrTransactionNotFound = errors.New("db: transaction not found")

	// ErrTransactionClosed is returned when operating on a closed transaction.
	ErrTransactionClosed = errors.New("db: transaction is closed")
)

Transaction errors

View Source
var (
	// ErrWALNotAvailable is returned when the requested WAL is no longer available.
	ErrWALNotAvailable = errors.New("db: WAL file not available")

	// ErrIteratorNotValid is returned when accessing an invalid iterator.
	ErrIteratorNotValid = errors.New("db: transaction log iterator is not valid")
)
View Source
var (
	ErrWideColumnTooShort = errors.New("wide_column: data too short")
	ErrWideColumnCorrupt  = errors.New("wide_column: corrupt data")
)

Errors for wide column operations

View Source
var (
	// ErrTxnNotPrepared is returned when trying to commit an unprepared transaction.
	ErrTxnNotPrepared = errors.New("db: transaction not prepared")

	// ErrTxnAlreadyPrepared is returned when trying to prepare an already prepared transaction.
	ErrTxnAlreadyPrepared = errors.New("db: transaction already prepared")

	// ErrTxnPrepareConflict is returned when prepare fails due to conflict.
	ErrTxnPrepareConflict = errors.New("db: prepare conflict")
)

Write-prepared transaction errors

View Source
var DefaultLockManagerOptions = txn.DefaultLockManagerOptions

DefaultLockManagerOptions returns default options.

View Source
var (
	// ErrInvalidTimestamp is returned when a timestamp is malformed.
	ErrInvalidTimestamp = errors.New("db: invalid timestamp")
)
View Source
var ErrIteratorInvalid = errors.New("db: iterator is not valid")

ErrIteratorInvalid indicates an operation was attempted on an invalid iterator.

View Source
var (
	// ErrKeyExpired is returned when a key has expired.
	ErrKeyExpired = errors.New("db: key has expired")
)
View Source
var ErrReadOnly = errors.New("db: database is opened in read-only mode")

ErrReadOnly is returned when attempting a write operation on a read-only database.

Functions

func AppendTimestampToKey

func AppendTimestampToKey(key []byte, ts []byte) []byte

AppendTimestampToKey appends a timestamp to a user key.

func DecodeU64Ts

func DecodeU64Ts(ts []byte) (uint64, error)

DecodeU64Ts decodes a uint64 timestamp from a byte slice. Reverses the encoding done by EncodeU64Ts.

func EncodeU64Ts

func EncodeU64Ts(ts uint64) []byte

EncodeU64Ts encodes a uint64 timestamp into a byte slice. The encoding uses big-endian with bitwise inversion so that byte-wise comparison produces DESCENDING order (larger timestamps come first). This is required because SST files use bytewise comparison for seeking.

Example:

  • ts=0 encodes as 0xFFFFFFFFFFFFFFFF (bytewise largest)
  • ts=100 encodes as ^100 (larger than ^200)
  • ts=MAX encodes as 0x0000000000000000 (bytewise smallest)

This ensures that when iterating, entries with larger (newer) timestamps appear before entries with smaller (older) timestamps for the same user key.

func EncodeWideColumns

func EncodeWideColumns(columns WideColumns) ([]byte, error)

EncodeWideColumns encodes wide columns to bytes.

func GetBuffer

func GetBuffer(minSize int) []byte

GetBuffer retrieves a byte slice from the global buffer pool.

func GetLatestOptionsFile

func GetLatestOptionsFile(fs vfs.FS, dbPath string) (string, error)

GetLatestOptionsFile finds the latest OPTIONS file in the database directory.

func IsBlobValue

func IsBlobValue(value []byte) bool

IsBlobValue checks if a value is a blob index (reference to blob file).

func IsFileDeletionsDisabled

func IsFileDeletionsDisabled() bool

IsFileDeletionsDisabled returns true if file deletions are disabled.

func ListColumnFamilies

func ListColumnFamilies(path string, opts *Options) ([]string, error)

ListColumnFamilies returns the list of column family names in the database.

func MaxU64Ts

func MaxU64Ts() []byte

MaxU64Ts returns the encoded maximum uint64 timestamp. With inverted encoding, max timestamp (0xFFFF...) encodes as 0x0000... (bytewise smallest).

func MinU64Ts

func MinU64Ts() []byte

MinU64Ts returns the encoded minimum uint64 timestamp. With inverted encoding, min timestamp (0) encodes as 0xFFFF... (bytewise largest).

func PutBuffer

func PutBuffer(buf []byte)

PutBuffer returns a byte slice to the global buffer pool.

func StripTimestampFromKey

func StripTimestampFromKey(keyWithTS []byte, tsSize int) (userKey, timestamp []byte)

StripTimestampFromKey removes the timestamp from a key. Returns the user key portion and the timestamp.

func WriteOptionsFile

func WriteOptionsFile(fs vfs.FS, dbPath string, opts *Options, fileNum uint64) error

WriteOptionsFile writes the current options to an OPTIONS file.

Types

type AssociativeMergeOperator

type AssociativeMergeOperator interface {
	// Name returns a unique identifier for this merge operator.
	Name() string

	// Merge merges a new value with an existing value.
	// If existingValue is nil, treat it as the identity element for the operation.
	Merge(key []byte, existingValue, value []byte) ([]byte, bool)
}

AssociativeMergeOperator is a simplified interface for associative operations. Use this when merging is associative: Merge(Merge(a, b), c) == Merge(a, Merge(b, c)) Examples: numeric addition, string concatenation, set union

type BackgroundErrorInfo

type BackgroundErrorInfo struct {
	// Reason is the reason for the error.
	Reason BackgroundErrorReason
	// Status is the error that occurred.
	Status error
}

BackgroundErrorInfo contains information about a background error.

type BackgroundErrorReason

type BackgroundErrorReason int

BackgroundErrorReason describes the reason for a background error.

const (
	// BackgroundErrorReasonFlush is for flush errors.
	BackgroundErrorReasonFlush BackgroundErrorReason = iota
	// BackgroundErrorReasonCompaction is for compaction errors.
	BackgroundErrorReasonCompaction
	// BackgroundErrorReasonWriteCallback is for write callback errors.
	BackgroundErrorReasonWriteCallback
	// BackgroundErrorReasonMemTable is for memtable errors.
	BackgroundErrorReasonMemTable
	// BackgroundErrorReasonManifestWrite is for manifest write errors.
	BackgroundErrorReasonManifestWrite
	// BackgroundErrorReasonFlushNoWAL is for flush no-WAL errors.
	BackgroundErrorReasonFlushNoWAL
	// BackgroundErrorReasonManifestWriteNoWAL is for manifest write no-WAL errors.
	BackgroundErrorReasonManifestWriteNoWAL
)

func (BackgroundErrorReason) String

func (r BackgroundErrorReason) String() string

String returns the string representation of the background error reason.

type BackupEngine

type BackupEngine struct {
	// contains filtered or unexported fields
}

BackupEngine manages database backups.

func CreateBackupEngine

func CreateBackupEngine(db DB, backupDir string) (*BackupEngine, error)

CreateBackupEngine creates a BackupEngine for the given database. The backup directory must be different from the database directory.

func (*BackupEngine) Close

func (be *BackupEngine) Close() error

Close closes the backup engine.

func (*BackupEngine) CreateNewBackup

func (be *BackupEngine) CreateNewBackup() (*BackupInfo, error)

CreateNewBackup creates a new backup of the database.

func (*BackupEngine) DeleteBackup

func (be *BackupEngine) DeleteBackup(backupID uint32) error

DeleteBackup deletes a backup.

func (*BackupEngine) GetBackupInfo

func (be *BackupEngine) GetBackupInfo() ([]BackupInfo, error)

GetBackupInfo returns information about all available backups.

func (*BackupEngine) PurgeOldBackups

func (be *BackupEngine) PurgeOldBackups(numToKeep int) (int, error)

PurgeOldBackups keeps the most recent N backups and deletes the rest.

func (*BackupEngine) RestoreDBFromBackup

func (be *BackupEngine) RestoreDBFromBackup(backupID uint32, restoreDir string) error

RestoreDBFromBackup restores the database from a backup. The restore directory should not exist.

type BackupInfo

type BackupInfo struct {
	ID        uint32    `json:"id"`
	Timestamp time.Time `json:"timestamp"`
	Size      int64     `json:"size"`
	NumFiles  int       `json:"num_files"`
}

BackupInfo contains information about a backup.

type BaseCompactionFilter

type BaseCompactionFilter struct{}

BaseCompactionFilter provides a base implementation of CompactionFilter that can be embedded in custom filters.

func (*BaseCompactionFilter) Filter

func (b *BaseCompactionFilter) Filter(level int, key, oldValue []byte) (CompactionFilterDecision, []byte)

Filter default implementation keeps all entries.

func (*BaseCompactionFilter) FilterMergeOperand

func (b *BaseCompactionFilter) FilterMergeOperand(level int, key, operand []byte) CompactionFilterDecision

FilterMergeOperand default implementation keeps all operands.

func (*BaseCompactionFilter) Name

func (b *BaseCompactionFilter) Name() string

Name returns "BaseCompactionFilter".

type BatchResult

type BatchResult struct {
	// Sequence is the sequence number of the first operation in the batch.
	Sequence uint64

	// WriteBatch is the batch of operations.
	WriteBatch *WriteBatch
}

BatchResult contains a write batch and its sequence number.

type BlobDBOptions

type BlobDBOptions struct {
	// Enable enables storing large values in blob files
	Enable bool

	// MinBlobSize is the minimum value size to store in a blob file (bytes)
	MinBlobSize int

	// BlobFileSize is the target size for blob files (bytes)
	BlobFileSize int64

	// BlobCompressionType is the compression algorithm for blob files
	BlobCompressionType CompressionType

	// EnableBlobGC enables garbage collection of unreferenced blobs
	EnableBlobGC bool

	// BlobGCAgeCutoff is the age fraction of files to consider for GC (0.0 to 1.0)
	BlobGCAgeCutoff float64
}

BlobDBOptions configures BlobDB behavior. These are user-facing configuration knobs.

Reference: RocksDB v10.7.5 include/rocksdb/advanced_options.h

func DefaultBlobDBOptions

func DefaultBlobDBOptions() BlobDBOptions

DefaultBlobDBOptions returns sensible defaults for BlobDB.

type BytewiseComparator

type BytewiseComparator struct{}

BytewiseComparator is the default comparator that compares keys lexicographically.

func (BytewiseComparator) Compare

func (c BytewiseComparator) Compare(a, b []byte) int

Compare compares two keys lexicographically.

func (BytewiseComparator) FindShortSuccessor

func (c BytewiseComparator) FindShortSuccessor(a []byte) []byte

FindShortSuccessor finds a short key >= a.

func (BytewiseComparator) FindShortestSeparator

func (c BytewiseComparator) FindShortestSeparator(a, b []byte) []byte

FindShortestSeparator finds a key between a and b.

func (BytewiseComparator) Name

func (c BytewiseComparator) Name() string

Name returns the comparator name.

type BytewiseComparatorWithU64Ts

type BytewiseComparatorWithU64Ts struct{}

BytewiseComparatorWithU64Ts is a comparator that supports uint64 timestamps. It orders keys bytewise, and for the same user key, larger (newer) timestamps come first.

The timestamp encoding uses bitwise inversion (see EncodeU64Ts) so that bytewise comparison automatically produces descending timestamp order. This allows the comparator to use simple bytewise comparison on the entire key (user_key + encoded_timestamp).

Reference: RocksDB v10.7.5

  • util/comparator.cc (BytewiseComparatorWithU64TsImpl)

func (BytewiseComparatorWithU64Ts) Compare

func (c BytewiseComparatorWithU64Ts) Compare(a, b []byte) int

Compare compares two keys with timestamps. Since timestamps are encoded with bitwise inversion (see EncodeU64Ts), bytewise comparison automatically produces the correct ordering: - For different user keys: sorted bytewise - For same user key: larger timestamps come first (due to inverted encoding)

func (BytewiseComparatorWithU64Ts) CompareTimestamp

func (c BytewiseComparatorWithU64Ts) CompareTimestamp(ts1, ts2 []byte) int

CompareTimestamp compares two timestamps (encoded format). Returns < 0 if ts1 < ts2, 0 if equal, > 0 if ts1 > ts2. Note: Timestamps are encoded with bitwise inversion, so we need to decode them first to get the correct comparison result.

func (BytewiseComparatorWithU64Ts) CompareWithoutTimestamp

func (c BytewiseComparatorWithU64Ts) CompareWithoutTimestamp(a, b []byte, aHasTS, bHasTS bool) int

CompareWithoutTimestamp compares two keys ignoring their timestamps.

func (BytewiseComparatorWithU64Ts) FindShortSuccessor

func (c BytewiseComparatorWithU64Ts) FindShortSuccessor(a []byte) []byte

FindShortSuccessor finds a short key >= a.

func (BytewiseComparatorWithU64Ts) FindShortestSeparator

func (c BytewiseComparatorWithU64Ts) FindShortestSeparator(a, b []byte) []byte

FindShortestSeparator finds a key between a and b.

func (BytewiseComparatorWithU64Ts) GetMaxTimestamp

func (c BytewiseComparatorWithU64Ts) GetMaxTimestamp() []byte

GetMaxTimestamp returns the maximum uint64 timestamp.

func (BytewiseComparatorWithU64Ts) GetMinTimestamp

func (c BytewiseComparatorWithU64Ts) GetMinTimestamp() []byte

GetMinTimestamp returns the minimum uint64 timestamp.

func (BytewiseComparatorWithU64Ts) Name

Name returns the comparator name.

func (BytewiseComparatorWithU64Ts) TimestampSize

func (c BytewiseComparatorWithU64Ts) TimestampSize() int

TimestampSize returns the size of the timestamp (8 bytes for uint64).

type CappedPrefixExtractor

type CappedPrefixExtractor struct {
	// contains filtered or unexported fields
}

CappedPrefixExtractor uses min(n, len(key)) bytes as the prefix. All keys are in domain.

func NewCappedPrefixExtractor

func NewCappedPrefixExtractor(capLen int) *CappedPrefixExtractor

NewCappedPrefixExtractor creates a prefix extractor that uses up to n bytes.

func (*CappedPrefixExtractor) InDomain

func (e *CappedPrefixExtractor) InDomain(key []byte) bool

InDomain always returns true for capped prefix extractor.

func (*CappedPrefixExtractor) Name

func (e *CappedPrefixExtractor) Name() string

Name returns the extractor name.

func (*CappedPrefixExtractor) Transform

func (e *CappedPrefixExtractor) Transform(key []byte) []byte

Transform extracts the prefix from the key.

type Checkpoint

type Checkpoint struct {
	// contains filtered or unexported fields
}

Checkpoint provides functionality to create database checkpoints.

func NewCheckpoint

func NewCheckpoint(database DB) (*Checkpoint, error)

NewCheckpoint creates a new Checkpoint instance for the given database.

func (*Checkpoint) CreateCheckpoint

func (cp *Checkpoint) CreateCheckpoint(checkpointDir string, logSizeForFlush uint64) error

CreateCheckpoint creates a checkpoint of the database at the specified path.

The checkpoint will contain: - All SST files (hard-linked or copied) - The MANIFEST file (copied) - The CURRENT file (copied) - WAL files if log_size_for_flush is 0 (meaning flush before checkpoint)

If logSizeForFlush is 0, the memtable will be flushed before creating the checkpoint, resulting in no WAL files in the checkpoint. If logSizeForFlush is > 0, WAL files smaller than this size will be copied. If logSizeForFlush is very large (e.g., math.MaxUint64), all WAL files are copied.

func (*Checkpoint) ExportColumnFamilyCheckpoint

func (cp *Checkpoint) ExportColumnFamilyCheckpoint(
	cfHandle ColumnFamilyHandle,
	exportDir string,
) error

ExportColumnFamilyCheckpoint exports a single column family to a checkpoint. This creates a minimal checkpoint containing only the specified column family's data.

type ChecksumType added in v0.3.4

type ChecksumType = checksum.Type

ChecksumType is an alias for the checksum type.

type ColumnFamilyHandle

type ColumnFamilyHandle interface {
	// ID returns the column family ID.
	ID() uint32

	// Name returns the column family name.
	Name() string

	// IsValid returns true if the handle is still valid (not dropped).
	IsValid() bool
}

ColumnFamilyHandle represents a reference to a column family. It can be passed to DB operations to specify which column family to use.

type ColumnFamilyOptions

type ColumnFamilyOptions struct {
	// Comparator for ordering keys within the column family.
	// If nil, uses the database's default comparator.
	Comparator Comparator

	// WriteBufferSize is the amount of data to build up in memory
	// before converting to a sorted on-disk file.
	WriteBufferSize int
}

ColumnFamilyOptions contains options for creating a column family.

func DefaultColumnFamilyOptions

func DefaultColumnFamilyOptions() ColumnFamilyOptions

DefaultColumnFamilyOptions returns default options for a column family.

type CompactRangeOptions

type CompactRangeOptions struct {
	// ChangeLevel when true, will move compacted files to the minimum level
	// capable of holding the data.
	ChangeLevel bool
	// TargetLevel specifies the target level for the compacted files.
	TargetLevel int
	// ExclusiveManualCompaction when true, only one manual compaction runs at a time.
	ExclusiveManualCompaction bool
}

CompactRangeOptions specifies options for manual compaction.

type CompactionFilter

type CompactionFilter interface {
	// Name returns the name of the compaction filter.
	// This is used for logging and identification.
	Name() string

	// Filter is called for each key-value pair during compaction.
	// Returns the decision (Keep, Remove, or Change) and optionally a new value.
	//
	// Parameters:
	//   - level: The compaction output level
	//   - key: The user key (not internal key)
	//   - oldValue: The current value
	//
	// Returns:
	//   - decision: Whether to keep, remove, or change the entry
	//   - newValue: If decision is FilterChange, this is the new value
	Filter(level int, key, oldValue []byte) (decision CompactionFilterDecision, newValue []byte)

	// FilterMergeOperand is called for merge operands.
	// This allows filtering of individual merge operands.
	// Default implementation returns FilterKeep.
	FilterMergeOperand(level int, key, operand []byte) CompactionFilterDecision
}

CompactionFilter is the interface for custom compaction filters. During compaction, the Filter method is called for each key-value pair, allowing the user to decide whether to keep, remove, or modify the entry.

type CompactionFilterContext

type CompactionFilterContext struct {
	// IsFull is true if this is a full compaction (all levels).
	IsFull bool

	// IsManual is true if this is a manually triggered compaction.
	IsManual bool

	// ColumnFamilyID is the ID of the column family being compacted.
	ColumnFamilyID uint32
}

CompactionFilterContext provides context about the current compaction.

type CompactionFilterDecision

type CompactionFilterDecision int

CompactionFilterDecision represents the decision made by a compaction filter.

const (
	// FilterKeep keeps the key-value pair unchanged.
	FilterKeep CompactionFilterDecision = iota

	// FilterRemove removes the key-value pair from the database.
	FilterRemove

	// FilterChange changes the value of the key-value pair.
	// The new value should be set via the compaction filter context.
	FilterChange
)

type CompactionFilterFactory

type CompactionFilterFactory interface {
	// Name returns the name of the factory.
	Name() string

	// CreateCompactionFilter creates a new compaction filter for a compaction.
	// The context provides information about the current compaction.
	CreateCompactionFilter(context CompactionFilterContext) CompactionFilter
}

CompactionFilterFactory creates compaction filters. A new filter is created for each compaction, allowing filters to maintain state during a single compaction.

type CompactionJobInfo

type CompactionJobInfo struct {
	// CFName is the column family name.
	CFName string
	// Status is the status of the compaction (nil for success).
	Status error
	// ThreadID is the ID of the thread that performed the compaction.
	ThreadID uint64
	// JobID is the unique identifier for this compaction job.
	JobID int
	// BaseInputLevel is the lowest input level.
	BaseInputLevel int
	// OutputLevel is the output level.
	OutputLevel int
	// InputFiles is the list of input file paths.
	InputFiles []string
	// OutputFiles is the list of output file paths.
	OutputFiles []string
	// NumInputRecords is the number of records in input files.
	NumInputRecords uint64
	// NumOutputRecords is the number of records in output files.
	NumOutputRecords uint64
	// NumCorruptKeys is the number of corrupt keys encountered.
	NumCorruptKeys uint64
	// TotalInputBytes is the total bytes read.
	TotalInputBytes uint64
	// TotalOutputBytes is the total bytes written.
	TotalOutputBytes uint64
	// NumInputFiles is the number of input files.
	NumInputFiles int
	// NumOutputFiles is the number of output files.
	NumOutputFiles int
	// IsManualCompaction indicates if this was a manual compaction.
	IsManualCompaction bool
	// CompactionReason is the reason for the compaction.
	CompactionReason CompactionReason
}

CompactionJobInfo contains information about a compaction job.

type CompactionOptions

type CompactionOptions struct {
	OutputLevel           int
	TargetLevel           int
	MaxSubcompactions     uint32
	OutputFilePathID      uint32
	CompressionType       CompressionType
	OutputFileSizeLimit   uint64
	MaxCompactionBytes    uint64
	PenultimateOutputPath bool
}

CompactionOptions for CompactFiles. Reference: RocksDB v10.7.5 include/rocksdb/options.h

type CompactionReason

type CompactionReason int

CompactionReason describes why a compaction was triggered.

const (
	// CompactionReasonUnknown is for unknown reasons.
	CompactionReasonUnknown CompactionReason = iota
	// CompactionReasonLevelL0FilesNum is for L0 file count trigger.
	CompactionReasonLevelL0FilesNum
	// CompactionReasonLevelMaxLevelSize is for level size trigger.
	CompactionReasonLevelMaxLevelSize
	// CompactionReasonManualCompaction is for manual compaction.
	CompactionReasonManualCompaction
	// CompactionReasonFilesMarkedForCompaction is for marked files.
	CompactionReasonFilesMarkedForCompaction
	// CompactionReasonBottomMostLevel is for bottom-most level compaction.
	CompactionReasonBottomMostLevel
	// CompactionReasonTTL is for TTL-based compaction.
	CompactionReasonTTL
	// CompactionReasonFlush is for post-flush compaction.
	CompactionReasonFlush
	// CompactionReasonExternalSSTIngestion is for external SST ingestion.
	CompactionReasonExternalSSTIngestion
	// CompactionReasonPeriodicCompaction is for periodic compaction.
	CompactionReasonPeriodicCompaction
)

func (CompactionReason) String

func (r CompactionReason) String() string

String returns the string representation of the compaction reason.

type CompactionStyle

type CompactionStyle int

CompactionStyle specifies the compaction strategy.

const (
	// CompactionStyleLevel is the default leveled compaction.
	// Files are organized into levels with each level having a size limit.
	// Optimized for read-heavy workloads.
	CompactionStyleLevel CompactionStyle = iota

	// CompactionStyleUniversal (size-tiered) is optimized for write-heavy workloads.
	// All files are kept in L0 and compacted together when size ratio is exceeded.
	// Lower write amplification but higher space amplification.
	CompactionStyleUniversal

	// CompactionStyleFIFO simply deletes the oldest files when the total size
	// exceeds the limit. Optimized for time-series data with no reads of old data.
	CompactionStyleFIFO
)

func (CompactionStyle) String

func (cs CompactionStyle) String() string

String returns the string representation of the compaction style.

type Comparator

type Comparator interface {
	// Compare returns a value < 0 if a < b, 0 if a == b, > 0 if a > b.
	Compare(a, b []byte) int

	// Name returns the name of the comparator.
	Name() string

	// FindShortestSeparator finds a key k such that a <= k < b.
	// This is used to shorten keys in index blocks.
	// If no such key exists, a should be returned unchanged.
	FindShortestSeparator(a, b []byte) []byte

	// FindShortSuccessor finds a short key that is >= a.
	// This is used to shorten keys at the end of an index block.
	FindShortSuccessor(a []byte) []byte
}

Comparator defines a total ordering over keys.

func DefaultComparator

func DefaultComparator() Comparator

DefaultComparator returns the default bytewise comparator.

type CompressionType

type CompressionType = compression.Type

CompressionType is an alias for the compression type.

type CountingEventListener

type CountingEventListener struct {
	NoOpEventListener
	FlushCount      int
	CompactionCount int
	FileCreateCount int
	FileDeleteCount int
	ErrorCount      int
	StallCount      int
	// contains filtered or unexported fields
}

CountingEventListener counts events for testing purposes.

func (*CountingEventListener) OnBackgroundError

func (l *CountingEventListener) OnBackgroundError(info *BackgroundErrorInfo)

func (*CountingEventListener) OnCompactionCompleted

func (l *CountingEventListener) OnCompactionCompleted(info *CompactionJobInfo)

func (*CountingEventListener) OnFlushCompleted

func (l *CountingEventListener) OnFlushCompleted(info *FlushJobInfo)

func (*CountingEventListener) OnStallConditionsChanged

func (l *CountingEventListener) OnStallConditionsChanged(info *WriteStallInfo)

func (*CountingEventListener) OnTableFileCreated

func (l *CountingEventListener) OnTableFileCreated(info *TableFileCreationInfo)

func (*CountingEventListener) OnTableFileDeleted

func (l *CountingEventListener) OnTableFileDeleted(info *TableFileDeletionInfo)

type DB

type DB interface {
	// Put sets the value for the given key in the default column family.
	Put(opts *WriteOptions, key, value []byte) error

	// PutCF sets the value for the given key in the specified column family.
	PutCF(opts *WriteOptions, cf ColumnFamilyHandle, key, value []byte) error

	// Get retrieves the value for the given key from the default column family.
	// Returns ErrNotFound if the key does not exist.
	Get(opts *ReadOptions, key []byte) ([]byte, error)

	// GetCF retrieves the value for the given key from the specified column family.
	GetCF(opts *ReadOptions, cf ColumnFamilyHandle, key []byte) ([]byte, error)

	// MultiGet retrieves multiple values for the given keys.
	// Returns a slice of values in the same order as keys.
	// If a key doesn't exist, the corresponding value is nil and error is ErrNotFound.
	MultiGet(opts *ReadOptions, keys [][]byte) ([][]byte, []error)

	// Delete removes the given key from the default column family.
	Delete(opts *WriteOptions, key []byte) error

	// SingleDelete removes the given key from the default column family.
	// Unlike Delete, SingleDelete is only valid for keys that have been Put exactly once
	// without any Merge operations. If there are multiple Put operations for a key,
	// SingleDelete may not work correctly.
	SingleDelete(opts *WriteOptions, key []byte) error

	// DeleteCF removes the given key from the specified column family.
	DeleteCF(opts *WriteOptions, cf ColumnFamilyHandle, key []byte) error

	// DeleteRange removes all keys in the range [startKey, endKey) from the default column family.
	DeleteRange(opts *WriteOptions, startKey, endKey []byte) error

	// DeleteRangeCF removes all keys in the range [startKey, endKey) from the specified column family.
	DeleteRangeCF(opts *WriteOptions, cf ColumnFamilyHandle, startKey, endKey []byte) error

	// Merge applies a merge operation for the given key in the default column family.
	// The merge operator specified in Options will be used to combine the operand
	// with any existing value during reads and compaction.
	Merge(opts *WriteOptions, key, value []byte) error

	// MergeCF applies a merge operation for the given key in the specified column family.
	MergeCF(opts *WriteOptions, cf ColumnFamilyHandle, key, value []byte) error

	// Write applies a batch of operations atomically.
	Write(opts *WriteOptions, batch *WriteBatch) error

	// NewIterator creates an iterator over the default column family.
	NewIterator(opts *ReadOptions) Iterator

	// NewIteratorCF creates an iterator over the specified column family.
	NewIteratorCF(opts *ReadOptions, cf ColumnFamilyHandle) Iterator

	// GetSnapshot creates a new snapshot of the database.
	GetSnapshot() *Snapshot

	// ReleaseSnapshot releases a previously acquired snapshot.
	ReleaseSnapshot(s *Snapshot)

	// Flush flushes the memtable to disk.
	Flush(opts *FlushOptions) error

	// Close closes the database, releasing all resources.
	Close() error

	// GetProperty returns the value of a database property.
	GetProperty(name string) (string, bool)

	// CreateColumnFamily creates a new column family.
	CreateColumnFamily(opts ColumnFamilyOptions, name string) (ColumnFamilyHandle, error)

	// DropColumnFamily drops the specified column family.
	DropColumnFamily(cf ColumnFamilyHandle) error

	// ListColumnFamilies returns the names of all column families.
	ListColumnFamilies() []string

	// DefaultColumnFamily returns a handle to the default column family.
	DefaultColumnFamily() ColumnFamilyHandle

	// GetColumnFamily returns a handle to the named column family, or nil if not found.
	GetColumnFamily(name string) ColumnFamilyHandle

	// CompactRange manually triggers compaction for the specified key range.
	// If start and end are nil, the entire database is compacted.
	CompactRange(opts *CompactRangeOptions, start, end []byte) error

	// BeginTransaction begins a new optimistic transaction.
	BeginTransaction(opts TransactionOptions, writeOpts *WriteOptions) Transaction

	// IngestExternalFile loads external SST files into the database.
	IngestExternalFile(paths []string, opts IngestExternalFileOptions) error

	// SyncWAL syncs the current WAL to disk, ensuring all data is durable.
	// This is more expensive than FlushWAL(false) but provides stronger durability.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1782-1789
	SyncWAL() error

	// FlushWAL flushes the WAL buffer to the file system.
	// If sync is true, it also syncs the WAL to disk (equivalent to SyncWAL).
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1775-1780
	FlushWAL(sync bool) error

	// GetLatestSequenceNumber returns the sequence number of the most recent transaction.
	// This is useful for tracking database state and replication.
	GetLatestSequenceNumber() uint64

	// GetLiveFiles returns a list of all files in the database except WAL files.
	// The files are relative to the dbname. The manifest file size is returned.
	// If flushMemtable is true, the memtable is flushed before getting files.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1929-1947
	GetLiveFiles(flushMemtable bool) (files []string, manifestFileSize uint64, err error)

	// GetLiveFilesMetaData returns metadata about all live SST files in the database.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1892-1897
	GetLiveFilesMetaData() []LiveFileMetaData

	// DisableFileDeletions prevents file deletions. Call EnableFileDeletions when done.
	// This is useful for making consistent backups.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h
	DisableFileDeletions() error

	// EnableFileDeletions re-enables file deletions after DisableFileDeletions.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h
	EnableFileDeletions() error

	// PauseBackgroundWork pauses all background work (compaction, flush).
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h
	PauseBackgroundWork() error

	// ContinueBackgroundWork resumes background work after PauseBackgroundWork.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h
	ContinueBackgroundWork() error

	// KeyMayExist checks if a key may exist using bloom filters.
	// Returns true if the key may exist, false if it definitely doesn't exist.
	// If value pointer is not nil, the value may be set if found in cache.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1022-1050
	KeyMayExist(opts *ReadOptions, key []byte, value *[]byte) (mayExist bool, valueFound bool)

	// NewIterators creates iterators for multiple column families.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1066-1069
	NewIterators(opts *ReadOptions, cfs []ColumnFamilyHandle) ([]Iterator, error)

	// GetApproximateSizes returns the approximate sizes of key ranges.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1533-1565
	GetApproximateSizes(ranges []Range, flags SizeApproximationFlags) ([]uint64, error)

	// GetOptions returns a copy of the current database options.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1741-1748
	GetOptions() Options

	// GetDBOptions returns a copy of the current database-wide options.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h line 1750
	GetDBOptions() Options

	// SetOptions dynamically changes database options.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1807-1809
	SetOptions(newOptions map[string]string) error

	// SetDBOptions dynamically changes database-wide options.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1810-1812
	SetDBOptions(newOptions map[string]string) error

	// GetIntProperty returns an integer property value.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1366-1368
	GetIntProperty(name string) (uint64, bool)

	// GetMapProperty returns a map property value.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1370-1372
	GetMapProperty(name string) (map[string]string, bool)

	// WaitForCompact waits for all compactions to complete.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1705-1708
	WaitForCompact(opts *WaitForCompactOptions) error

	// LockWAL locks the WAL, preventing new writes.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1791-1800
	LockWAL() error

	// UnlockWAL unlocks the WAL.
	// Reference: RocksDB v10.7.5 include/rocksdb/db.h lines 1801-1806
	UnlockWAL() error
}

DB is the main interface for interacting with the database.

func Open

func Open(path string, opts *Options) (DB, error)

Open opens the database at the specified path.

Example
package main

import (
	"fmt"
	"os"

	"github.com/aalhour/rockyardkv"
)

func main() {
	dir, err := os.MkdirTemp("", "rockyardkv-example-*")
	if err != nil {
		panic(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()

	opts := rockyardkv.DefaultOptions()
	opts.CreateIfMissing = true

	db, err := rockyardkv.Open(dir, opts)
	if err != nil {
		panic(err)
	}
	defer func() { _ = db.Close() }()

	if err := db.Put(rockyardkv.DefaultWriteOptions(), []byte("k"), []byte("v")); err != nil {
		panic(err)
	}

	val, err := db.Get(rockyardkv.DefaultReadOptions(), []byte("k"))
	if err != nil {
		panic(err)
	}

	fmt.Println(string(val))
}
Output:
v

type EventListener

type EventListener interface {
	// OnFlushCompleted is called when a flush job completes.
	OnFlushCompleted(info *FlushJobInfo)

	// OnFlushBegin is called when a flush job begins.
	OnFlushBegin(info *FlushJobInfo)

	// OnCompactionCompleted is called when a compaction job completes.
	OnCompactionCompleted(info *CompactionJobInfo)

	// OnCompactionBegin is called when a compaction job begins.
	OnCompactionBegin(info *CompactionJobInfo)

	// OnTableFileCreated is called when a table file is created.
	OnTableFileCreated(info *TableFileCreationInfo)

	// OnTableFileDeleted is called when a table file is deleted.
	OnTableFileDeleted(info *TableFileDeletionInfo)

	// OnBackgroundError is called when a background error occurs.
	OnBackgroundError(info *BackgroundErrorInfo)

	// OnStallConditionsChanged is called when stall conditions change.
	OnStallConditionsChanged(info *WriteStallInfo)
}

EventListener receives notifications about database events. All callbacks should be thread-safe and non-blocking.

type ExternalSstFileInfo

type ExternalSstFileInfo struct {
	// FilePath is the path to the SST file.
	FilePath string

	// SmallestKey is the smallest user key in the file.
	SmallestKey []byte

	// LargestKey is the largest user key in the file.
	LargestKey []byte

	// SmallestRangeDelKey is the smallest range deletion user key (if any).
	SmallestRangeDelKey []byte

	// LargestRangeDelKey is the largest range deletion user key (if any).
	LargestRangeDelKey []byte

	// SequenceNumber is the sequence number assigned to all keys in the file.
	// For externally created files, this is typically 0.
	SequenceNumber uint64

	// FileSize is the size of the file in bytes.
	FileSize uint64

	// NumEntries is the number of point entries (Put/Delete) in the file.
	NumEntries uint64

	// NumRangeDelEntries is the number of range deletion entries in the file.
	NumRangeDelEntries uint64

	// FileChecksum is the checksum of the file (if computed).
	FileChecksum string

	// Version is the SST file format version.
	Version int32
}

ExternalSstFileInfo contains information about an SST file created by SstFileWriter. This matches the C++ RocksDB ExternalSstFileInfo structure.

type FIFOCompactionOptions

type FIFOCompactionOptions struct {
	// MaxTableFilesSize is the maximum total size before deletion.
	// Default: 1GB
	MaxTableFilesSize uint64

	// TTL is the time-to-live for files before deletion.
	// Default: 0 (disabled)
	TTL time.Duration

	// AllowCompaction allows intra-L0 compaction.
	// Default: false
	AllowCompaction bool
}

FIFOCompactionOptions contains options for FIFO compaction.

func DefaultFIFOCompactionOptions

func DefaultFIFOCompactionOptions() *FIFOCompactionOptions

DefaultFIFOCompactionOptions returns default options.

type FixedPrefixExtractor

type FixedPrefixExtractor struct {
	// contains filtered or unexported fields
}

FixedPrefixExtractor uses the first n bytes of each key as the prefix. Keys shorter than n bytes are out of domain.

func NewFixedPrefixExtractor

func NewFixedPrefixExtractor(prefixLen int) *FixedPrefixExtractor

NewFixedPrefixExtractor creates a prefix extractor that uses the first n bytes.

func (*FixedPrefixExtractor) InDomain

func (e *FixedPrefixExtractor) InDomain(key []byte) bool

InDomain returns true if the key has at least prefixLen bytes.

func (*FixedPrefixExtractor) Name

func (e *FixedPrefixExtractor) Name() string

Name returns the extractor name.

func (*FixedPrefixExtractor) Transform

func (e *FixedPrefixExtractor) Transform(key []byte) []byte

Transform extracts the prefix from the key.

type FlushJobInfo

type FlushJobInfo struct {
	// CFName is the column family name.
	CFName string
	// FilePath is the path to the output SST file.
	FilePath string
	// ThreadID is the ID of the thread that performed the flush.
	ThreadID uint64
	// JobID is the unique identifier for this flush job.
	JobID int
	// TriggeredWritesSlowdown indicates if flush was triggered by write slowdown.
	TriggeredWritesSlowdown bool
	// TriggeredWritesStop indicates if flush was triggered by write stop.
	TriggeredWritesStop bool
	// SmallestSeqno is the smallest sequence number in the flushed file.
	SmallestSeqno uint64
	// LargestSeqno is the largest sequence number in the flushed file.
	LargestSeqno uint64
	// TableProperties contains properties of the flushed SST file.
	TableProperties map[string]string
	// FlushReason is the reason why the flush was triggered.
	FlushReason FlushReason
}

FlushJobInfo contains information about a flush job.

type FlushOptions

type FlushOptions struct {
	// Wait indicates whether to wait for the flush to complete.
	Wait bool

	// AllowWriteStall indicates whether to allow write stalls.
	AllowWriteStall bool
}

FlushOptions contains options for flush operations.

func DefaultFlushOptions

func DefaultFlushOptions() *FlushOptions

DefaultFlushOptions returns FlushOptions with default values.

type FlushReason

type FlushReason int

FlushReason describes why a flush was triggered.

const (
	// FlushReasonOthers is for unspecified reasons.
	FlushReasonOthers FlushReason = iota
	// FlushReasonGetLiveFiles is for GetLiveFiles().
	FlushReasonGetLiveFiles
	// FlushReasonShutDown is for database shutdown.
	FlushReasonShutDown
	// FlushReasonExternalFileIngestion is for external file ingestion.
	FlushReasonExternalFileIngestion
	// FlushReasonManualFlush is for manual flush via Flush().
	FlushReasonManualFlush
	// FlushReasonWriteBufferFull is when write buffer is full.
	FlushReasonWriteBufferFull
	// FlushReasonWriteBufferManager is for write buffer manager.
	FlushReasonWriteBufferManager
	// FlushReasonWALFileFull is when WAL file is full.
	FlushReasonWALFileFull
	// FlushReasonManualCompaction is for manual compaction.
	FlushReasonManualCompaction
	// FlushReasonAutoCompaction is for automatic compaction.
	FlushReasonAutoCompaction
)

func (FlushReason) String

func (r FlushReason) String() string

String returns the string representation of the flush reason.

type GenericRateLimiter

type GenericRateLimiter struct {
	// contains filtered or unexported fields
}

GenericRateLimiter is a token-bucket based rate limiter.

func NewGenericRateLimiter

func NewGenericRateLimiter(opts *RateLimiterOptions) *GenericRateLimiter

NewGenericRateLimiter creates a new rate limiter.

func (*GenericRateLimiter) GetBytesPerSecond

func (rl *GenericRateLimiter) GetBytesPerSecond() int64

GetBytesPerSecond returns the current rate limit.

func (*GenericRateLimiter) GetTotalBytesThrough

func (rl *GenericRateLimiter) GetTotalBytesThrough(priority IOPriority) int64

GetTotalBytesThrough returns total bytes passed through the limiter.

func (*GenericRateLimiter) GetTotalRequests

func (rl *GenericRateLimiter) GetTotalRequests(priority IOPriority) int64

GetTotalRequests returns total request count.

func (*GenericRateLimiter) IsRateLimited

func (rl *GenericRateLimiter) IsRateLimited(priority IOPriority) bool

IsRateLimited returns true if the priority is rate limited.

func (*GenericRateLimiter) Request

func (rl *GenericRateLimiter) Request(bytes int64, priority IOPriority)

Request requests bytes to be written/read.

func (*GenericRateLimiter) SetBytesPerSecond

func (rl *GenericRateLimiter) SetBytesPerSecond(bytesPerSecond int64)

SetBytesPerSecond dynamically sets the rate limit.

type HistogramData

type HistogramData struct {
	Median  float64
	P95     float64
	P99     float64
	Average float64
	StdDev  float64
	Max     float64
	Min     float64
	Count   uint64
	Sum     uint64
}

HistogramData contains histogram statistics.

type HistogramType

type HistogramType int

HistogramType represents different types of histograms.

const (
	// HistogramDBGet is the histogram for db.Get() latency.
	HistogramDBGet HistogramType = iota
	// HistogramDBWrite is the histogram for db.Write() latency.
	HistogramDBWrite
	// HistogramCompactionTime is the histogram for compaction time.
	HistogramCompactionTime
	// HistogramFlushTime is the histogram for flush time.
	HistogramFlushTime
	// HistogramTableSyncMicros is the histogram for table sync time.
	HistogramTableSyncMicros
	// HistogramWALFileSyncMicros is the histogram for WAL sync time.
	HistogramWALFileSyncMicros
	// HistogramManifestFileSyncMicros is the histogram for manifest sync time.
	HistogramManifestFileSyncMicros
	// HistogramTableOpenIOMicros is the histogram for table open I/O time.
	HistogramTableOpenIOMicros
	// HistogramMultiGetIO is the histogram for multiget I/O.
	HistogramMultiGetIO
	// HistogramReadBlockGetMicros is the histogram for block read time.
	HistogramReadBlockGetMicros
	// HistogramWriteStallDuration is the histogram for write stall duration.
	HistogramWriteStallDuration
	// HistogramBytesPerRead is the histogram for bytes per read.
	HistogramBytesPerRead
	// HistogramBytesPerWrite is the histogram for bytes per write.
	HistogramBytesPerWrite
	// HistogramBytesPerMultiGet is the histogram for bytes per multiget.
	HistogramBytesPerMultiGet
	// HistogramBytesCompressed is the histogram for compressed bytes.
	HistogramBytesCompressed
	// HistogramBytesDecompressed is the histogram for decompressed bytes.
	HistogramBytesDecompressed
	// HistogramCompressionTimesNanos is the histogram for compression time.
	HistogramCompressionTimesNanos
	// HistogramDecompressionTimesNanos is the histogram for decompression time.
	HistogramDecompressionTimesNanos
	// HistogramSSTableBatchOpMicros is the histogram for SST batch operations.
	HistogramSSTableBatchOpMicros

	// HistogramEnumMax is the maximum histogram type for sizing arrays.
	HistogramEnumMax
)

func (HistogramType) String

func (h HistogramType) String() string

String returns the name of the histogram type.

type IOPriority

type IOPriority int

IOPriority specifies the priority of I/O operations.

const (
	// IOPriorityLow is for background operations like compaction.
	IOPriorityLow IOPriority = iota
	// IOPriorityHigh is for user reads and writes.
	IOPriorityHigh
	// IOPriorityTotal is the count of priorities.
	IOPriorityTotal
)

type IngestExternalFileOptions

type IngestExternalFileOptions struct {
	// MoveFiles: if true, move the files instead of copying them.
	// The input files will be unlinked after successful ingestion.
	MoveFiles bool

	// SnapshotConsistency: if false, ingested file keys could appear in
	// existing snapshots that were created before the file was ingested.
	SnapshotConsistency bool

	// AllowGlobalSeqNo: enables assigning a global sequence number to each
	// ingested file. If false, we will use the sequence numbers in the
	// ingested file as is.
	AllowGlobalSeqNo bool

	// AllowBlockingFlush: if true, IngestExternalFile() will trigger and
	// block for flushing memtable(s) if there is overlap between ingested
	// files and memtable(s). If false, ingestion will fail if overlap exists.
	AllowBlockingFlush bool

	// IngestBehind: if true, duplicate keys in the file being ingested will
	// be skipped rather than overwriting existing data. All files will be
	// ingested at the bottommost level with seqno=0.
	IngestBehind bool

	// FailIfNotBottommostLevel: if true, ingestion will fail if files cannot
	// be placed in the bottommost level.
	FailIfNotBottommostLevel bool

	// VerifyChecksumsBeforeIngest: if true, verify the checksums of each
	// block of the external SST file before ingestion.
	VerifyChecksumsBeforeIngest bool
}

IngestExternalFileOptions configures the behavior of IngestExternalFile. This matches the C++ RocksDB IngestExternalFileOptions structure.

func DefaultIngestExternalFileOptions

func DefaultIngestExternalFileOptions() IngestExternalFileOptions

DefaultIngestExternalFileOptions returns the default ingestion options.

type Iterator

type Iterator interface {
	// Valid returns true if the iterator is positioned at a valid entry.
	Valid() bool

	// SeekToFirst positions the iterator at the first key.
	SeekToFirst()

	// SeekToLast positions the iterator at the last key.
	SeekToLast()

	// Seek positions the iterator at the first key >= target.
	Seek(target []byte)

	// SeekForPrev positions the iterator at the last key <= target.
	SeekForPrev(target []byte)

	// Next moves the iterator to the next key.
	Next()

	// Prev moves the iterator to the previous key.
	Prev()

	// Key returns the key at the current position.
	// REQUIRES: Valid()
	Key() []byte

	// Value returns the value at the current position.
	// REQUIRES: Valid()
	Value() []byte

	// Error returns any error that has occurred.
	Error() error

	// Close releases resources associated with the iterator.
	Close() error
}

Iterator provides a way to iterate over keys in the database.

type LiveFileMetaData

type LiveFileMetaData struct {
	// Name is the file name (without the directory path).
	Name string

	// Directory is the directory containing the file.
	Directory string

	// FileNumber is the file number.
	FileNumber uint64

	// Size is the file size in bytes.
	Size uint64

	// ColumnFamilyName is the name of the column family this file belongs to.
	ColumnFamilyName string

	// Level is the level at which this file resides.
	Level int

	// SmallestKey is the smallest user key in the file.
	SmallestKey []byte

	// LargestKey is the largest user key in the file.
	LargestKey []byte

	// SmallestSeqno is the smallest sequence number in the file.
	SmallestSeqno uint64

	// LargestSeqno is the largest sequence number in the file.
	LargestSeqno uint64

	// NumEntries is the number of entries in the file.
	NumEntries uint64

	// NumDeletions is the number of deletion entries in the file.
	NumDeletions uint64

	// BeingCompacted is true if the file is currently being compacted.
	BeingCompacted bool
}

LiveFileMetaData describes a live SST file in the database. Reference: RocksDB v10.7.5 include/rocksdb/metadata.h lines 168-172

type LockManagerOptions

type LockManagerOptions = txn.LockManagerOptions

LockManagerOptions configures the lock manager.

type LockType

type LockType = txn.LockType

LockType represents the type of lock.

type Logger

type Logger = logging.Logger

Logger is an alias for the logging.Logger interface. This allows users to pass their own logger implementation.

type MaxOperator

type MaxOperator struct{}

MaxOperator is a merge operator that keeps the maximum value.

func (*MaxOperator) FullMerge

func (o *MaxOperator) FullMerge(key []byte, existingValue []byte, operands [][]byte) ([]byte, bool)

FullMerge returns the maximum of all values.

func (*MaxOperator) Name

func (o *MaxOperator) Name() string

Name returns the name of this merge operator.

func (*MaxOperator) PartialMerge

func (o *MaxOperator) PartialMerge(key []byte, left, right []byte) ([]byte, bool)

PartialMerge returns the maximum of two operands.

type MergeOperator

type MergeOperator interface {
	// Name returns a unique identifier for this merge operator.
	// Used to check compatibility when opening an existing database.
	Name() string

	// FullMerge performs a merge operation.
	//
	// Parameters:
	// - key: The key associated with this merge operation
	// - existingValue: The existing value (nil if key doesn't exist)
	// - operands: List of merge operands to apply, oldest first
	//
	// Returns:
	// - newValue: The result of the merge
	// - ok: Whether the merge succeeded
	//
	// If ok is false, the merge is considered failed and treated as an error.
	FullMerge(key []byte, existingValue []byte, operands [][]byte) (newValue []byte, ok bool)

	// PartialMerge merges two operands into a single operand.
	// This is an optimization that allows combining operands before FullMerge.
	//
	// Parameters:
	// - key: The key associated with this merge operation
	// - leftOperand: The first operand
	// - rightOperand: The second operand
	//
	// Returns:
	// - newOperand: The combined operand
	// - ok: Whether the partial merge succeeded
	//
	// If ok is false, the operands cannot be combined and both must be kept.
	// PartialMerge is optional - returning (nil, false) is always valid.
	PartialMerge(key []byte, leftOperand, rightOperand []byte) (newOperand []byte, ok bool)
}

MergeOperator is the interface for user-defined merge operations.

A MergeOperator specifies the semantics of a merge operation, which only the client knows. It could be numeric addition, list append, string concatenation, or any custom operation.

RocksDB calls the merge operator during: - Get operations (to compute the final value) - Compaction (to combine merge operands) - Iteration (to compute values on the fly)

There are two types of merge operators: 1. AssociativeMergeOperator - for simple operations like addition 2. MergeOperator - for complex operations requiring full control

func WrapAssociativeMergeOperator added in v0.3.4

func WrapAssociativeMergeOperator(op AssociativeMergeOperator) MergeOperator

WrapAssociativeMergeOperator adapts an AssociativeMergeOperator into a MergeOperator.

type NoOpEventListener

type NoOpEventListener struct{}

NoOpEventListener is a default implementation that does nothing. Embed this in your listener if you only want to handle specific events.

func (*NoOpEventListener) OnBackgroundError

func (l *NoOpEventListener) OnBackgroundError(info *BackgroundErrorInfo)

func (*NoOpEventListener) OnCompactionBegin

func (l *NoOpEventListener) OnCompactionBegin(info *CompactionJobInfo)

func (*NoOpEventListener) OnCompactionCompleted

func (l *NoOpEventListener) OnCompactionCompleted(info *CompactionJobInfo)

func (*NoOpEventListener) OnFlushBegin

func (l *NoOpEventListener) OnFlushBegin(info *FlushJobInfo)

func (*NoOpEventListener) OnFlushCompleted

func (l *NoOpEventListener) OnFlushCompleted(info *FlushJobInfo)

func (*NoOpEventListener) OnStallConditionsChanged

func (l *NoOpEventListener) OnStallConditionsChanged(info *WriteStallInfo)

func (*NoOpEventListener) OnTableFileCreated

func (l *NoOpEventListener) OnTableFileCreated(info *TableFileCreationInfo)

func (*NoOpEventListener) OnTableFileDeleted

func (l *NoOpEventListener) OnTableFileDeleted(info *TableFileDeletionInfo)

type NoopPrefixExtractor

type NoopPrefixExtractor struct{}

NoopPrefixExtractor returns the entire key as the prefix. This effectively disables prefix optimization.

func NewNoopPrefixExtractor

func NewNoopPrefixExtractor() *NoopPrefixExtractor

NewNoopPrefixExtractor creates a no-op prefix extractor.

func (*NoopPrefixExtractor) InDomain

func (e *NoopPrefixExtractor) InDomain(key []byte) bool

InDomain always returns true.

func (*NoopPrefixExtractor) Name

func (e *NoopPrefixExtractor) Name() string

Name returns the extractor name.

func (*NoopPrefixExtractor) Transform

func (e *NoopPrefixExtractor) Transform(key []byte) []byte

Transform returns the entire key.

type Options

type Options struct {
	// CreateIfMissing causes Open to create the database if it does not exist.
	CreateIfMissing bool

	// ErrorIfExists causes Open to return an error if the database already exists.
	ErrorIfExists bool

	// ParanoidChecks enables additional checks for data integrity.
	ParanoidChecks bool

	// FS is the filesystem implementation to use.
	// If nil, the OS filesystem is used.
	FS vfs.FS

	// Comparator defines the order of keys in the database.
	// If nil, a default bytewise comparator is used.
	Comparator Comparator

	// WriteBufferSize is the size of a single memtable.
	// Default: 64MB
	WriteBufferSize int

	// MaxWriteBufferNumber is the maximum number of memtables to keep in memory.
	// Default: 2
	MaxWriteBufferNumber int

	// MaxOpenFiles is the maximum number of SST files to keep open.
	// Default: 1000
	MaxOpenFiles int

	// BlockSize is the approximate size of data blocks within SST files.
	// Default: 4KB
	BlockSize int

	// BlockRestartInterval is how often to create restart points in blocks.
	// Default: 16
	BlockRestartInterval int

	// ChecksumType specifies the checksum algorithm for SST files.
	// Default: CRC32C
	ChecksumType ChecksumType

	// FormatVersion is the SST file format version.
	// Default: 3
	FormatVersion uint32

	// MergeOperator specifies the merge operator for merge operations.
	// If nil, Merge operations will return an error.
	MergeOperator MergeOperator

	// PrefixExtractor extracts prefixes from keys for prefix-based operations.
	// When set, bloom filters are built for prefixes instead of whole keys,
	// and prefix seek can be used for efficient iteration within a prefix.
	// If nil, no prefix optimization is used.
	PrefixExtractor PrefixExtractor

	// Level0FileNumCompactionTrigger is the number of files in level-0 that
	// triggers compaction to level-1.
	// Default: 4
	Level0FileNumCompactionTrigger int

	// MaxBytesForLevelBase is the maximum total data size for level-1.
	// Default: 256MB
	MaxBytesForLevelBase int64

	// BloomFilterBitsPerKey is the number of bits per key for bloom filters.
	// 0 disables bloom filters. Default: 10
	BloomFilterBitsPerKey int

	// Level0SlowdownWritesTrigger is the number of L0 files that triggers
	// write slowdown. When L0 file count exceeds this, writes are delayed.
	// Default: 20
	Level0SlowdownWritesTrigger int

	// Level0StopWritesTrigger is the number of L0 files that stops writes.
	// When L0 file count exceeds this, all writes are blocked until
	// compaction reduces the count.
	// Default: 36
	Level0StopWritesTrigger int

	// DisableAutoCompactions disables background compaction.
	// When true, no write stalling occurs based on L0 file count.
	// Default: false
	DisableAutoCompactions bool

	// CompactionFilter is called for each key-value pair during compaction.
	// It can filter out or modify entries during compaction.
	// If nil, no filtering is applied.
	CompactionFilter CompactionFilter

	// CompactionFilterFactory creates a new CompactionFilter for each compaction.
	// This takes precedence over CompactionFilter if both are set.
	// If nil, CompactionFilter is used directly.
	CompactionFilterFactory CompactionFilterFactory

	// CompactionStyle specifies the compaction strategy.
	// Default: CompactionStyleLevel
	CompactionStyle CompactionStyle

	// UniversalCompactionOptions contains options for universal compaction.
	// Only used when CompactionStyle is CompactionStyleUniversal.
	UniversalCompactionOptions *UniversalCompactionOptions

	// FIFOCompactionOptions contains options for FIFO compaction.
	// Only used when CompactionStyle is CompactionStyleFIFO.
	FIFOCompactionOptions *FIFOCompactionOptions

	// RateLimiter controls the rate of I/O operations.
	// If nil, no rate limiting is applied.
	RateLimiter RateLimiter

	// Compression specifies the compression algorithm for SST blocks.
	// Default: NoCompression
	Compression CompressionType

	// MaxSubcompactions is the maximum number of subcompactions per compaction job.
	// Subcompactions allow parallel compaction within a single job by dividing
	// the key range. Higher values can improve compaction throughput on multi-core
	// systems but increase memory usage.
	// Default: 1 (no parallel subcompaction)
	MaxSubcompactions int

	// UseDirectReads enables O_DIRECT for reading data.
	// This bypasses the OS page cache and reads directly from disk.
	// Beneficial for reducing memory pressure and cache pollution.
	// Requires aligned buffers and may not be supported on all platforms.
	// Reference: RocksDB v10.7.5 include/rocksdb/options.h line 1022-1024
	// Default: false
	UseDirectReads bool

	// UseDirectIOForFlushAndCompaction enables O_DIRECT for background
	// flush and compaction writes. This bypasses the OS page cache.
	// Reference: RocksDB v10.7.5 include/rocksdb/options.h line 1026-1028
	// Default: false
	UseDirectIOForFlushAndCompaction bool

	// Logger is the logger for database operations.
	// If nil, a default logger writing to stderr is used.
	Logger Logger
}

Options contains all configuration options for opening a database.

func DefaultOptions

func DefaultOptions() *Options

DefaultOptions returns a new Options with default values.

type PessimisticTransaction

type PessimisticTransaction struct {
	// contains filtered or unexported fields
}

PessimisticTransaction implements a transaction with pessimistic concurrency control. It uses Two-Phase Locking (2PL) to ensure serializability: - Growing phase: locks are acquired but never released - Shrinking phase (after commit/rollback): all locks are released

func (*PessimisticTransaction) Commit

func (txn *PessimisticTransaction) Commit() error

Commit applies the transaction and releases all locks.

func (*PessimisticTransaction) Delete

func (txn *PessimisticTransaction) Delete(key []byte) error

Delete acquires an exclusive lock and removes the key.

func (*PessimisticTransaction) DeleteCF

func (txn *PessimisticTransaction) DeleteCF(cf ColumnFamilyHandle, key []byte) error

DeleteCF acquires an exclusive lock and removes the key from the specified column family.

func (*PessimisticTransaction) Get

func (txn *PessimisticTransaction) Get(key []byte) ([]byte, error)

Get retrieves the value for the given key. This does NOT acquire a lock (use GetForUpdate for that).

func (*PessimisticTransaction) GetCF

func (txn *PessimisticTransaction) GetCF(cf ColumnFamilyHandle, key []byte) ([]byte, error)

GetCF retrieves the value from the specified column family.

func (*PessimisticTransaction) GetForUpdate

func (txn *PessimisticTransaction) GetForUpdate(key []byte, exclusive bool) ([]byte, error)

GetForUpdate acquires a lock and retrieves the value for the given key. This is the key method for pessimistic concurrency control.

func (*PessimisticTransaction) GetForUpdateCF

func (txn *PessimisticTransaction) GetForUpdateCF(cf ColumnFamilyHandle, key []byte, exclusive bool) ([]byte, error)

GetForUpdateCF acquires a lock and retrieves the value from the specified column family.

func (*PessimisticTransaction) GetNumLocks

func (txn *PessimisticTransaction) GetNumLocks() int

GetNumLocks returns the number of locks held by this transaction.

func (*PessimisticTransaction) GetSnapshot

func (txn *PessimisticTransaction) GetSnapshot() *Snapshot

GetSnapshot returns the transaction's snapshot.

func (*PessimisticTransaction) GetWriteBatchSize

func (txn *PessimisticTransaction) GetWriteBatchSize() uint32

GetWriteBatchSize returns the number of entries in the write batch.

func (*PessimisticTransaction) ID

func (txn *PessimisticTransaction) ID() uint64

ID returns the transaction ID.

func (*PessimisticTransaction) IsExpired

func (txn *PessimisticTransaction) IsExpired() bool

IsExpired returns true if the transaction has expired.

func (*PessimisticTransaction) PopSavePoint

func (txn *PessimisticTransaction) PopSavePoint() error

PopSavePoint removes the last savepoint without rolling back.

func (*PessimisticTransaction) Put

func (txn *PessimisticTransaction) Put(key, value []byte) error

Put acquires an exclusive lock and sets the value for the given key.

func (*PessimisticTransaction) PutCF

func (txn *PessimisticTransaction) PutCF(cf ColumnFamilyHandle, key, value []byte) error

PutCF acquires an exclusive lock and sets the value in the specified column family.

func (*PessimisticTransaction) Rollback

func (txn *PessimisticTransaction) Rollback() error

Rollback discards the transaction and releases all locks.

func (*PessimisticTransaction) RollbackToSavePoint

func (txn *PessimisticTransaction) RollbackToSavePoint() error

RollbackToSavePoint rolls back to the last savepoint.

func (*PessimisticTransaction) SetSavePoint

func (txn *PessimisticTransaction) SetSavePoint() error

SetSavePoint creates a savepoint at the current state.

func (*PessimisticTransaction) SetSnapshot

func (txn *PessimisticTransaction) SetSnapshot()

SetSnapshot sets the transaction's snapshot to the current database state.

type PessimisticTransactionOptions

type PessimisticTransactionOptions struct {
	// SetSnapshot determines if the transaction should set a snapshot at creation.
	SetSnapshot bool

	// LockTimeout is the timeout for acquiring locks.
	LockTimeout time.Duration

	// DeadlockDetect enables deadlock detection.
	DeadlockDetect bool

	// Expiration is the transaction expiration time (0 = no expiration).
	Expiration time.Duration

	// ReadOnly makes the transaction read-only (no writes allowed).
	ReadOnly bool
}

PessimisticTransactionOptions configures a pessimistic transaction.

func DefaultPessimisticTransactionOptions

func DefaultPessimisticTransactionOptions() PessimisticTransactionOptions

DefaultPessimisticTransactionOptions returns default options.

type PinnableWideColumns

type PinnableWideColumns struct {
	Columns WideColumns
	// contains filtered or unexported fields
}

PinnableWideColumns wraps WideColumns with a reference to underlying storage. Similar to RocksDB's PinnableWideColumns.

func (*PinnableWideColumns) Release

func (p *PinnableWideColumns) Release()

Release releases the pinned memory.

type PrefixExtractor

type PrefixExtractor interface {
	// Name returns a unique identifier for this prefix extractor.
	// The name is stored in SST files and used for compatibility checks.
	Name() string

	// Transform extracts the prefix from the given key.
	// The returned slice may reference the input key's memory.
	// REQUIRES: InDomain(key) == true
	Transform(key []byte) []byte

	// InDomain returns true if the key has a valid prefix.
	// If false, the key is considered "out of domain" and prefix bloom
	// filters will not be used for it.
	InDomain(key []byte) bool
}

PrefixExtractor extracts prefixes from keys for prefix-based operations. This is equivalent to RocksDB's SliceTransform interface.

IMPORTANT: Together PrefixExtractor and Comparator must satisfy:

If Compare(k1, k2) <= 0 and Compare(k2, k3) <= 0 and
   InDomain(k1) and InDomain(k3) and Transform(k1) == Transform(k3),
Then InDomain(k2) and Transform(k2) == Transform(k1)

In other words, all keys with the same prefix must be contiguous by comparator order.

type Range

type Range struct {
	Start []byte
	Limit []byte
}

Range represents a key range for size approximation. Reference: RocksDB v10.7.5 include/rocksdb/db.h

type RateLimiter

type RateLimiter interface {
	// Request requests bytes to be written/read.
	// It blocks until enough quota is available.
	Request(bytes int64, priority IOPriority)

	// SetBytesPerSecond dynamically sets the rate limit.
	SetBytesPerSecond(bytesPerSecond int64)

	// GetBytesPerSecond returns the current rate limit.
	GetBytesPerSecond() int64

	// GetTotalBytesThrough returns total bytes passed through the limiter.
	GetTotalBytesThrough(priority IOPriority) int64

	// GetTotalRequests returns total request count.
	GetTotalRequests(priority IOPriority) int64

	// IsRateLimited returns true if the priority is rate limited.
	IsRateLimited(priority IOPriority) bool
}

RateLimiter controls the rate of I/O operations.

func NewRateLimiter

func NewRateLimiter(bytesPerSecond int64) RateLimiter

NewRateLimiter creates a rate limiter with the specified bytes per second. This is a convenience function.

type RateLimiterMode

type RateLimiterMode int

RateLimiterMode specifies when rate limiting should be applied.

const (
	// RateLimiterModeReadsOnly applies rate limiting only to reads.
	RateLimiterModeReadsOnly RateLimiterMode = iota
	// RateLimiterModeWritesOnly applies rate limiting only to writes (compaction, flush).
	RateLimiterModeWritesOnly
	// RateLimiterModeAllIO applies rate limiting to all I/O.
	RateLimiterModeAllIO
)

type RateLimiterOptions

type RateLimiterOptions struct {
	// BytesPerSecond is the maximum rate in bytes per second.
	BytesPerSecond int64

	// RefillPeriod is how often to refill the token bucket.
	// Default: 100ms
	RefillPeriod time.Duration

	// Fairness controls how aggressively high-priority requests preempt low-priority.
	// Range: 1-100, lower = more aggressive. Default: 10
	Fairness int64

	// Mode specifies what I/O to rate limit.
	Mode RateLimiterMode
}

RateLimiterOptions contains options for creating a rate limiter.

func DefaultRateLimiterOptions

func DefaultRateLimiterOptions() *RateLimiterOptions

DefaultRateLimiterOptions returns default options.

type ReadOnlyDB added in v0.3.4

type ReadOnlyDB interface {
	DB

	// NewCheckpoint returns a checkpoint helper bound to the same underlying DB.
	NewCheckpoint() *Checkpoint
}

ReadOnlyDB is a DB opened in read-only mode.

Contract: ReadOnlyDB never allows write operations; write methods return ErrReadOnly.

func OpenForReadOnly

func OpenForReadOnly(path string, opts *Options, errorIfWALExists bool) (ReadOnlyDB, error)

OpenForReadOnly opens a database in read-only mode. If errorIfWALExists is true, an error is returned if there are WAL files that would need to be replayed (indicating unclean shutdown).

type ReadOptions

type ReadOptions struct {
	// VerifyChecksums enables checksum verification when reading.
	VerifyChecksums bool

	// FillCache indicates whether to fill the block cache on reads.
	FillCache bool

	// Snapshot provides a consistent view of the database.
	// If nil, the most recent state is used.
	Snapshot *Snapshot

	// Timestamp specifies the timestamp for reading.
	// Read will return the latest data visible to the specified timestamp.
	// All timestamps of the same database must be of the same length.
	// For iterators, IterStartTimestamp is the lower bound (older) and
	// Timestamp serves as the upper bound.
	// If nil, timestamps are not used.
	//
	// Reference: RocksDB v10.7.5 include/rocksdb/options.h (ReadOptions::timestamp)
	Timestamp []byte

	// IterStartTimestamp is the lower bound (older) timestamp for iterators.
	// Versions of the same record that fall in the timestamp range
	// [IterStartTimestamp, Timestamp] will be returned.
	// If nil, only the most recent version visible to Timestamp is returned.
	//
	// Reference: RocksDB v10.7.5 include/rocksdb/options.h (ReadOptions::iter_start_ts)
	IterStartTimestamp []byte

	// TotalOrderSeek enables total order seek.
	// When true, prefix bloom filters are bypassed and all keys are considered.
	// When false (default), prefix seek optimization is used if a prefix extractor
	// is configured.
	TotalOrderSeek bool

	// PrefixSameAsStart optimizes iteration when the user knows the iteration
	// will stay within the same prefix.
	// When true, the iterator may skip to the next data block if it determines
	// all keys in the current block have a different prefix.
	PrefixSameAsStart bool

	// IterateUpperBound sets an upper bound for iteration.
	// The iterator will stop before any key >= this bound.
	// This can be used with prefix seek to efficiently limit iteration.
	IterateUpperBound []byte

	// IterateLowerBound sets a lower bound for iteration.
	// The iterator will skip any key < this bound.
	IterateLowerBound []byte
}

ReadOptions contains options for read operations.

func DefaultReadOptions

func DefaultReadOptions() *ReadOptions

DefaultReadOptions returns ReadOptions with default values.

type RecoveredPreparedTxn

type RecoveredPreparedTxn struct {
	Name       string
	PrepareSeq uint64
	Keys       [][]byte // Keys that were written in this transaction
}

RecoveredPreparedTxn represents a prepared transaction that was found during recovery but was neither committed nor rolled back.

type RemoveByPrefixFilter

type RemoveByPrefixFilter struct {
	BaseCompactionFilter
	Prefix []byte
}

RemoveByPrefixFilter removes keys with a specific prefix.

func (*RemoveByPrefixFilter) Filter

func (f *RemoveByPrefixFilter) Filter(level int, key, oldValue []byte) (CompactionFilterDecision, []byte)

Filter removes keys that have the specified prefix.

func (*RemoveByPrefixFilter) Name

func (f *RemoveByPrefixFilter) Name() string

Name returns the filter name.

type RemoveByRangeFilter

type RemoveByRangeFilter struct {
	BaseCompactionFilter
	StartKey []byte // Inclusive
	EndKey   []byte // Exclusive
}

RemoveByRangeFilter removes keys within a specific range.

func (*RemoveByRangeFilter) Filter

func (f *RemoveByRangeFilter) Filter(level int, key, oldValue []byte) (CompactionFilterDecision, []byte)

Filter removes keys within the specified range.

func (*RemoveByRangeFilter) Name

func (f *RemoveByRangeFilter) Name() string

Name returns the filter name.

type ReplicationDB added in v0.3.4

type ReplicationDB interface {
	DB

	// GetUpdatesSince returns an iterator positioned at the first write batch whose sequence number is >= seqNumber.
	GetUpdatesSince(seqNumber uint64, readOpts TransactionLogIteratorReadOptions) (*TransactionLogIterator, error)

	// GetSortedWalFiles returns WAL files sorted by log number.
	GetSortedWalFiles() ([]WalFile, error)
}

ReplicationDB exposes WAL inspection and transaction-log iteration APIs.

Contract: Implementations provide a stable view of WAL files and batches, suitable for replication or incremental backup.

type ReverseBytewiseComparatorWithU64Ts

type ReverseBytewiseComparatorWithU64Ts struct{}

ReverseBytewiseComparatorWithU64Ts is the reverse ordering version. User keys are compared in reverse bytewise order, but timestamps still use descending order (larger timestamps come first for same user key).

Reference: RocksDB v10.7.5

  • util/comparator.cc (ReverseBytewiseComparatorWithU64TsImpl)

func (ReverseBytewiseComparatorWithU64Ts) Compare

func (c ReverseBytewiseComparatorWithU64Ts) Compare(a, b []byte) int

Compare compares two keys with timestamps in reverse order. User keys are compared in reverse bytewise order. With inverted timestamp encoding, timestamps are automatically in descending order.

func (ReverseBytewiseComparatorWithU64Ts) CompareTimestamp

func (c ReverseBytewiseComparatorWithU64Ts) CompareTimestamp(ts1, ts2 []byte) int

CompareTimestamp compares two timestamps (encoded format). Returns < 0 if ts1 < ts2, 0 if equal, > 0 if ts1 > ts2.

func (ReverseBytewiseComparatorWithU64Ts) CompareWithoutTimestamp

func (c ReverseBytewiseComparatorWithU64Ts) CompareWithoutTimestamp(a, b []byte, aHasTS, bHasTS bool) int

CompareWithoutTimestamp compares two keys ignoring timestamps in reverse order.

func (ReverseBytewiseComparatorWithU64Ts) FindShortSuccessor

func (c ReverseBytewiseComparatorWithU64Ts) FindShortSuccessor(a []byte) []byte

FindShortSuccessor is not typically used for reverse comparator.

func (ReverseBytewiseComparatorWithU64Ts) FindShortestSeparator

func (c ReverseBytewiseComparatorWithU64Ts) FindShortestSeparator(a, b []byte) []byte

FindShortestSeparator is not typically used for reverse comparator.

func (ReverseBytewiseComparatorWithU64Ts) GetMaxTimestamp

func (c ReverseBytewiseComparatorWithU64Ts) GetMaxTimestamp() []byte

GetMaxTimestamp returns the maximum uint64 timestamp.

func (ReverseBytewiseComparatorWithU64Ts) GetMinTimestamp

func (c ReverseBytewiseComparatorWithU64Ts) GetMinTimestamp() []byte

GetMinTimestamp returns the minimum uint64 timestamp.

func (ReverseBytewiseComparatorWithU64Ts) Name

Name returns the comparator name.

func (ReverseBytewiseComparatorWithU64Ts) TimestampSize

func (c ReverseBytewiseComparatorWithU64Ts) TimestampSize() int

TimestampSize returns the size of the timestamp.

type SecondaryDB added in v0.3.4

type SecondaryDB interface {
	DB

	// TryCatchUpWithPrimary refreshes the secondary view by reading new MANIFEST records.
	TryCatchUpWithPrimary() error

	// NewCheckpoint returns a checkpoint helper bound to the same underlying DB.
	NewCheckpoint() *Checkpoint
}

SecondaryDB is a DB opened as a secondary instance of a primary.

Contract: SecondaryDB never allows write operations; write methods return ErrReadOnly.

func OpenAsSecondary

func OpenAsSecondary(primaryPath, secondaryPath string, opts *Options) (SecondaryDB, error)

OpenAsSecondary opens a database as a secondary instance. The secondary can read data from the primary but cannot write. primaryPath is the path to the primary database directory. secondaryPath is an optional path for the secondary's local state.

type SizeApproximationFlags

type SizeApproximationFlags uint8

SizeApproximationFlags controls what is included in size estimates. Reference: RocksDB v10.7.5 include/rocksdb/db.h

const (
	// SizeApproximationNone includes nothing.
	SizeApproximationNone SizeApproximationFlags = 0
	// SizeApproximationIncludeMemtables includes memtable sizes.
	SizeApproximationIncludeMemtables SizeApproximationFlags = 1 << 0
	// SizeApproximationIncludeFiles includes SST file sizes.
	SizeApproximationIncludeFiles SizeApproximationFlags = 1 << 1
)

type SizeApproximationOptions

type SizeApproximationOptions struct {
	IncludeMemtables bool
	IncludeFiles     bool
}

SizeApproximationOptions controls size approximation behavior. Reference: RocksDB v10.7.5 include/rocksdb/db.h

type Snapshot

type Snapshot struct {
	// contains filtered or unexported fields
}

Snapshot provides a consistent read view of the database. The contents of a snapshot are guaranteed to be consistent.

func (*Snapshot) Release

func (s *Snapshot) Release()

Release releases the snapshot. After calling Release, the snapshot should not be used.

func (*Snapshot) Sequence

func (s *Snapshot) Sequence() uint64

Sequence returns the sequence number at which this snapshot was taken.

type SstFileWriter

type SstFileWriter struct {
	// contains filtered or unexported fields
}

SstFileWriter creates SST files that can be ingested into a database. All keys in files generated by SstFileWriter have sequence number = 0.

This class is NOT thread-safe.

Usage:

writer, _ := NewSstFileWriter(opts)
writer.Open("/path/to/file.sst")
writer.Put(key1, value1)
writer.Put(key2, value2)
info, _ := writer.Finish()

Keys MUST be added in sorted order according to the comparator.

func NewSstFileWriter

func NewSstFileWriter(opts SstFileWriterOptions) *SstFileWriter

NewSstFileWriter creates a new SstFileWriter with the given options.

func (*SstFileWriter) Abandon

func (w *SstFileWriter) Abandon() error

Abandon abandons the current file and cleans up resources.

func (*SstFileWriter) Delete

func (w *SstFileWriter) Delete(key []byte) error

Delete adds a deletion marker for the key to the SST file. Keys must be added in sorted order according to the comparator.

func (*SstFileWriter) DeleteRange

func (w *SstFileWriter) DeleteRange(startKey, endKey []byte) error

DeleteRange adds a range deletion [startKey, endKey) to the SST file. Range deletions are stored in a separate meta-block.

func (*SstFileWriter) FileSize

func (w *SstFileWriter) FileSize() uint64

FileSize returns the current file size in bytes.

func (*SstFileWriter) Finish

func (w *SstFileWriter) Finish() (*ExternalSstFileInfo, error)

Finish completes the SST file and returns information about it. After calling Finish, the writer cannot be used again without calling Open.

func (*SstFileWriter) Merge

func (w *SstFileWriter) Merge(key, value []byte) error

Merge adds a merge operand for the key to the SST file. Keys must be added in sorted order according to the comparator.

func (*SstFileWriter) Open

func (w *SstFileWriter) Open(filePath string) error

Open prepares the writer to create an SST file at the given path.

func (*SstFileWriter) Put

func (w *SstFileWriter) Put(key, value []byte) error

Put adds a key-value pair to the SST file. Keys must be added in sorted order according to the comparator.

type SstFileWriterOptions

type SstFileWriterOptions struct {
	// Comparator for key ordering. If nil, uses bytewise comparator.
	Comparator Comparator

	// Compression type for the SST file.
	Compression CompressionType

	// BlockSize is the target size for data blocks.
	BlockSize int

	// BlockRestartInterval is the number of keys between restart points.
	BlockRestartInterval int

	// FilterBitsPerKey for creating bloom filters. 0 means no filter.
	FilterBitsPerKey int

	// FormatVersion is the SST file format version.
	FormatVersion uint32
}

SstFileWriterOptions configures the SstFileWriter.

func DefaultSstFileWriterOptions

func DefaultSstFileWriterOptions() SstFileWriterOptions

DefaultSstFileWriterOptions returns default options for SstFileWriter.

type Statistics

type Statistics interface {
	// GetTickerCount returns the current value of a ticker.
	GetTickerCount(tickerType TickerType) uint64

	// RecordTick increments a ticker by count.
	RecordTick(tickerType TickerType, count uint64)

	// SetTickerCount sets the ticker to a specific value.
	SetTickerCount(tickerType TickerType, count uint64)

	// GetHistogramData returns histogram statistics.
	GetHistogramData(histogramType HistogramType) HistogramData

	// MeasureTime records a value to a histogram.
	MeasureTime(histogramType HistogramType, value uint64)

	// Reset clears all statistics.
	Reset()

	// String returns a formatted string of all statistics.
	String() string
}

Statistics collects and reports database metrics.

func NewStatistics

func NewStatistics() Statistics

NewStatistics creates a new Statistics instance.

type StringAppendOperator

type StringAppendOperator struct {
	Delimiter string
}

StringAppendOperator is a merge operator that concatenates strings with a delimiter.

func (*StringAppendOperator) FullMerge

func (o *StringAppendOperator) FullMerge(key []byte, existingValue []byte, operands [][]byte) ([]byte, bool)

FullMerge concatenates all operands with the delimiter.

func (*StringAppendOperator) Name

func (o *StringAppendOperator) Name() string

Name returns the name of this merge operator.

func (*StringAppendOperator) PartialMerge

func (o *StringAppendOperator) PartialMerge(key []byte, left, right []byte) ([]byte, bool)

PartialMerge concatenates two operands with the delimiter.

type TTLCompactionFilter

type TTLCompactionFilter struct {
	BaseCompactionFilter
	TTL time.Duration
}

TTLCompactionFilter removes expired keys during compaction.

func NewTTLCompactionFilter

func NewTTLCompactionFilter(ttl time.Duration) *TTLCompactionFilter

NewTTLCompactionFilter creates a new TTL compaction filter.

func (*TTLCompactionFilter) Filter

func (f *TTLCompactionFilter) Filter(level int, key, oldValue []byte) (CompactionFilterDecision, []byte)

Filter removes expired entries during compaction.

func (*TTLCompactionFilter) Name

func (f *TTLCompactionFilter) Name() string

Name returns the filter name.

type TTLCompactionFilterFactory

type TTLCompactionFilterFactory struct {
	TTL time.Duration
}

TTLCompactionFilterFactory creates TTL compaction filters.

func (*TTLCompactionFilterFactory) CreateCompactionFilter

func (f *TTLCompactionFilterFactory) CreateCompactionFilter(context CompactionFilterContext) CompactionFilter

CreateCompactionFilter creates a new TTL compaction filter.

func (*TTLCompactionFilterFactory) Name

Name returns the factory name.

type TTLDB

type TTLDB struct {
	// contains filtered or unexported fields
}

TTLDB wraps a database with TTL support. Values are automatically timestamped on write and expired entries are filtered on read and removed during compaction.

func OpenWithTTL

func OpenWithTTL(path string, opts *Options, ttl time.Duration) (*TTLDB, error)

OpenWithTTL opens a database with TTL support. The TTL duration specifies how long entries remain valid.

func (*TTLDB) Close

func (t *TTLDB) Close() error

Close closes the database.

func (*TTLDB) Delete

func (t *TTLDB) Delete(opts *WriteOptions, key []byte) error

Delete removes a key.

func (*TTLDB) Flush

func (t *TTLDB) Flush(opts *FlushOptions) error

Flush flushes the memtable.

func (*TTLDB) Get

func (t *TTLDB) Get(opts *ReadOptions, key []byte) ([]byte, error)

Get retrieves a value, returning ErrNotFound if expired.

func (*TTLDB) GetSnapshot

func (t *TTLDB) GetSnapshot() *Snapshot

GetSnapshot returns a snapshot.

func (*TTLDB) NewIterator

func (t *TTLDB) NewIterator(opts *ReadOptions) Iterator

NewIterator returns a TTL-aware iterator.

func (*TTLDB) Put

func (t *TTLDB) Put(opts *WriteOptions, key, value []byte) error

Put stores a key-value pair with TTL timestamp.

func (*TTLDB) PutWithExpiry

func (t *TTLDB) PutWithExpiry(opts *WriteOptions, key, value []byte, creationTime time.Time) error

PutWithExpiry stores a key-value pair with a specific creation time.

func (*TTLDB) ReleaseSnapshot

func (t *TTLDB) ReleaseSnapshot(snapshot *Snapshot)

ReleaseSnapshot releases a snapshot.

func (*TTLDB) SetTTL

func (t *TTLDB) SetTTL(ttl time.Duration)

SetTTL updates the TTL duration.

func (*TTLDB) Underlying

func (t *TTLDB) Underlying() DB

Underlying returns the underlying database.

type TableFileCreationInfo

type TableFileCreationInfo struct {
	// DBName is the database name.
	DBName string
	// CFName is the column family name.
	CFName string
	// FilePath is the path to the created file.
	FilePath string
	// FileSize is the size of the file in bytes.
	FileSize uint64
	// JobID is the ID of the job that created the file.
	JobID int
	// Reason is why the file was created.
	Reason TableFileCreationReason
	// Status is the creation status (nil for success).
	Status error
	// TableProperties contains properties of the created SST file.
	TableProperties map[string]string
}

TableFileCreationInfo contains information about a table file creation.

type TableFileCreationReason

type TableFileCreationReason int

TableFileCreationReason describes why a table file was created.

const (
	// TableFileCreationReasonFlush is for flush.
	TableFileCreationReasonFlush TableFileCreationReason = iota
	// TableFileCreationReasonCompaction is for compaction.
	TableFileCreationReasonCompaction
	// TableFileCreationReasonRecovery is for recovery.
	TableFileCreationReasonRecovery
	// TableFileCreationReasonMisc is for miscellaneous reasons.
	TableFileCreationReasonMisc
)

func (TableFileCreationReason) String

func (r TableFileCreationReason) String() string

String returns the string representation of the table file creation reason.

type TableFileDeletionInfo

type TableFileDeletionInfo struct {
	// DBName is the database name.
	DBName string
	// FilePath is the path to the deleted file.
	FilePath string
	// JobID is the ID of the job that deleted the file.
	JobID int
	// Status is the deletion status (nil for success).
	Status error
}

TableFileDeletionInfo contains information about a table file deletion.

type TickerType

type TickerType int

TickerType represents different types of counters.

const (
	// TickerBlockCacheMiss is the count of block cache misses.
	TickerBlockCacheMiss TickerType = iota
	// TickerBlockCacheHit is the count of block cache hits.
	TickerBlockCacheHit
	// TickerBytesWritten is the total bytes written to the database.
	TickerBytesWritten
	// TickerBytesRead is the total bytes read from the database.
	TickerBytesRead
	// TickerNumberKeysWritten is the count of keys written.
	TickerNumberKeysWritten
	// TickerNumberKeysRead is the count of keys read.
	TickerNumberKeysRead
	// TickerNumberSeekNext is the count of Iterator.Next() calls.
	TickerNumberSeekNext
	// TickerNumberSeekPrev is the count of Iterator.Prev() calls.
	TickerNumberSeekPrev
	// TickerNumberSeek is the count of Iterator.Seek() calls.
	TickerNumberSeek
	// TickerWALFileBytes is the total bytes written to WAL.
	TickerWALFileBytes
	// TickerWALFileSynced is the count of WAL file syncs.
	TickerWALFileSynced
	// TickerCompactReadBytes is bytes read during compaction.
	TickerCompactReadBytes
	// TickerCompactWriteBytes is bytes written during compaction.
	TickerCompactWriteBytes
	// TickerFlushWriteBytes is bytes written during flush.
	TickerFlushWriteBytes
	// TickerMemtableHit is the count of memtable hits.
	TickerMemtableHit
	// TickerMemtableMiss is the count of memtable misses.
	TickerMemtableMiss
	// TickerGetHitL0 is the count of L0 hits.
	TickerGetHitL0
	// TickerGetHitL1 is the count of L1 hits.
	TickerGetHitL1
	// TickerGetHitL2AndUp is the count of L2+ hits.
	TickerGetHitL2AndUp
	// TickerBloomFilterUseful is the count of bloom filter useful rejections.
	TickerBloomFilterUseful
	// TickerBloomFilterFullPositive is the count of bloom filter full positives.
	TickerBloomFilterFullPositive
	// TickerBloomFilterFullTruePositive is the count of bloom filter true positives.
	TickerBloomFilterFullTruePositive
	// TickerNumberDBSeek is the count of db.Get() calls.
	TickerNumberDBSeek
	// TickerNumberDBNext is the count of iterator next calls.
	TickerNumberDBNext
	// TickerNumberDBPrev is the count of iterator prev calls.
	TickerNumberDBPrev
	// TickerNumberDBSeekFound is the count of db.Get() calls that found the key.
	TickerNumberDBSeekFound
	// TickerNumberDBSeekNotFound is the count of db.Get() calls that didn't find the key.
	TickerNumberDBSeekNotFound
	// TickerIterBytesRead is the total bytes read by iterators.
	TickerIterBytesRead
	// TickerWriteWithWAL is the count of writes with WAL.
	TickerWriteWithWAL
	// TickerWriteWithoutWAL is the count of writes without WAL.
	TickerWriteWithoutWAL
	// TickerStallMicros is the total microseconds spent in write stalls.
	TickerStallMicros

	// Block cache granular metrics (reference: RocksDB statistics.h)
	// TickerBlockCacheIndexMiss is the count of index block cache misses.
	TickerBlockCacheIndexMiss
	// TickerBlockCacheIndexHit is the count of index block cache hits.
	TickerBlockCacheIndexHit
	// TickerBlockCacheFilterMiss is the count of filter block cache misses.
	TickerBlockCacheFilterMiss
	// TickerBlockCacheFilterHit is the count of filter block cache hits.
	TickerBlockCacheFilterHit
	// TickerBlockCacheDataMiss is the count of data block cache misses.
	TickerBlockCacheDataMiss
	// TickerBlockCacheDataHit is the count of data block cache hits.
	TickerBlockCacheDataHit
	// TickerBlockCacheAdd is the count of blocks added to cache.
	TickerBlockCacheAdd
	// TickerBlockCacheBytesRead is the total bytes read from block cache.
	TickerBlockCacheBytesRead
	// TickerBlockCacheBytesWrite is the total bytes written to block cache.
	TickerBlockCacheBytesWrite

	// Compaction key drop reasons (reference: RocksDB statistics.h)
	// TickerCompactionKeyDropNewerEntry is keys dropped due to newer entry.
	TickerCompactionKeyDropNewerEntry
	// TickerCompactionKeyDropObsolete is keys dropped as obsolete.
	TickerCompactionKeyDropObsolete
	// TickerCompactionKeyDropRangeDel is keys dropped due to range delete.
	TickerCompactionKeyDropRangeDel

	// File operations
	// TickerNoFileOpens is the count of file opens.
	TickerNoFileOpens
	// TickerNoFileErrors is the count of file errors.
	TickerNoFileErrors

	// MultiGet statistics
	// TickerNumberMultiGetCalls is the count of MultiGet calls.
	TickerNumberMultiGetCalls
	// TickerNumberMultiGetKeysRead is the count of keys read in MultiGet.
	TickerNumberMultiGetKeysRead
	// TickerNumberMultiGetKeysFound is the count of keys found in MultiGet.
	TickerNumberMultiGetKeysFound
	// TickerNumberMultiGetBytesRead is the bytes read in MultiGet.
	TickerNumberMultiGetBytesRead

	// TickerNumberMergeFailures is the count of merge operation failures.
	TickerNumberMergeFailures

	// TickerEnumMax is the maximum ticker type for sizing arrays.
	TickerEnumMax
)

func (TickerType) String

func (t TickerType) String() string

String returns the name of the ticker type.

type TimestampedComparator

type TimestampedComparator interface {
	Comparator

	// TimestampSize returns the size of the timestamp in bytes.
	// Returns 0 if timestamps are not supported.
	TimestampSize() int

	// CompareTimestamp compares two timestamps.
	// Returns < 0 if ts1 < ts2, 0 if equal, > 0 if ts1 > ts2.
	CompareTimestamp(ts1, ts2 []byte) int

	// CompareWithoutTimestamp compares two keys ignoring their timestamps.
	CompareWithoutTimestamp(a, b []byte, aHasTS, bHasTS bool) int

	// GetMaxTimestamp returns the maximum timestamp value.
	GetMaxTimestamp() []byte

	// GetMinTimestamp returns the minimum timestamp value.
	GetMinTimestamp() []byte
}

TimestampedComparator is a comparator that supports user-defined timestamps. It extends the base Comparator interface with timestamp-aware methods.

type TimestampedDB

type TimestampedDB struct {
	// contains filtered or unexported fields
}

TimestampedDB wraps a DB and provides timestamp-aware operations.

func OpenTimestampedDB

func OpenTimestampedDB(path string, opts *Options) (*TimestampedDB, error)

OpenTimestampedDB opens a database with timestamp support. The options must have a TimestampedComparator set.

func WrapWithTimestamp

func WrapWithTimestamp(db DB, comparator TimestampedComparator) (*TimestampedDB, error)

WrapWithTimestamp wraps an existing DB with timestamp support.

func (*TimestampedDB) Close

func (t *TimestampedDB) Close() error

Close closes the database.

func (*TimestampedDB) DB

func (t *TimestampedDB) DB() DB

DB returns the underlying database.

func (*TimestampedDB) Delete

func (t *TimestampedDB) Delete(opts *WriteOptions, key []byte) error

Delete deletes a key using the maximum timestamp.

func (*TimestampedDB) DeleteWithTimestamp

func (t *TimestampedDB) DeleteWithTimestamp(opts *WriteOptions, key, timestamp []byte) error

DeleteWithTimestamp deletes a key at the given timestamp.

func (*TimestampedDB) Flush

func (t *TimestampedDB) Flush(opts *FlushOptions) error

Flush flushes the memtable to disk.

func (*TimestampedDB) Get

func (t *TimestampedDB) Get(opts *ReadOptions, key []byte) ([]byte, error)

Get retrieves the value for a key at the maximum timestamp.

func (*TimestampedDB) GetWithTimestamp

func (t *TimestampedDB) GetWithTimestamp(opts *ReadOptions, key, timestamp []byte) (value, foundTS []byte, err error)

GetWithTimestamp retrieves the value for a key at the given timestamp. Returns the value, the timestamp of the found record, and any error.

func (*TimestampedDB) NewIterator

func (t *TimestampedDB) NewIterator(opts *ReadOptions) Iterator

NewIterator creates an iterator over the database. The iterator returns keys with timestamps appended.

func (*TimestampedDB) NewTimestampedIterator

func (t *TimestampedDB) NewTimestampedIterator(opts *ReadOptions) *TimestampedIterator

NewTimestampedIterator creates a timestamp-aware iterator. It filters keys based on the timestamp in ReadOptions.

func (*TimestampedDB) Put

func (t *TimestampedDB) Put(opts *WriteOptions, key, value []byte) error

Put stores a key-value pair using the maximum timestamp. This is a convenience method for applications that want to write with the "current" timestamp.

func (*TimestampedDB) PutWithTimestamp

func (t *TimestampedDB) PutWithTimestamp(opts *WriteOptions, key, value, timestamp []byte) error

PutWithTimestamp stores a key-value pair with the given timestamp.

func (*TimestampedDB) TimestampSize

func (t *TimestampedDB) TimestampSize() int

TimestampSize returns the size of timestamps in bytes.

type TimestampedIterator

type TimestampedIterator struct {
	// contains filtered or unexported fields
}

TimestampedIterator wraps an iterator and filters by timestamp.

func (*TimestampedIterator) Close

func (ti *TimestampedIterator) Close() error

Close closes the iterator.

func (*TimestampedIterator) Error

func (ti *TimestampedIterator) Error() error

Error returns any error encountered by the iterator.

func (*TimestampedIterator) Key

func (ti *TimestampedIterator) Key() []byte

Key returns the current key (with timestamp).

func (*TimestampedIterator) Next

func (ti *TimestampedIterator) Next()

Next advances the iterator to the next valid entry.

func (*TimestampedIterator) Prev

func (ti *TimestampedIterator) Prev()

Prev moves the iterator to the previous valid entry.

func (*TimestampedIterator) Seek

func (ti *TimestampedIterator) Seek(target []byte)

Seek positions the iterator at the first key >= target. The target should be a user key without timestamp.

func (*TimestampedIterator) SeekForPrev

func (ti *TimestampedIterator) SeekForPrev(target []byte)

SeekForPrev positions the iterator at the last key <= target.

func (*TimestampedIterator) SeekToFirst

func (ti *TimestampedIterator) SeekToFirst()

SeekToFirst positions the iterator at the first valid key.

func (*TimestampedIterator) SeekToLast

func (ti *TimestampedIterator) SeekToLast()

SeekToLast positions the iterator at the last valid key.

func (*TimestampedIterator) Timestamp

func (ti *TimestampedIterator) Timestamp() []byte

Timestamp returns the timestamp of the current key.

func (*TimestampedIterator) UserKey

func (ti *TimestampedIterator) UserKey() []byte

UserKey returns the current user key (without timestamp).

func (*TimestampedIterator) Valid

func (ti *TimestampedIterator) Valid() bool

Valid returns true if the iterator is positioned at a valid entry.

func (*TimestampedIterator) Value

func (ti *TimestampedIterator) Value() []byte

Value returns the current value.

type TimingEventListener

type TimingEventListener struct {
	NoOpEventListener
	FlushTimes      []time.Time
	CompactionTimes []time.Time
	// contains filtered or unexported fields
}

TimingEventListener records timing information for testing.

func (*TimingEventListener) OnCompactionCompleted

func (l *TimingEventListener) OnCompactionCompleted(info *CompactionJobInfo)

func (*TimingEventListener) OnFlushCompleted

func (l *TimingEventListener) OnFlushCompleted(info *FlushJobInfo)

type Transaction

type Transaction interface {
	// Put sets the value for the given key within the transaction.
	Put(key, value []byte) error

	// PutCF sets the value for the given key in the specified column family.
	PutCF(cf ColumnFamilyHandle, key, value []byte) error

	// Get retrieves the value for the given key.
	// It first checks the transaction's write batch, then falls back to the database.
	Get(key []byte) ([]byte, error)

	// GetCF retrieves the value from the specified column family.
	GetCF(cf ColumnFamilyHandle, key []byte) ([]byte, error)

	// GetForUpdate retrieves the value and marks it for conflict detection.
	// For optimistic transactions, this tracks the key for validation at commit.
	// For pessimistic transactions, this acquires a lock.
	GetForUpdate(key []byte, exclusive bool) ([]byte, error)

	// Delete removes the key from the transaction.
	Delete(key []byte) error

	// DeleteCF removes the key from the specified column family.
	DeleteCF(cf ColumnFamilyHandle, key []byte) error

	// Commit validates and applies the transaction.
	// Returns ErrTransactionConflict if there are write conflicts.
	Commit() error

	// Rollback discards the transaction.
	Rollback() error

	// SetSnapshot sets the transaction's snapshot to the current database state.
	SetSnapshot()

	// GetSnapshot returns the transaction's snapshot.
	GetSnapshot() *Snapshot
}

Transaction represents an optimistic transaction. Changes made within a transaction are isolated from other transactions until the transaction is committed. At commit time, the transaction validates that no conflicts have occurred with other concurrent transactions.

type TransactionDB

type TransactionDB struct {
	// contains filtered or unexported fields
}

TransactionDB wraps a regular DB and provides transactional operations. It supports both optimistic and pessimistic concurrency control.

func OpenTransactionDB

func OpenTransactionDB(path string, dbOpts *Options, txnDBOpts TransactionDBOptions) (*TransactionDB, error)

OpenTransactionDB opens or creates a TransactionDB.

func WrapDB

func WrapDB(database DB, txnDBOpts TransactionDBOptions) (*TransactionDB, error)

WrapDB wraps an existing database as a TransactionDB.

func (*TransactionDB) BeginTransaction

func (txnDB *TransactionDB) BeginTransaction(opts PessimisticTransactionOptions, writeOpts *WriteOptions) *PessimisticTransaction

BeginTransaction begins a new pessimistic transaction.

func (*TransactionDB) Close

func (txnDB *TransactionDB) Close() error

Close closes the TransactionDB and the underlying database.

func (*TransactionDB) Delete

func (txnDB *TransactionDB) Delete(key []byte) error

Delete removes a key from the database (outside of a transaction).

func (*TransactionDB) Flush

func (txnDB *TransactionDB) Flush(opts *FlushOptions) error

Flush flushes all memtables to disk.

func (*TransactionDB) Get

func (txnDB *TransactionDB) Get(key []byte) ([]byte, error)

Get retrieves a value from the database.

func (*TransactionDB) GetAllPreparedTransactions

func (txnDB *TransactionDB) GetAllPreparedTransactions() []*PessimisticTransaction

GetAllPreparedTransactions returns all transactions (for recovery).

func (*TransactionDB) GetDB

func (txnDB *TransactionDB) GetDB() DB

GetDB returns the underlying database.

func (*TransactionDB) GetProperty

func (txnDB *TransactionDB) GetProperty(property string) (string, bool)

GetProperty returns a database property.

func (*TransactionDB) GetSnapshot

func (txnDB *TransactionDB) GetSnapshot() *Snapshot

GetSnapshot creates a snapshot of the database.

func (*TransactionDB) GetTransactionByID

func (txnDB *TransactionDB) GetTransactionByID(txnID uint64) *PessimisticTransaction

GetTransactionByID returns the transaction with the given ID, or nil if not found.

func (*TransactionDB) GetWithOptions

func (txnDB *TransactionDB) GetWithOptions(readOpts *ReadOptions, key []byte) ([]byte, error)

GetWithOptions retrieves a value with read options.

func (*TransactionDB) NewIterator

func (txnDB *TransactionDB) NewIterator(opts *ReadOptions) Iterator

NewIterator creates an iterator over the database.

func (*TransactionDB) NumActiveTransactions

func (txnDB *TransactionDB) NumActiveTransactions() int

NumActiveTransactions returns the number of active transactions.

func (*TransactionDB) Put

func (txnDB *TransactionDB) Put(key, value []byte) error

Put sets a value in the database (outside of a transaction).

func (*TransactionDB) ReleaseSnapshot

func (txnDB *TransactionDB) ReleaseSnapshot(snapshot *Snapshot)

ReleaseSnapshot releases a snapshot.

type TransactionDBOptions

type TransactionDBOptions struct {
	// LockManagerOptions configures the lock manager.
	LockManagerOptions LockManagerOptions

	// MaxNumLocks is the maximum number of locks to track (0 = unlimited).
	MaxNumLocks uint64

	// NumStripes is the number of lock stripes for lock manager striping (for future use).
	NumStripes int

	// TransactionLockTimeout is the default lock timeout for transactions.
	TransactionLockTimeout int64 // in milliseconds
}

TransactionDBOptions configures a TransactionDB.

func DefaultTransactionDBOptions

func DefaultTransactionDBOptions() TransactionDBOptions

DefaultTransactionDBOptions returns default options.

type TransactionLogIterator

type TransactionLogIterator struct {
	// contains filtered or unexported fields
}

TransactionLogIterator iterates over WAL records.

func (*TransactionLogIterator) Close

func (iter *TransactionLogIterator) Close() error

Close releases resources associated with the iterator.

func (*TransactionLogIterator) GetBatch

func (iter *TransactionLogIterator) GetBatch() (*BatchResult, error)

GetBatch returns the current write batch and its sequence number. REQUIRES: Valid() returns true.

func (*TransactionLogIterator) Next

func (iter *TransactionLogIterator) Next()

Next moves the iterator to the next write batch.

func (*TransactionLogIterator) Status

func (iter *TransactionLogIterator) Status() error

Status returns any error encountered by the iterator.

func (*TransactionLogIterator) Valid

func (iter *TransactionLogIterator) Valid() bool

Valid returns true if the iterator is positioned at a valid write batch.

type TransactionLogIteratorReadOptions

type TransactionLogIteratorReadOptions struct {
	// VerifyChecksums enables checksum verification when reading WAL records.
	VerifyChecksums bool
}

TransactionLogIteratorReadOptions controls transaction log iterator behavior.

func DefaultTransactionLogIteratorReadOptions

func DefaultTransactionLogIteratorReadOptions() TransactionLogIteratorReadOptions

DefaultTransactionLogIteratorReadOptions returns default read options.

type TransactionOptions

type TransactionOptions struct {
	// SetSnapshot determines if the transaction should set a snapshot at creation.
	SetSnapshot bool
}

TransactionOptions configures a transaction.

func DefaultTransactionOptions

func DefaultTransactionOptions() TransactionOptions

DefaultTransactionOptions returns default transaction options.

type TransactionState

type TransactionState int

TransactionState represents the state of a write-prepared transaction.

const (
	// TxnStateStarted is the initial state.
	TxnStateStarted TransactionState = iota
	// TxnStatePrepared means the transaction has been prepared but not committed.
	TxnStatePrepared
	// TxnStateCommitted means the transaction has been committed.
	TxnStateCommitted
	// TxnStateRolledBack means the transaction was rolled back.
	TxnStateRolledBack
)

type UInt64AddOperator

type UInt64AddOperator struct{}

UInt64AddOperator is a merge operator that treats values as uint64 and adds them.

func (*UInt64AddOperator) FullMerge

func (o *UInt64AddOperator) FullMerge(key []byte, existingValue []byte, operands [][]byte) ([]byte, bool)

FullMerge adds all operands to the existing value.

func (*UInt64AddOperator) Name

func (o *UInt64AddOperator) Name() string

Name returns the name of this merge operator.

func (*UInt64AddOperator) PartialMerge

func (o *UInt64AddOperator) PartialMerge(key []byte, left, right []byte) ([]byte, bool)

PartialMerge adds two operands together.

type UniversalCompactionOptions

type UniversalCompactionOptions struct {
	// SizeRatio is the percentage trigger for size ratio compaction.
	// Default: 1
	SizeRatio int

	// MinMergeWidth is the minimum number of files to merge.
	// Default: 2
	MinMergeWidth int

	// MaxMergeWidth is the maximum number of files to merge.
	// Default: unlimited
	MaxMergeWidth int

	// MaxSizeAmplificationPercent triggers full compaction when exceeded.
	// Default: 200
	MaxSizeAmplificationPercent int

	// AllowTrivialMove allows trivial moves when possible.
	// Default: false
	AllowTrivialMove bool
}

UniversalCompactionOptions contains options for universal compaction.

func DefaultUniversalCompactionOptions

func DefaultUniversalCompactionOptions() *UniversalCompactionOptions

DefaultUniversalCompactionOptions returns default options.

type WaitForCompactOptions

type WaitForCompactOptions struct {
	// AbortOnPause makes WaitForCompact abort if compaction is paused.
	AbortOnPause bool
	// FlushFirst flushes memtable before waiting for compaction.
	FlushFirst bool
	// CloseDB closes the database after waiting (for graceful shutdown).
	CloseDB bool
	// Timeout is the maximum time to wait. Zero means wait forever.
	Timeout time.Duration
}

WaitForCompactOptions controls WaitForCompact behavior. Reference: RocksDB v10.7.5 include/rocksdb/options.h

type WalFile

type WalFile struct {
	// PathName returns the full path to the WAL file.
	PathName string

	// LogNumber is the WAL file number.
	LogNumber uint64

	// Type indicates whether the file is live or archived.
	Type WalFileType

	// StartSequence is the starting sequence number of the first write batch.
	StartSequence uint64

	// SizeBytes is the size of the file in bytes.
	SizeBytes uint64
}

WalFile represents a WAL file.

type WalFileType

type WalFileType int

WalFileType indicates whether a WAL file is live or archived.

const (
	// WalFileTypeLive indicates the WAL file is still being written to.
	WalFileTypeLive WalFileType = iota

	// WalFileTypeArchived indicates the WAL file has been archived.
	WalFileTypeArchived
)

type WideColumn

type WideColumn struct {
	Name  []byte
	Value []byte
}

WideColumn represents a named column with a value.

type WideColumnDB

type WideColumnDB struct {
	// contains filtered or unexported fields
}

WideColumnDB wraps a DB to provide wide column operations.

func NewWideColumnDB

func NewWideColumnDB(db DB) *WideColumnDB

NewWideColumnDB creates a new WideColumnDB wrapper.

func (*WideColumnDB) DeleteEntity

func (w *WideColumnDB) DeleteEntity(key []byte) error

DeleteEntity deletes an entity.

func (*WideColumnDB) GetEntity

func (w *WideColumnDB) GetEntity(key []byte) (*PinnableWideColumns, error)

GetEntity retrieves an entity's columns.

func (*WideColumnDB) MergeEntity

func (w *WideColumnDB) MergeEntity(key []byte, newColumns WideColumns) error

MergeEntity merges new columns into an existing entity. Existing columns with the same name are overwritten.

func (*WideColumnDB) PutEntity

func (w *WideColumnDB) PutEntity(key []byte, columns WideColumns) error

PutEntity stores an entity (key with multiple named columns).

type WideColumnDBInterface

type WideColumnDBInterface interface {
	// PutEntity stores an entity (key with multiple named columns)
	PutEntity(key []byte, columns WideColumns) error

	// GetEntity retrieves an entity's columns
	GetEntity(key []byte) (*PinnableWideColumns, error)

	// DeleteEntity deletes an entity
	DeleteEntity(key []byte) error
}

WideColumnDBInterface defines the interface for wide column operations. This is an extension to the standard DB interface.

type WideColumns

type WideColumns []WideColumn

WideColumns is a slice of WideColumn.

func DecodeWideColumns

func DecodeWideColumns(data []byte) (WideColumns, error)

DecodeWideColumns decodes wide columns from bytes.

func (WideColumns) Get

func (wc WideColumns) Get(name []byte) []byte

Get returns the value of a column by name, or nil if not found.

func (WideColumns) GetString

func (wc WideColumns) GetString(name string) []byte

GetString returns the value of a column by name as a string.

func (WideColumns) Len

func (wc WideColumns) Len() int

Len returns the number of columns.

func (WideColumns) Less

func (wc WideColumns) Less(i, j int) bool

Less returns true if column i's name is less than column j's name.

func (WideColumns) Sort

func (wc WideColumns) Sort()

Sort sorts the columns by name.

func (WideColumns) Swap

func (wc WideColumns) Swap(i, j int)

Swap swaps two columns.

type WriteBatch

type WriteBatch struct {
	// contains filtered or unexported fields
}

WriteBatch holds a collection of writes to be applied atomically. Keys and values are copied, so you can modify them after calling Put/Delete.

A WriteBatch can be reused by calling Clear() after Write().

Example:

wb := db.NewWriteBatch()
wb.Put([]byte("key1"), []byte("value1"))
wb.Put([]byte("key2"), []byte("value2"))
wb.Delete([]byte("key3"))
err := database.Write(writeOpts, wb)
wb.Clear() // Reuse the batch

func NewWriteBatch

func NewWriteBatch() *WriteBatch

NewWriteBatch creates a new empty WriteBatch.

func (*WriteBatch) Clear

func (wb *WriteBatch) Clear()

Clear resets the batch to empty, allowing it to be reused.

func (*WriteBatch) Count

func (wb *WriteBatch) Count() uint32

Count returns the number of operations in the batch.

func (*WriteBatch) Data

func (wb *WriteBatch) Data() []byte

Data returns the raw batch data (for advanced use only).

func (*WriteBatch) Delete

func (wb *WriteBatch) Delete(key []byte)

Delete adds a deletion for the key to the batch.

func (*WriteBatch) DeleteCF

func (wb *WriteBatch) DeleteCF(cfID uint32, key []byte)

DeleteCF adds a deletion for the key to the batch for the specified column family.

func (*WriteBatch) DeleteRange

func (wb *WriteBatch) DeleteRange(startKey, endKey []byte)

DeleteRange adds a range deletion [startKey, endKey) to the batch.

func (*WriteBatch) DeleteRangeCF

func (wb *WriteBatch) DeleteRangeCF(cfID uint32, startKey, endKey []byte)

DeleteRangeCF adds a range deletion to the batch for the specified column family.

func (*WriteBatch) Merge

func (wb *WriteBatch) Merge(key, value []byte)

Merge adds a merge operand for the key to the batch.

func (*WriteBatch) MergeCF

func (wb *WriteBatch) MergeCF(cfID uint32, key, value []byte)

MergeCF adds a merge operand to the batch for the specified column family.

func (*WriteBatch) Put

func (wb *WriteBatch) Put(key, value []byte)

Put adds a key-value pair to the batch.

func (*WriteBatch) PutCF

func (wb *WriteBatch) PutCF(cfID uint32, key, value []byte)

PutCF adds a key-value pair to the batch for the specified column family.

func (*WriteBatch) SingleDelete

func (wb *WriteBatch) SingleDelete(key []byte)

SingleDelete adds a single deletion for the key to the batch. SingleDelete is more efficient than Delete when the key has only one version.

func (*WriteBatch) SingleDeleteCF

func (wb *WriteBatch) SingleDeleteCF(cfID uint32, key []byte)

SingleDeleteCF adds a single deletion to the batch for the specified column family.

type WriteBufferManager

type WriteBufferManager struct {
	// contains filtered or unexported fields
}

WriteBufferManager controls memory usage across memtables. It can be shared across multiple DBs or column families.

When memory usage exceeds the buffer limit, it triggers a flush in the DB that receives the next write operation.

Reference: RocksDB's WriteBufferManager class.

func NewWriteBufferManager

func NewWriteBufferManager(bufferSize uint64, allowStall bool) *WriteBufferManager

NewWriteBufferManager creates a new WriteBufferManager.

Parameters:

  • bufferSize: Maximum memory to use (0 = unlimited)
  • allowStall: If true, stall writes when memory usage exceeds limit

Example:

// 256MB limit, stall when exceeded
wbm := NewWriteBufferManager(256*1024*1024, true)

// Unlimited memory, no stalling
wbm := NewWriteBufferManager(0, false)

func (*WriteBufferManager) BufferSize

func (wbm *WriteBufferManager) BufferSize() uint64

BufferSize returns the configured buffer size limit.

func (*WriteBufferManager) Enabled

func (wbm *WriteBufferManager) Enabled() bool

Enabled returns true if memory limiting is enabled.

func (*WriteBufferManager) FreeMem

func (wbm *WriteBufferManager) FreeMem(mem uint64)

FreeMem frees previously reserved memory. This should be called when a memtable is flushed and freed.

func (*WriteBufferManager) IsStalled

func (wbm *WriteBufferManager) IsStalled() bool

IsStalled returns true if writes are currently stalled.

func (*WriteBufferManager) MemoryUsage

func (wbm *WriteBufferManager) MemoryUsage() uint64

MemoryUsage returns the current total memory usage.

func (*WriteBufferManager) MutableMemtableMemoryUsage

func (wbm *WriteBufferManager) MutableMemtableMemoryUsage() uint64

MutableMemtableMemoryUsage returns the memory used by active (mutable) memtables.

func (*WriteBufferManager) ReserveMem

func (wbm *WriteBufferManager) ReserveMem(mem uint64)

ReserveMem reserves memory for a new allocation. This should be called when memory is allocated for a memtable.

func (*WriteBufferManager) ResetStats

func (wbm *WriteBufferManager) ResetStats()

ResetStats resets the statistics.

func (*WriteBufferManager) ScheduleFreeMem

func (wbm *WriteBufferManager) ScheduleFreeMem(mem uint64)

ScheduleFreeMem marks memory as no longer active (scheduled for free). This is called when a memtable becomes immutable and is waiting for flush.

func (*WriteBufferManager) ShouldFlush

func (wbm *WriteBufferManager) ShouldFlush() bool

ShouldFlush returns true if a flush should be triggered. This happens when memory usage exceeds 7/8 of the buffer limit.

func (*WriteBufferManager) Stats

func (wbm *WriteBufferManager) Stats() WriteBufferStats

Stats returns a copy of the statistics.

func (*WriteBufferManager) UsageRatio

func (wbm *WriteBufferManager) UsageRatio() float64

UsageRatio returns the current memory usage as a ratio (0.0 to 1.0+).

func (*WriteBufferManager) WaitIfStalled

func (wbm *WriteBufferManager) WaitIfStalled() bool

WaitIfStalled blocks until memory pressure is relieved. This implements write stalling based on memory usage. Returns true if the caller was stalled.

type WriteBufferStats

type WriteBufferStats struct {
	TotalReserved   uint64 // Total memory reserved
	TotalFreed      uint64 // Total memory freed
	PeakUsage       uint64 // Peak memory usage
	FlushTriggers   uint64 // Number of times flush was triggered
	StallEvents     uint64 // Number of stall events
	StallDurationNs uint64 // Total stall duration in nanoseconds
}

WriteBufferStats tracks memory management statistics.

type WriteOptions

type WriteOptions struct {
	// Sync causes writes to be flushed to the WAL and fsynced before returning.
	// This provides the strongest durability guarantee but reduces throughput.
	Sync bool

	// DisableWAL disables the write-ahead log for this write.
	//
	// WARNING: With DisableWAL=true, writes go directly to the memtable.
	// If the process crashes before Flush() is called, data will be lost.
	// This matches C++ RocksDB behavior exactly.
	//
	// Use only when you can tolerate data loss in exchange for higher throughput.
	// Call Flush() explicitly before shutdown to persist unflushed data.
	DisableWAL bool
}

WriteOptions contains options for write operations.

func DefaultWriteOptions

func DefaultWriteOptions() *WriteOptions

DefaultWriteOptions returns WriteOptions with default values.

type WritePreparedTxn

type WritePreparedTxn struct {
	*PessimisticTransaction
	// contains filtered or unexported fields
}

WritePreparedTxn implements a write-prepared transaction.

func (*WritePreparedTxn) Commit

func (txn *WritePreparedTxn) Commit() error

Commit commits a prepared transaction. This implements the second phase of two-phase commit.

func (*WritePreparedTxn) GetCommitSeq

func (txn *WritePreparedTxn) GetCommitSeq() uint64

GetCommitSeq returns the commit sequence number.

func (*WritePreparedTxn) GetName

func (txn *WritePreparedTxn) GetName() string

GetName returns the transaction name.

func (*WritePreparedTxn) GetPrepareSeq

func (txn *WritePreparedTxn) GetPrepareSeq() uint64

GetPrepareSeq returns the prepare sequence number.

func (*WritePreparedTxn) GetState

func (txn *WritePreparedTxn) GetState() TransactionState

GetState returns the current transaction state.

func (*WritePreparedTxn) Prepare

func (txn *WritePreparedTxn) Prepare() error

Prepare prepares the transaction for commit. After Prepare, the writes are in the WAL/memtable but not yet visible. This implements the first phase of two-phase commit.

func (*WritePreparedTxn) Rollback

func (txn *WritePreparedTxn) Rollback() error

Rollback rolls back the transaction.

func (*WritePreparedTxn) SetName

func (txn *WritePreparedTxn) SetName(name string) error

SetName sets the transaction name for external coordination.

func (*WritePreparedTxn) WaitAfterPrepare

func (txn *WritePreparedTxn) WaitAfterPrepare(d time.Duration)

WaitAfterPrepare waits for the specified duration after prepare. This is useful for testing 2PC recovery scenarios.

type WritePreparedTxnDB

type WritePreparedTxnDB struct {
	*TransactionDB
	// contains filtered or unexported fields
}

WritePreparedTxnDB extends TransactionDB with write-prepared transaction support.

func OpenWritePreparedTxnDB

func OpenWritePreparedTxnDB(path string, opts *Options, txnOpts TransactionDBOptions) (*WritePreparedTxnDB, error)

OpenWritePreparedTxnDB opens a database with write-prepared transaction support.

func (*WritePreparedTxnDB) BeginWritePreparedTransaction

func (db *WritePreparedTxnDB) BeginWritePreparedTransaction(opts PessimisticTransactionOptions, writeOpts *WriteOptions) *WritePreparedTxn

BeginWritePreparedTransaction starts a new write-prepared transaction.

func (*WritePreparedTxnDB) CommitPreparedTransaction

func (db *WritePreparedTxnDB) CommitPreparedTransaction(name string) error

CommitPreparedTransaction commits a recovered prepared transaction by name.

func (*WritePreparedTxnDB) GetAllPreparedTransactions

func (db *WritePreparedTxnDB) GetAllPreparedTransactions() []*RecoveredPreparedTxn

GetAllPreparedTransactions returns all prepared transactions that were recovered after a crash and need to be resolved (committed or rolled back).

func (*WritePreparedTxnDB) RollbackPreparedTransaction

func (db *WritePreparedTxnDB) RollbackPreparedTransaction(name string) error

RollbackPreparedTransaction rolls back a recovered prepared transaction by name.

type WriteStallCause

type WriteStallCause int

WriteStallCause indicates why writes are being stalled.

const (
	// WriteStallCauseNone means no stall.
	WriteStallCauseNone WriteStallCause = iota
	// WriteStallCauseMemtableLimit means too many unflushed memtables.
	WriteStallCauseMemtableLimit
	// WriteStallCauseL0FileCountLimit means too many L0 files.
	WriteStallCauseL0FileCountLimit
	// WriteStallCausePendingCompactionBytes means too many pending compaction bytes.
	WriteStallCausePendingCompactionBytes
)

func (WriteStallCause) String

func (c WriteStallCause) String() string

String returns a human-readable description of the stall cause.

type WriteStallCondition

type WriteStallCondition int

WriteStallCondition describes the write stall condition.

const (
	// WriteStallConditionNormal means no stall.
	WriteStallConditionNormal WriteStallCondition = iota
	// WriteStallConditionDelayed means writes are delayed.
	WriteStallConditionDelayed
	// WriteStallConditionStopped means writes are stopped.
	WriteStallConditionStopped
)

func (WriteStallCondition) String

func (c WriteStallCondition) String() string

String returns the string representation of the write stall condition.

type WriteStallController added in v0.3.4

type WriteStallController interface {
	DB

	ReleaseWriteStall()
}

WriteStallController exposes write-stall release for shutdown/unblocking.

Contract: ReleaseWriteStall is safe to call at shutdown to unblock any stalled writers.

type WriteStallInfo

type WriteStallInfo struct {
	// CFName is the column family name.
	CFName string
	// Condition is the stall condition.
	Condition WriteStallCondition
	// Prev is the previous stall condition.
	Prev WriteStallCondition
}

WriteStallInfo contains information about a write stall.

Directories

Path Synopsis
cmd
adversarialtest command
Adversarial test runner for RockyardKV.
Adversarial test runner for RockyardKV.
apileakcheck command
campaignrunner command
Package main implements the campaign runner CLI.
Package main implements the campaign runner CLI.
crashtest command
Crash test orchestrator for RockyardKV.
Crash test orchestrator for RockyardKV.
goldentest command
ldb command
Database inspection tool for RockyardKV.
Database inspection tool for RockyardKV.
manifestdump command
MANIFEST dump utility for RockyardKV.
MANIFEST dump utility for RockyardKV.
smoketest command
End-to-end smoke test for RockyardKV.
End-to-end smoke test for RockyardKV.
sstdump command
SST inspection tool for RockyardKV.
SST inspection tool for RockyardKV.
stresstest command
Full stress test for RockyardKV.
Full stress test for RockyardKV.
traceanalyzer command
Trace analyzer for RockyardKV.
Trace analyzer for RockyardKV.
examples
backup command
Package main demonstrates backup and restore in RockyardKV.
Package main demonstrates backup and restore in RockyardKV.
basic command
Package main demonstrates basic RockyardKV operations.
Package main demonstrates basic RockyardKV operations.
batch command
Package main demonstrates atomic batch writes in RockyardKV.
Package main demonstrates atomic batch writes in RockyardKV.
column_families command
Package main demonstrates column families in RockyardKV.
Package main demonstrates column families in RockyardKV.
compaction_filter command
Package main demonstrates compaction filters in RockyardKV.
Package main demonstrates compaction filters in RockyardKV.
compression command
Package main demonstrates compression options in RockyardKV.
Package main demonstrates compression options in RockyardKV.
iteration command
Package main demonstrates iterator usage in RockyardKV.
Package main demonstrates iterator usage in RockyardKV.
merge command
Package main demonstrates merge operators in RockyardKV.
Package main demonstrates merge operators in RockyardKV.
snapshots command
Package main demonstrates snapshots in RockyardKV.
Package main demonstrates snapshots in RockyardKV.
sst_ingestion command
Package main demonstrates SST file ingestion in RockyardKV.
Package main demonstrates SST file ingestion in RockyardKV.
transactions command
Package main demonstrates transactions in RockyardKV.
Package main demonstrates transactions in RockyardKV.
internal
batch
Package batch implements the WriteBatch format for atomic writes.
Package batch implements the WriteBatch format for atomic writes.
blob
Package blob implements BlobDB - separated value storage for large values.
Package blob implements BlobDB - separated value storage for large values.
block
builder.go implements block building with prefix compression.
builder.go implements block building with prefix compression.
cache
Package cache provides caching implementations for RockyardKV.
Package cache provides caching implementations for RockyardKV.
campaign
artifact.go defines run artifacts, campaign summaries, and persistence logic.
artifact.go defines run artifacts, campaign summaries, and persistence logic.
checksum
Package checksum provides checksum algorithms compatible with RocksDB.
Package checksum provides checksum algorithms compatible with RocksDB.
compaction
Package compaction implements the compaction logic for RockyardKV.
Package compaction implements the compaction logic for RockyardKV.
compression
Package compression provides compression and decompression for RocksDB blocks.
Package compression provides compression and decompression for RocksDB blocks.
dbformat
Package dbformat provides the internal key format used by RocksDB.
Package dbformat provides the internal key format used by RocksDB.
encoding
Package encoding provides binary encoding/decoding primitives that are bit-compatible with RocksDB's util/coding.h implementation.
Package encoding provides binary encoding/decoding primitives that are bit-compatible with RocksDB's util/coding.h implementation.
filter
Package filter implements Bloom filters for SST files.
Package filter implements Bloom filters for SST files.
flush
Package flush implements the flush operation that writes memtable to SST files.
Package flush implements the flush operation that writes memtable to SST files.
iterator
Package iterator provides iterator implementations for RockyardKV.
Package iterator provides iterator implementations for RockyardKV.
logging
Package logging provides the logging interface and default implementations for RockyardKV.
Package logging provides the logging interface and default implementations for RockyardKV.
manifest
Package manifest provides encoding and decoding for RocksDB MANIFEST files.
Package manifest provides encoding and decoding for RocksDB MANIFEST files.
mempool
Package mempool provides memory pooling utilities.
Package mempool provides memory pooling utilities.
memtable
Package memtable implements in-memory sorted data structures for RocksDB.
Package memtable implements in-memory sorted data structures for RocksDB.
options
Package options implements OPTIONS file parsing for database configuration.
Package options implements OPTIONS file parsing for database configuration.
rangedel
fragmenter.go implements range tombstone fragmentation.
fragmenter.go implements range tombstone fragmentation.
table
Package table provides SST file reading and writing.
Package table provides SST file reading and writing.
testutil
Artifact bundle for test reproducibility.
Artifact bundle for test reproducibility.
trace
reader.go implements trace file reading for operation replay.
reader.go implements trace file reading for operation replay.
txn
version
builder.go implements VersionBuilder for applying edits to versions.
builder.go implements VersionBuilder for applying edits to versions.
wal
Package wal provides Write-Ahead Log (WAL) reader and writer implementations that are bit-compatible with RocksDB's log format.
Package wal provides Write-Ahead Log (WAL) reader and writer implementations that are bit-compatible with RocksDB's log format.
Package vfs provides filesystem abstractions including fault injection for testing.
Package vfs provides filesystem abstractions including fault injection for testing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL