memdb

package module
v0.0.0-...-d2d2b77 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 7, 2017 License: MPL-2.0 Imports: 9 Imported by: 0

README

go-memdb

Provides the memdb package that implements a simple in-memory database built on immutable radix trees. The database provides Atomicity, Consistency and Isolation from ACID. Being that it is in-memory, it does not provide durability. The database is instantiated with a schema that specifies the tables and indicies that exist and allows transactions to be executed.

The database provides the following:

  • Multi-Version Concurrency Control (MVCC) - By leveraging immutable radix trees the database is able to support any number of concurrent readers without locking, and allows a writer to make progress.

  • Transaction Support - The database allows for rich transactions, in which multiple objects are inserted, updated or deleted. The transactions can span multiple tables, and are applied atomically. The database provides atomicity and isolation in ACID terminology, such that until commit the updates are not visible.

  • Rich Indexing - Tables can support any number of indexes, which can be simple like a single field index, or more advanced compound field indexes. Certain types like UUID can be efficiently compressed from strings into byte indexes for reduced storage requirements.

For the underlying immutable radix trees, see go-immutable-radix.

Documentation

The full documentation is available on Godoc.

Example

Below is a simple example of usage

// Create a sample struct
type Person struct {
    Email string
    Name  string
    Age   int
}

// Create the DB schema
schema := &memdb.DBSchema{
    Tables: map[string]*memdb.TableSchema{
        "person": &memdb.TableSchema{
            Name: "person",
            Indexes: map[string]*memdb.IndexSchema{
                "id": &memdb.IndexSchema{
                    Name:    "id",
                    Unique:  true,
                    Indexer: &memdb.StringFieldIndex{Field: "Email"},
                },
            },
        },
    },
}

// Create a new data base
db, err := memdb.NewMemDB(schema)
if err != nil {
    panic(err)
}

// Create a write transaction
txn := db.Txn(true)

// Insert a new person
p := &Person{"joe@aol.com", "Joe", 30}
if err := txn.Insert("person", p); err != nil {
    panic(err)
}

// Commit the transaction
txn.Commit()

// Create read-only transaction
txn = db.Txn(false)
defer txn.Abort()

// Lookup by email
raw, err := txn.First("person", "id", "joe@aol.com")
if err != nil {
    panic(err)
}

// Say hi!
fmt.Printf("Hello %s!", raw.(*Person).Name)

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var MapType = reflect.MapOf(reflect.TypeOf(""), reflect.TypeOf("")).Kind()

Functions

This section is empty.

Types

type CompoundIndex 

type CompoundIndex struct {
	Indexes []Indexer

	// AllowMissing results in an index based on only the indexers
	// that return data. If true, you may end up with 2/3 columns
	// indexed which might be useful for an index scan. Otherwise,
	// the CompoundIndex requires all indexers to be satisfied.
	AllowMissing bool
}

CompoundIndex is used to build an index using multiple sub-indexes Prefix based iteration is supported as long as the appropriate prefix of indexers support it. All sub-indexers are only assumed to expect a single argument.

func (*CompoundIndex) FromArgs 

func (c *CompoundIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*CompoundIndex) FromObject 

func (c *CompoundIndex) FromObject(raw interface{}) (bool, []byte, error)

func (*CompoundIndex) PrefixFromArgs 

func (c *CompoundIndex) PrefixFromArgs(args ...interface{}) ([]byte, error)

type ConditionalIndex 

type ConditionalIndex struct {
	Conditional ConditionalIndexFunc
}

ConditionalIndex builds an index based on a condition specified by a passed user function. This function may examine the passed object and return a boolean to encapsulate an arbitrarily complex conditional.

func (*ConditionalIndex) FromArgs 

func (c *ConditionalIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*ConditionalIndex) FromObject 

func (c *ConditionalIndex) FromObject(obj interface{}) (bool, []byte, error)

type ConditionalIndexFunc 

type ConditionalIndexFunc func(obj interface{}) (bool, error)

ConditionalIndexFunc is the required function interface for a ConditionalIndex.

type DBSchema 

type DBSchema struct {
	Tables map[string]*TableSchema
}

DBSchema contains the full database schema used for MemDB

func (*DBSchema) Validate 

func (s *DBSchema) Validate() error

Validate is used to validate the database schema

type FieldSetIndex 

type FieldSetIndex struct {
	Field string
}

FieldSetIndex is used to extract a field from an object using reflection and builds an index on whether the field is set by comparing it against its type's nil value.

func (*FieldSetIndex) FromArgs 

func (f *FieldSetIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*FieldSetIndex) FromObject 

func (f *FieldSetIndex) FromObject(obj interface{}) (bool, []byte, error)

type IndexSchema 

type IndexSchema struct {
	Name         string
	AllowMissing bool
	Unique       bool
	Indexer      Indexer
}

IndexSchema contains the schema for an index

func (*IndexSchema) Validate 

func (s *IndexSchema) Validate() error

type Indexer 

type Indexer interface {
	// ExactFromArgs is used to build an exact index lookup
	// based on arguments
	FromArgs(args ...interface{}) ([]byte, error)
}

Indexer is an interface used for defining indexes

type MemDB 

type MemDB struct {
	// contains filtered or unexported fields
}

MemDB is an in-memory database. It provides a table abstraction, which is used to store objects (rows) with multiple indexes based on values. The database makes use of immutable radix trees to provide transactions and MVCC.

func NewMemDB 

func NewMemDB(schema *DBSchema) (*MemDB, error)

NewMemDB creates a new MemDB with the given schema

func (*MemDB) Snapshot 

func (db *MemDB) Snapshot() *MemDB

Snapshot is used to capture a point-in-time snapshot of the database that will not be affected by any write operations to the existing DB.

func (*MemDB) Txn 

func (db *MemDB) Txn(write bool) *Txn

Txn is used to start a new transaction, in either read or write mode. There can only be a single concurrent writer, but any number of readers.

type MultiIndexer 

type MultiIndexer interface {
	// FromObject is used to extract index values from an
	// object or to indicate that the index value is missing.
	FromObject(raw interface{}) (bool, [][]byte, error)
}

MultiIndexer is an interface used for defining indexes generating multiple entries per object

type PrefixIndexer 

type PrefixIndexer interface {
	// PrefixFromArgs returns a prefix that should be used
	// for scanning based on the arguments
	PrefixFromArgs(args ...interface{}) ([]byte, error)
}

PrefixIndexer can optionally be implemented for any indexes that support prefix based iteration. This may not apply to all indexes.

type ResultIterator 

type ResultIterator interface {
	Next() interface{}
}

ResultIterator is used to iterate over a list of results from a Get query on a table.

type SingleIndexer 

type SingleIndexer interface {
	// FromObject is used to extract an index value from an
	// object or to indicate that the index value is missing.
	FromObject(raw interface{}) (bool, []byte, error)
}

SingleIndexer is an interface used for defining indexes generating a single entry per object

type StringFieldIndex 

type StringFieldIndex struct {
	Field     string
	Lowercase bool
}

StringFieldIndex is used to extract a field from an object using reflection and builds an index on that field.

func (*StringFieldIndex) FromArgs 

func (s *StringFieldIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*StringFieldIndex) FromObject 

func (s *StringFieldIndex) FromObject(obj interface{}) (bool, []byte, error)

func (*StringFieldIndex) PrefixFromArgs 

func (s *StringFieldIndex) PrefixFromArgs(args ...interface{}) ([]byte, error)

type StringMapFieldIndex 

type StringMapFieldIndex struct {
	Field     string
	Lowercase bool
}

StringMapFieldIndex is used to extract a field of type map[string]string from an object using reflection and builds an index on that field.

func (*StringMapFieldIndex) FromArgs 

func (s *StringMapFieldIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*StringMapFieldIndex) FromObject 

func (s *StringMapFieldIndex) FromObject(obj interface{}) (bool, [][]byte, error)

type StringSliceFieldIndex 

type StringSliceFieldIndex struct {
	Field     string
	Lowercase bool
}

StringSliceFieldIndex is used to extract a field from an object using reflection and builds an index on that field.

func (*StringSliceFieldIndex) FromArgs 

func (s *StringSliceFieldIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*StringSliceFieldIndex) FromObject 

func (s *StringSliceFieldIndex) FromObject(obj interface{}) (bool, [][]byte, error)

func (*StringSliceFieldIndex) PrefixFromArgs 

func (s *StringSliceFieldIndex) PrefixFromArgs(args ...interface{}) ([]byte, error)

type TableSchema 

type TableSchema struct {
	Name    string
	Indexes map[string]*IndexSchema
}

TableSchema contains the schema for a single table

func (*TableSchema) Validate 

func (s *TableSchema) Validate() error

Validate is used to validate the table schema

type Txn 

type Txn struct {
	// contains filtered or unexported fields
}

Txn is a transaction against a MemDB. This can be a read or write transaction.

func (*Txn) Abort 

func (txn *Txn) Abort()

Abort is used to cancel this transaction. This is a noop for read transactions.

func (*Txn) Commit 

func (txn *Txn) Commit()

Commit is used to finalize this transaction. This is a noop for read transactions.

func (*Txn) Defer 

func (txn *Txn) Defer(fn func())

Defer is used to push a new arbitrary function onto a stack which gets called when a transaction is committed and finished. Deferred functions are called in LIFO order, and only invoked at the end of write transactions.

func (*Txn) Delete 

func (txn *Txn) Delete(table string, obj interface{}) error

Delete is used to delete a single object from the given table This object must already exist in the table

func (*Txn) DeleteAll 

func (txn *Txn) DeleteAll(table, index string, args ...interface{}) (int, error)

DeleteAll is used to delete all the objects in a given table matching the constraints on the index

func (*Txn) First 

func (txn *Txn) First(table, index string, args ...interface{}) (interface{}, error)

First is used to return the first matching object for the given constraints on the index

func (*Txn) Get 

func (txn *Txn) Get(table, index string, args ...interface{}) (ResultIterator, error)

Get is used to construct a ResultIterator over all the rows that match the given constraints of an index.

func (*Txn) Insert 

func (txn *Txn) Insert(table string, obj interface{}) error

Insert is used to add or update an object into the given table

func (*Txn) LongestPrefix 

func (txn *Txn) LongestPrefix(table, index string, args ...interface{}) (interface{}, error)

LongestPrefix is used to fetch the longest prefix match for the given constraints on the index. Note that this will not work with the memdb StringFieldIndex because it adds null terminators which prevent the algorithm from correctly finding a match (it will get to right before the null and fail to find a leaf node). This should only be used where the prefix given is capable of matching indexed entries directly, which typically only applies to a custom indexer. See the unit test for an example.

type UUIDFieldIndex 

type UUIDFieldIndex struct {
	Field string
}

UUIDFieldIndex is used to extract a field from an object using reflection and builds an index on that field by treating it as a UUID. This is an optimization to using a StringFieldIndex as the UUID can be more compactly represented in byte form.

func (*UUIDFieldIndex) FromArgs 

func (u *UUIDFieldIndex) FromArgs(args ...interface{}) ([]byte, error)

func (*UUIDFieldIndex) FromObject 

func (u *UUIDFieldIndex) FromObject(obj interface{}) (bool, []byte, error)

func (*UUIDFieldIndex) PrefixFromArgs 

func (u *UUIDFieldIndex) PrefixFromArgs(args ...interface{}) ([]byte, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.