devicering

package module
v0.0.0-...-f1fb307 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 6, 2018 License: BSD-3-Clause Imports: 16 Imported by: 1

README

Ring

Development Repository

COPY OF github.com/gholt/ring back before the major refactors of that package. This devicering package is for use with github.com/gholt/msgring and github.com/gholt/store etc. until such time as I can update it all to work with the new ring ideas.

Experimental: No stable version of this package yet exists; it is still in early development.

If you're not entirely sure what consistent hashing is, reading Basic Hash Ring might help.

Package ring contains tools for building and using a consistent hashing ring with replicas, automatic partitioning (ring ranges), and keeping replicas of the same partitions in as distinct tiered nodes as possible (tiers might be devices, servers, cabinets, rooms, data centers, geographical regions, etc.)

Here's a quick example of building a ring and discovering what items are assigned to what nodes:

package main

import (
    "fmt"
    "hash/fnv"

    "github.com/gholt/ring"
)

func main() {
    // Note that we're ignoring errors for the purpose of a shorter example.
    // The 64 indicates how many bits can be used in a uint64 for node IDs;
    // 64 is fine unless you have a specific use case.
    builder := ring.NewBuilder(64)
    // (active, capacity, no tiers, no addresses, meta, no conf)
    builder.AddNode(true, 1, nil, nil, "NodeA", nil)
    builder.AddNode(true, 1, nil, nil, "NodeB", nil)
    builder.AddNode(true, 1, nil, nil, "NodeC", nil)
    // This rebalances if necessary and provides a usable Ring instance.
    ring := builder.Ring()
    // This value indicates how many bits are in use for determining ring
    // partitions.
    partitionBitCount := ring.PartitionBitCount()
    for _, item := range []string{"First", "Second", "Third"} {
        // We're using fnv hashing here, but you can use whatever you like.
        // We don't actually recommend fnv, but it's useful for this example.
        hasher := fnv.New64a()
        hasher.Write([]byte(item))
        partition := uint32(hasher.Sum64() >> (64 - partitionBitCount))
        // We can just grab the first node since this example just uses one
        // replica. See Builder.SetReplicaCount for more information.
        node := ring.ResponsibleNodes(partition)[0]
        fmt.Printf("%s is handled by %v\n", item, node.Meta())
    }
}

The output would be:

First is handled by NodeC
Second is handled by NodeB
Third is handled by NodeB

API Documentation
Basic Hash Ring
Partition Ring vs. Hash Ring

Other interesting ideas in this space:
Jump consistent hashing - dgryski implementation also dgryski shared key-value store
Multi-probe consistent hashing - dgryski implementation
GreenCHT replication scheme

This is the latest development area for the package.
Eventually a stable version of the package will be established but, for now, all things about this package are subject to change.

Copyright See AUTHORS. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.

Documentation

Overview

Package devicering contains tools for building and using a consistent hashing ring with replicas, automatic partitioning (ring ranges), and keeping replicas of the same partitions in as distinct tiered nodes as possible (tiers might be devices, servers, cabinets, rooms, data centers, geographical regions, etc.)

It also contains tools for using a ring as a messaging hub, easing communication between nodes in the ring.

Here's a quick example of building a ring and discovering what items are assigned to what nodes:

package main

import (
    "fmt"
    "hash/fnv"

    "github.com/gholt/ring"
)

func main() {
    // Note that we're ignoring errors for the purpose of a shorter example.
    // The 64 indicates how many bits can be used in a uint64 for node IDs;
    // 64 is fine unless you have a specific use case.
    builder := ring.NewBuilder(64)
    // (active, capacity, no tiers, no addresses, meta, no config)
    builder.AddNode(true, 1, nil, nil, "NodeA", nil)
    builder.AddNode(true, 1, nil, nil, "NodeB", nil)
    builder.AddNode(true, 1, nil, nil, "NodeC", nil)
    // This rebalances if necessary and provides a usable Ring instance.
    ring := builder.Ring()
    // This value indicates how many bits are in use for determining ring
    // partitions.
    partitionBitCount := ring.PartitionBitCount()
    for _, item := range []string{"First", "Second", "Third"} {
        // We're using fnv hashing here, but you can use whatever you like.
        // We don't actually recommend fnv, but it's useful for this example.
        hasher := fnv.New64a()
        hasher.Write([]byte(item))
        partition := uint32(hasher.Sum64() >> (64 - partitionBitCount))
        // We can just grab the first node since this example just uses one
        // replica. See Builder.SetReplicaCount for more information.
        node := ring.ResponsibleNodes(partition)[0]
        fmt.Printf("%s is handled by %v\n", item, node.Meta())
    }
}

The output would be:

First is handled by NodeC
Second is handled by NodeB
Third is handled by NodeB

Index

Constants

View Source
const (
	// BUILDERVERSION is the builder file format version written to and checked
	// for in the builder file header. If the on disk format of the builder changes
	// this version should be incremented.
	BUILDERVERSION = "RINGBUILDERv0001"
)
View Source
const RINGVERSION = "RINGv00000000001"

RINGVERSION is the ring file format version written to and checked for in the ring file header. If the on disk format of the ring changes this version should be incremented.

Variables

This section is empty.

Functions

func CLI

func CLI(args []string, output io.Writer, ansiColor bool) error

CLI is the "ring" command line interface; it's included in the ring package itself so you can easily stub it to whatever executable name you might want, with code like:

package main

import (
    "fmt"
    "os"

    "github.com/gholt/ring"
    "github.com/gholt/brimtext"
)

func main() {
    if err := ring.CLI(os.Args, os.Stdout, false); err != nil {
        fmt.Fprintln(os.Stderr, brimtext.Sentence(err.Error()))
        os.Exit(1)
    }
}

The individual subcommands are also included in the ring package, prefixed with CLI, so you can build your command line interface with just the commands you want, or leverage them to build a web interface, etc. The ansiColor value indicates if you'd like ANSI color escape sequences embedded in the output.

func CLIAddOrSet

func CLIAddOrSet(b *Builder, args []string, n BuilderNode, output io.Writer) error

CLIAddOrSet adds a new node or updates an existing node; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLIConfig

func CLIConfig(r Ring, b *Builder, args []string, output io.Writer) (changed bool, err error)

CLIConfig displays or sets the top-level config in the ring or builder; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLIConfigFile

func CLIConfigFile(r Ring, b *Builder, args []string, output io.Writer) (changed bool, err error)

CLIConfigFile displays or sets the top-level config in the ring or builder; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLICreate

func CLICreate(filename string, args []string, output io.Writer) error

CLICreate creates a new builder; see the output of CLIHelp for detailed information.

func CLIHelp

func CLIHelp(args []string, output io.Writer, ansiColor bool, markdown bool) error

CLIHelp outputs the help text for the default CLI. The ansiColor value indicates if you'd like ANSI color escape sequences embedded in the output. The markdown value indicates if you'd like the raw Markdown help text instead of the processed text.

func CLIInfo

func CLIInfo(r Ring, b *Builder, output io.Writer) error

CLIInfo outputs information about the ring or builder such as node count, partition count, etc.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLINode

func CLINode(r Ring, b *Builder, args []string, full bool, output io.Writer) (changed bool, err error)

CLINode outputs a list of nodes in the ring or builder, with optional filtering and also allows setting attributes on those nodes; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLINodeReport

func CLINodeReport(n Node) string

func CLIPartition

func CLIPartition(r Ring, b *Builder, args []string, output io.Writer) error

CLIPartition outputs information about a partition's node assignments; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLIPretendElapsed

func CLIPretendElapsed(b *Builder, args []string, output io.Writer) error

CLIPretendElapsed updates a builder, pretending some time has elapsed for testing purposes; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLIRemove

func CLIRemove(b *Builder, args []string, output io.Writer) error

CLIRemove removes a node from the builder; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLIRing

func CLIRing(b *Builder, filename string, output io.Writer) error

CLIRing writes a new ring file based on the builder; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func CLISetReplicas

func CLISetReplicas(b *Builder, args []string, output io.Writer) error

CLISetReplicas changes the replica count for the builder; see the output of CLIHelp for detailed information.

func CLITier

func CLITier(r Ring, b *Builder, args []string, output io.Writer) error

CLITier outputs a list of tiers in the ring or builder; see the output of CLIHelp for detailed information.

Provide either the ring or the builder, but not both; set the other to nil. Normally the results from RingOrBuilder.

func PersistRingOrBuilder

func PersistRingOrBuilder(r Ring, b *Builder, filename string) error

PersistRingOrBuilder persists a given ring/builder to the provided filename

func RingOrBuilder

func RingOrBuilder(fileName string) (Ring, *Builder, error)

RingOrBuilder attempts to determine whether a file is a Ring or Builder file and then loads it accordingly.

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder is used to construct Rings over time. Rings are the immutable state of a Builder's assignments at a given point in time.

func LoadBuilder

func LoadBuilder(r io.Reader) (*Builder, error)

LoadBuilder creates a new Builder instance based on the persisted data from the Reader (presumably previously saved with the Persist method).

func NewBuilder

func NewBuilder(idBits int) *Builder

NewBuilder creates an empty Builder with all default settings.

idBits indicates how many bits (1-64) may be used for node IDs; it must be set at Builder creation and cannot change once created.

func (*Builder) AddNode

func (b *Builder) AddNode(active bool, capacity uint32, tiers []string, addresses []string, meta string, config []byte) (BuilderNode, error)

AddNode will add a new node to the builder for data assigment. Actual data assignment won't ocurr until the Ring method is called, so you can add multiple nodes or alter node values after creation if desired.

func (*Builder) Config

func (b *Builder) Config() []byte

Config is the raw encoded global configuration.

func (*Builder) IDBits

func (b *Builder) IDBits() int

IDBits is the number of bits in use for node IDs.

func (*Builder) MaxPartitionBitCount

func (b *Builder) MaxPartitionBitCount() uint16

MaxPartitionBitCount caps how large the ring can grow. The default is 23, which means 2**23 or 8,388,608 partitions, which is about 100M for a 3 replica ring (each partition replica assignment is an int32).

func (*Builder) MoveWait

func (b *Builder) MoveWait() uint16

MoveWait is the number of minutes that should elapse before reassigning a replica of a partition again.

func (*Builder) Node

func (b *Builder) Node(nodeID uint64) BuilderNode

Node returns the node instance identified, if there is one.

func (*Builder) Nodes

func (b *Builder) Nodes() NodeSlice

Nodes returns a NodeSlice of the nodes the Builder references, but each Node in the slice can be typecast into a BuilderNode if needed.

func (*Builder) Persist

func (b *Builder) Persist(w io.Writer) error

Persist saves the Builder state to the given Writer for later reloading via the LoadBuilder method.

func (*Builder) PointsAllowed

func (b *Builder) PointsAllowed() byte

PointsAllowed is the number of percentage points over or under that the ring will try to keep data assignments within. The default is 1 for one percent extra or less data.

func (*Builder) PretendElapsed

func (b *Builder) PretendElapsed(minutes uint16)

PretendElapsed shifts the last movement records by the number of minutes given. This can be useful in testing, as the ring algorithms will not reassign replicas for a partition more often than once per MoveWait in order to let reassignments take effect before moving the same data yet again.

func (*Builder) RemoveNode

func (b *Builder) RemoveNode(nodeID uint64)

RemoveNode will remove the node from the list of nodes for this builder/ring. Note that this can be relatively expensive as all nodes that had been added after the removed node had been originally added will have their internal indexes shifted down one and all the replica-to-partition-to-node indexing will have to be updated, as well as clearing any assignments that were to the removed node. Normally it is better to just leave a "dead" node in place and simply set it as inactive.

func (*Builder) ReplicaCount

func (b *Builder) ReplicaCount() int

func (*Builder) Ring

func (b *Builder) Ring() Ring

Ring returns a Ring instance of the data defined by the builder. This will cause any pending rebalancing actions to be performed. The Ring returned will be immutable; to obtain updated ring data, Ring() must be called again.

func (*Builder) SetConfig

func (b *Builder) SetConfig(config []byte)

func (*Builder) SetMaxPartitionBitCount

func (b *Builder) SetMaxPartitionBitCount(count uint16)

func (*Builder) SetMoveWait

func (b *Builder) SetMoveWait(minutes uint16)

func (*Builder) SetPointsAllowed

func (b *Builder) SetPointsAllowed(points byte)

func (*Builder) SetReplicaCount

func (b *Builder) SetReplicaCount(count int)

func (*Builder) Tiers

func (b *Builder) Tiers() [][]string

Tiers returns the tier values in use at each level. Note that an empty string is always an available value at any level, although it is not returned from this method.

type BuilderNode

type BuilderNode interface {
	Node
	SetActive(value bool)
	SetCapacity(value uint32)
	SetTier(level int, value string)
	ReplaceTiers(tiers []string)
	SetAddress(index int, value string)
	ReplaceAddresses(addrs []string)
	SetMeta(value string)
	SetConfig(config []byte)
}

BuilderNode extends Node to allow for updating attributes. A Ring needs immutable nodes as the assignments are static at that point, but the Builder doesn't have that restriction.

type Node

type Node interface {
	// ID uniquely identifies this node; it will be non-zero as zero is used to
	// indicate "no node".
	ID() uint64
	// Active indicates whether the node should be in use or not. Nodes may be
	// deactivated for a while (during a maintenance, for example) and then
	// reactivated later. While deactivated, the builder will reassign all data
	// previously assigned to the node.
	Active() bool
	// Capacity indicates the amount of data that should be assigned to a node
	// relative to other nodes. It can be in any unit of designation as long as
	// all nodes use the same designation. Most commonly this is the number of
	// gigabytes the node can store, but could be based on CPU capacity or
	// another resource if that makes more sense to balance.
	Capacity() uint32
	// Tiers indicate the layout of the node with respect to other nodes. For
	// example, the lowest tier, tier 0, might be the server ip (where each
	// node represents a drive on that server). The next tier, 1, might then be
	// the power zone the server is in. The number of tiers is flexible, so
	// later an additional tier for geographic region could be added.
	Tiers() []string
	// Tier returns just the single tier value for the level.
	Tier(level int) string
	// Addresses give location information for the node; probably something
	// like ip:port. This is a list for those use cases where different
	// processes use different networks (such as replication using a
	// replication-only network).
	Addresses() []string
	// Address returns just the single address for the index.
	Address(index int) string
	// Meta is additional information for the node; not defined or used by the
	// builder or ring directly.
	Meta() string
	// Config contains the raw configuration bytes for this node.
	Config() []byte
}

Node represents an endpoint for ring data or other ring-based services.

type NodeSlice

type NodeSlice []Node

func (NodeSlice) Filter

func (ns NodeSlice) Filter(filters []string) (NodeSlice, error)

Filter will return a new NodeSlice with just the nodes that match the filters given; multiple filters are ANDed. The basic filter syntax is that "attribute=value" will filter to just nodes whose attribute exactly match the value and "attribute~=value" will similarly filter but treat the value as a regular expression (per the http://golang.org/pkg/regexp/ implementation). The available attributes to filter on are:

id          A node's id (uint64 represented as %d).
active      Whether a node is active or not (use "true" or "false").
capacity    A node's capacity.
tier        Any tier of a node.
tierX       A node's specific tier level specified by X.
address     Any address of a node.
addressX    A node's specific address index specified by X.
meta        A node's meta attribute.

For example:

ring.Nodes().Filter([]string{"active=true", `address=10\.1\.2\..*`})

type Ring

type Ring interface {
	// Version is the time.Now().UnixNano() of when the Ring data was
	// established.
	//
	// Version can indicate changes in ring data; for example, if a server is
	// currently working with one version of ring data and receives requests
	// that are based on a lesser version of ring data, it can ignore those
	// requests or send an "obsoleted" response or something along those lines.
	// Similarly, if the server receives requests for a greater version of ring
	// data, it can ignore those requests or try to obtain a newer ring
	// version.
	Version() int64
	// Config returns the raw encoded global configuration. This configuration
	// data isn't used by the ring itself, but can be useful in storing
	// configuration data for users of the ring.
	Config() []byte
	// Node returns the node instance identified, if there is one.
	Node(nodeID uint64) Node
	// Nodes returns a NodeSlice of the nodes the Ring references.
	Nodes() NodeSlice
	// NodeCount returns the number of nodes the Ring references.
	NodeCount() int
	// Tiers returns the tier values in use at each level. Note that an empty
	// string is always an available value at any level, although it is not
	// returned from this method.
	Tiers() [][]string
	// PartitionBitCount is the number of bits that can be used to determine a
	// partition number for the current data in the ring. For example, to
	// convert a uint64 hash value into a partition number you could use
	// hashValue >> (64 - ring.PartitionBitCount()). The PartitionBitCount also
	// indicates how many partitions the Ring has; for example, a value of 16
	// would indicate 2**16 or 65,536 partitions.
	PartitionBitCount() uint16
	// ReplicaCount specifies how many replicas the Ring has.
	ReplicaCount() int
	// LocalNode returns the node the ring is locally bound to, if any. This
	// local node binding is used by things such as MsgRing to know what items
	// are bound for the local instance or need to be sent to remote ones, etc.
	LocalNode() Node
	// SetLocalNode sets the node the ring is locally bound to, if any. This
	// local node binding is used by things such as MsgRing to know what items
	// are bound for the local instance or need to be sent to remote ones, etc.
	SetLocalNode(nodeID uint64)
	// Responsible will return true if LocalNode is set and one of the
	// partition's replicas is assigned to that local node.
	//
	// Note that the partition value is not bounds checked; an invalid
	// partition will cause a panic. See the documentation for the Ring
	// interface itself for further discussion.
	Responsible(partition uint32) bool
	// ResponsibleReplica will return the replica index >= 0 if LocalNode is
	// set and one of the partition's replicas is assigned to that local node;
	// it will return -1 if LocalNode is not responsible for the partition.
	//
	// Note that the partition value is not bounds checked; an invalid
	// partition will cause a panic. See the documentation for the Ring
	// interface itself for further discussion.
	ResponsibleReplica(partition uint32) int
	// ResponsibleNodes will return the list of nodes that are responsible for
	// the replicas of the partition.
	//
	// Note that the partition value is not bounds checked; an invalid
	// partition will cause a panic. See the documentation for the Ring
	// interface itself for further discussion.
	ResponsibleNodes(partition uint32) NodeSlice
	// Stats gives information about the ring and its health; the MaxUnder and
	// MaxOver values specifically indicate how balanced the ring is.
	Stats() *Stats
	// Persist saves the Ring state to the given Writer for later reloading via
	// the LoadRing method.
	Persist(w io.Writer) error
}

Ring is the immutable snapshot of data assignments to nodes.

The immutable characteristic is important, as it means code that uses a Ring can make calculations around its attributes and not worry about those attributes changing from call to call. For example, the PartitionBitCount can be used to generate a list of partitions that are then used with ResponsibleNodes without worrying the PartitionBitCount changed during the calculations. Changes to the Ring are done by the Builder, which will generate a new immutable Ring instance to be distributed to the users of the older Ring instance. There is one exception, however; the LocalNode attribute is mutable as its function is to represent the local user of that Ring instance.

Note that with several methods the partition value is not bounds checked; an invalid partition will cause a panic. This behavior is for speed reasons, as bounds checking every call would be wasteful in most use cases that already guarantee proper bounding. You could easily create a BoundedRing that wraps a Ring instance to provide such bounding if you really need it; but considering most uses of partitions involve bit shifting based on the PartitionBitCount, such bounding doesn't seem worth it.

func LoadRing

func LoadRing(rd io.Reader) (Ring, error)

LoadRing creates a new Ring instance based on the persisted data from the Reader (presumably previously saved with the Ring.Persist method).

type Stats

type Stats struct {
	ReplicaCount      int
	ActiveNodeCount   int
	InactiveNodeCount int
	PartitionBitCount uint16
	PartitionCount    int
	ActiveCapacity    uint64
	InactiveCapacity  uint64
	// MaxUnderNodePercentage is the percentage a node is underweight, or has
	// less data assigned to it than its capacity would indicate it desires.
	MaxUnderNodePercentage float64
	MaxUnderNodeID         uint64
	// MaxOverNodePercentage is the percentage a node is overweight, or has
	// more data assigned to it than its capacity would indicate it desires.
	MaxOverNodePercentage float64
	MaxOverNodeID         uint64
}

Stats gives an overview of the state and health of a Ring. It is returned by the Ring.Stats() method.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL