kelips

package module
v0.0.0-...-8b64a5b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 1, 2018 License: MPL-2.0 Imports: 25 Imported by: 0

README

go-kelips

go-kelips implements the kelips DHT in golang.

It has a pluggable design with the following available interfaces:

  • Transport
  • ContactStorage
  • TupleStorage

A gossip layer has also been included. Its design is to augment the core kelips node as to not be invasive and allow the gossip layer to be pluggable as well.

Documentation

Overview

Package kelips provides a kelips DHT implementation in go

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AffinityGroup

type AffinityGroup interface {
	// IsLocal returns true if this node belongs the affinity group
	IsLocal() bool
	// Contact returns this nodes group contact information
	Contact() GroupContact
	// Insert a key into the affinity group returning the homenode
	Insert(key []byte) (string, error)
	// Lookup a key returning the homenode
	Lookup(*Request) (string, error)
	// Add a peer to the group
	AddPeer(peer PeerContact) error
	// Remove a peer from the group
	RemovePeer(peer PeerContact) error
	// Starts all go-routines for the group
	Start()
}

AffinityGroup implements a kelips affinity group

type Config

type Config struct {
	K                 int64                 // Number of affinity groups
	HashFunc          func() hash.Hash      // Hash function
	TupleTTL          time.Duration         // TTL from last seen before removing
	TupleExpireMinInt time.Duration         // Interval min to check for expirations
	TupleExpireMaxInt time.Duration         // Interval max to check for expirations
	Transport         Transport             // Network transport
	Tuples            TupleStorage          // Tuple store
	Contacts          ContactStorageFactory // Contact store
	Logger            *log.Logger
}

Config holds the kelips config to init a new instance

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns a sane default Kelips config

func (*Config) Validate

func (conf *Config) Validate()

Validate validates the config and sets defaults

type ContactStorage

type ContactStorage interface {
	// Adds a PeerContact to the affinity group
	Add(PeerContact) error
	// Remove a peer from the store
	Remove(PeerContact) error
	// List returns all known peers in the affinity group
	List() []PeerContact
	// GetClosest returns the closest node to the querying node
	GetClosest() (PeerContact, bool)
	// GetRandom returns a random peer from the storage
	GetRandom() (PeerContact, bool)
}

ContactStorage stores peer contact information.

type ContactStorageFactory

type ContactStorageFactory interface {
	// New should return a new ContactStorage.  This is called once per
	// affinity group. Home is true if the contact store will be used for
	// the local affinity group
	New(id int64, home bool) ContactStorage
}

ContactStorageFactory implements an interface to create contact storages for groups

type Gossip

type Gossip struct {
	// contains filtered or unexported fields
}

Gossip is the gossip component of kelips. It handles all contact and tuple liveliness, pinging as messaging arrive on the gossip network

func NewGossip

func NewGossip(kconf *Config, conf *gossip.Config) (*Gossip, error)

NewGossip returns a new Gossip instance

func (*Gossip) Join

func (st *Gossip) Join(peers ...string) (int, error)

Join joins the inter-group gossip pool and the home gossip group assuming a home node has been provided

func (*Gossip) ListenMux

func (st *Gossip) ListenMux(m uint16) (net.Listener, error)

ListenMux returns a muxed listener with the given id. This must be called before starting gossip

func (*Gossip) ListenTCP

func (st *Gossip) ListenTCP() net.Listener

ListenTCP returns a non-muxed native tcp listener

func (*Gossip) New

func (st *Gossip) New(id int64, homeNode bool) ContactStorage

New returns a new contact store backed by gossip depending on whether this is a home group. It implements the ContactStorageFactory interface

func (*Gossip) Register

func (st *Gossip) Register(k *Kelips) error

Register registers the kelips instance and starts all go-routines. This should be the last call once eveything has been initialized. Join can be called after this one

type GroupContact

type GroupContact struct {
	ID   int64  // Group id
	Host string // Host
}

GroupContact is a contact within a group

func (GroupContact) String

func (gc GroupContact) String() string

type HTTPTransport

type HTTPTransport struct {
	// contains filtered or unexported fields
}

HTTPTransport implements a HTTP based Transport interface

func NewHTTPTransport

func NewHTTPTransport(enableMagic bool) *HTTPTransport

NewHTTPTransport returns a new HTTPTransport. If enableMagic is true, a muxed client with the magic number is used instead of the default

func (*HTTPTransport) AddPeer

func (trans *HTTPTransport) AddPeer(contact GroupContact, host PeerContact) error

AddPeer makes a remote request to add a peer to a group

func (*HTTPTransport) Insert

func (trans *HTTPTransport) Insert(contact GroupContact, key []byte) (string, error)

Insert key at remote group

func (*HTTPTransport) Lookup

func (trans *HTTPTransport) Lookup(contact GroupContact, r *Request) (string, error)

Lookup should return the home node of the key

func (*HTTPTransport) Register

func (trans *HTTPTransport) Register(contact GroupContact, group AffinityGroup)

Register the affinity group with the transport

func (*HTTPTransport) ServeHTTP

func (trans *HTTPTransport) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*HTTPTransport) Shutdown

func (trans *HTTPTransport) Shutdown(ctx context.Context) error

Shutdown gracefully shuts down the transport

func (*HTTPTransport) Start

func (trans *HTTPTransport) Start(ln net.Listener) error

Start starts serving on the transport in a separate go-routine

type InmemTuples

type InmemTuples struct {
	// contains filtered or unexported fields
}

InmemTuples implements an inmemory TupleStorage interface

func NewInmemTuples

func NewInmemTuples() *InmemTuples

NewInmemTuples returns a new instance of InmemTuples

func (*InmemTuples) Delete

func (tuples *InmemTuples) Delete(keys ...[]byte) int

Delete satisfies the TupleStorage interface

func (*InmemTuples) Expire

func (tuples *InmemTuples) Expire(d time.Duration) int

Expire satisfies the TupleStorage interface

func (*InmemTuples) ExpireHost

func (tuples *InmemTuples) ExpireHost(host string) int

ExpireHost satisfies the TupleStorage interface

func (*InmemTuples) Insert

func (tuples *InmemTuples) Insert(tpls ...*Tuple) int

Insert satisfies the TupleStorage interface

func (*InmemTuples) List

func (tuples *InmemTuples) List() []*Tuple

List satisfies the TupleStorage interface

func (*InmemTuples) Lookup

func (tuples *InmemTuples) Lookup(key []byte) *Tuple

Lookup satisfies the TupleStorage interface

func (*InmemTuples) Ping

func (tuples *InmemTuples) Ping(keys ...[]byte) int

Ping satisfies the TupleStorage interface

type Kelips

type Kelips struct {
	// contains filtered or unexported fields
}

Kelips is the user interface to interact with the kelips DHT

func New

func New(host string, conf *Config) *Kelips

New returns a new Kelips instance based on the advertisable address and config. advAddr is the address others will use to connect to this node

func (*Kelips) AddPeer

func (klp *Kelips) AddPeer(host PeerContact) (int64, error)

AddPeer adds the peer as a contact to the affinity group it belongs to

func (*Kelips) Insert

func (klp *Kelips) Insert(key []byte) (string, error)

Insert inserts the key into the DHT

func (*Kelips) Lookup

func (klp *Kelips) Lookup(req *Request) (string, error)

Lookup returns known peers for the given key

func (*Kelips) RemovePeer

func (klp *Kelips) RemovePeer(host PeerContact) (int64, error)

RemovePeer removes a peer from a group

func (*Kelips) Shutdown

func (klp *Kelips) Shutdown(ctx context.Context) error

Shutdown shuts down the kelips node

func (*Kelips) Start

func (klp *Kelips) Start(ln net.Listener) error

Start starts listening for connections on the given listener and starts all groups. This is non-blocking

type Peer

type Peer struct {
	Host string
	// contains filtered or unexported fields
}

Peer node

func (*Peer) Address

func (p *Peer) Address() string

Address satisfies the PeerContact interface

type PeerContact

type PeerContact interface {
	// Returns the ip:port of the peer
	Address() string
}

PeerContact implements a kelips peer

type Request

type Request struct {
	Key        []byte       // Key to lookup or insert
	TTL        int          // number of hops
	Originator GroupContact // Group originating the request
}

Request is a lookup or insert request

type Transport

type Transport interface {
	Insert(contact GroupContact, key []byte) (string, error)
	// Lookup should return the home node of the key
	Lookup(contact GroupContact, req *Request) (string, error)
	// Add a peer to the group
	AddPeer(contact GroupContact, host PeerContact) error
	// Registers the affinity group with the transport
	Register(contact GroupContact, group AffinityGroup)
	// Start the transport.  This should be non-blocking
	Start(net.Listener) error
	// Shutdown the transport
	Shutdown(ctx context.Context) error
}

Transport implements a kelips transport interface

type Tuple

type Tuple struct {
	// Tuple key
	Key []byte
	// Host on which the data associated to the key lives
	Host string
	// contains filtered or unexported fields
}

Tuple holds a key to host mapping along with the heartbeat count

func (*Tuple) Clone

func (t *Tuple) Clone() *Tuple

Clone returns a clone of the tuple

type TupleStorage

type TupleStorage interface {
	// Ping should increment the tuple counter and the last seen time
	Ping(key ...[]byte) int
	// Remove all tuples not seen in the last d time.Duration
	Expire(d time.Duration) int
	// Remove all tuples with the given host
	ExpireHost(host string) int
	// Insert the tuple
	Insert(...*Tuple) int
	// Delete all given keys returning the number of keys deleted
	Delete(keys ...[]byte) int
	// Returns nil if a tuple for the key is not found
	Lookup(key []byte) *Tuple
	// List all tuples in the store
	List() []*Tuple
}

TupleStorage implements a tuple storage interface

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL