intercom

package module
v0.0.0-...-e550ec4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 11, 2018 License: Apache-2.0 Imports: 39 Imported by: 0

README

Build Status

intercom

intercom is a containerized plugin system over gRPC. My goal was to be able to deploy Kubernetes Deployments or StatefulSets along with a Service that would act as a blackbox interface defined by the gRPC protocol.

The go-plugin project provided for a lot of the initial inspiration and direction of this project, however I wanted to make it native to Kubernetes by leveraging Pods and Services.

Future Enhancements

Here's some wishful thinking for this project:

  • the protocol wouldn't even have to be gRPC
  • the protocol would be built at a much lower level such as the container runtime level (e.g. Docker, rocket). These runtimes would define an interface of the container by a similar method as Expose in Docker.

Documentation

Overview

Package intercom is a generated protocol buffer package.

It is generated from these files:

grpc_broker.proto

It has these top-level messages:

ConnInfo

Index

Constants

View Source
const CoreProtocolVersion = 1

CoreProtocolVersion is the ProtocolVersion of the plugin system itself. We will increment this whenever we change any protocol behavior. This will invalidate any prior plugins but will at least allow us to iterate on the core in a safe way. We will do our best to do this very infrequently.

View Source
const DefaultServerPort = 50077

DefaultServerPort is the default server port for net listeners

View Source
const GRPCServiceName = "plugin"

GRPCServiceName is the name of the service that the health check should return as passing.

Variables

View Source
var (
	// ErrMissingDeployment is returned when a deployment is not configured
	ErrMissingDeployment = errors.New("deployment is missing")

	// ErrMissingService is returned when a service is not configured
	// todo: have a default service?
	ErrMissingService = errors.New("service is missing")

	// ErrChecksumsDoNotMatch is returned when binary's checksum doesn't match
	// the one provided in the SecureConfig.
	ErrChecksumsDoNotMatch = errors.New("checksums did not match")

	// ErrSecureNoChecksum is returned when an empty checksum is provided to the
	// SecureConfig.
	ErrSecureConfigNoChecksum = errors.New("no checksum provided")

	// ErrSecureNoHash is returned when a nil Hash object is provided to the
	// SecureConfig.
	ErrSecureConfigNoHash = errors.New("no hash implementation provided")
)

Error types

View Source
var DefaultRetry = wait.Backoff{
	Steps:    5,
	Duration: 10 * time.Millisecond,
	Factor:   1.0,
	Jitter:   0.1,
}

DefaultRetry is a default retry backoff settings when retrying API calls

Functions

func DefaultGRPCServer

func DefaultGRPCServer(opts []grpc.ServerOption) *grpc.Server

DefaultGRPCServer can be used with the "GRPCServer" field for Server as a default factory method to create a gRPC server with no extra options.

func Discover

func Discover(kubeClient kubernetes.Interface, namespace string) (*corev1.ServiceList, error)

Discover discovers plugins that are in a given namespace.

The directory doesn't need to be absolute. For example, "." will work fine.

func ForwardContainerOutput

func ForwardContainerOutput(exec remotecommand.Executor, stdOutWriter io.Writer, stdErrWriter io.Writer) error

ForwardContainerOutput initiates the transfer of standard shell output from the Container to the writers

func GetContainerLogs

func GetContainerLogs(restConfig *rest.Config, req *rest.Request) (remotecommand.Executor, error)

GetContainerLogs returns the remotecommand.Executor for the pod's container

func IsRetryableKubeAPIError

func IsRetryableKubeAPIError(err error) bool

IsRetryableKubeAPIError returns if the error is a retryable kubernetes error

func RegisterGRPCBrokerServer

func RegisterGRPCBrokerServer(s *grpc.Server, srv GRPCBrokerServer)

func Serve

func Serve(opts *ServeConfig)

Serve serves the plugins given by ServeConfig.

Serve doesn't return until the plugin is done being executed. Any errors will be outputted to os.Stderr.

This is the method that plugins should call in their main() functions.

func ValidateDeployment

func ValidateDeployment(d *appsv1.Deployment) error

ValidateDeployment for the plugin

func ValidateService

func ValidateService(s *corev1.Service) error

ValidateService for the plugin

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client handles the lifecycle of a plugin application. It deploys plugins, connects to them, dispenses interface implementations, and handles killing the containers.

Plugin hosts should use one Client for each plugin executable. To dispense a plugin type, use the `Client.Client` function, and then cal `Dispense`. This awkward API is mostly historical but is used to split the client that deals with subprocess management and the client that does RPC management.

See NewClient and ClientConfig for using a Client.

func NewClient

func NewClient(config *ClientConfig) (c *Client)

NewClient creates a new plugin client which manages the lifecycle of an external plugin and gets the address for the RPC connection.

The client must be cleaned up at some point by calling Kill(). If the client is a managed client (created with NewManagedClient) you can just call CleanupClients at the end of your program and they will be properly cleaned.

func (*Client) Client

func (c *Client) Client() (ClientProtocol, error)

Client returns the protocol client for this connection.

Subsequent calls to this will return the same client.

func (*Client) Exited

func (c *Client) Exited() bool

Exited tells whether or not the deployment has been destroyed.

func (*Client) Kill

func (c *Client) Kill()

Kill the executing deployment and associated pods and perform any cleanup tasks necessary such as capturing any remaining logs and so on.

This method blocks until the deployment+service is successfully destroyed.

This method can safely be called multiple times.

func (*Client) Start

func (c *Client) Start() (addr net.Addr, err error)

Start the deployment and the service, communicating with it to negotiate a port for RPC connections, and returning the address to connect via RPC.

This method is safe to call multiple times. Subsequent calls have no effect. Once a client has been started once, it cannot be started again, even if it was killed.

type ClientConfig

type ClientConfig struct {
	// HandshakeConfig is the configuration that must match servers.
	HandshakeConfig

	// Plugins are the plugins that can be consumed.
	Plugins map[string]GRPCPlugin

	// KubeConfig contains the kubernetes rest config
	KubeConfig *rest.Config

	// Namespace contains the namespace for this deployment
	Namespace string

	// Deployment contains the kubernetes deployment for this plugin
	Deployment *appsv1.Deployment

	// Service contains the kubernetes service layer for this plugin
	Service *corev1.Service

	// TLSConfig is used to enable TLS on the RPC client.
	TLSConfig *tls.Config

	// Managed represents if the client should be managed by the
	// plugin package or not. If true, then by calling CleanupClients,
	// it will automatically be cleaned up. Otherwise, the client
	// user is fully responsible for making sure to Kill all plugin
	// clients. By default the client is _not_ managed.
	Managed bool

	// StartTimeout is the timeout to wait for the plugin to say it
	// has started successfully.
	StartTimeout time.Duration

	// If non-nil, then the stderr of the client will be written to here
	// (as well as the log). This is the original os.Stderr of the subprocess.
	// This isn't the output of synced stderr.
	Stderr io.Writer

	// SyncStdout, SyncStderr can be set to override the
	// respective os.Std* values in the plugin. Care should be taken to
	// avoid races here. If these are nil, then this will automatically be
	// hooked up to os.Stdin, Stdout, and Stderr, respectively.
	//
	// If the default values (nil) are used, then this package will not
	// sync any of these streams.
	SyncStdout io.Writer
	SyncStderr io.Writer

	// Logger is the logger that the client will used. If none is provided,
	// it will default to hclog's default logger.
	Logger hclog.Logger
}

ClientConfig is the configuration used to initialize a new plugin client. After being used to initialize a plugin client, that configuration must not be modified again.

type ClientProtocol

type ClientProtocol interface {
	io.Closer

	// Dispense dispenses a new instance of the plugin with the given name.
	Dispense(string) (interface{}, error)

	// Ping checks that the client connection is still healthy.
	Ping() error
}

ClientProtocol is an interface that must be implemented for new plugin protocols to be clients.

type ConnInfo

type ConnInfo struct {
	ServiceID uint32 `protobuf:"varint,1,opt,name=serviceID" json:"serviceID,omitempty"`
	Network   string `protobuf:"bytes,2,opt,name=network" json:"network,omitempty"`
	Address   string `protobuf:"bytes,3,opt,name=address" json:"address,omitempty"`
}

func (*ConnInfo) Descriptor

func (*ConnInfo) Descriptor() ([]byte, []int)

func (*ConnInfo) GetAddress

func (m *ConnInfo) GetAddress() string

func (*ConnInfo) GetNetwork

func (m *ConnInfo) GetNetwork() string

func (*ConnInfo) GetServiceID

func (m *ConnInfo) GetServiceID() uint32

func (*ConnInfo) ProtoMessage

func (*ConnInfo) ProtoMessage()

func (*ConnInfo) Reset

func (m *ConnInfo) Reset()

func (*ConnInfo) String

func (m *ConnInfo) String() string

type GRPCBroker

type GRPCBroker struct {
	sync.Mutex
	// contains filtered or unexported fields
}

GRPCBroker is responsible for brokering connections by unique ID.

It is used by plugins to create multiple gRPC connections and data streams between the plugin process and the host process.

This allows a plugin to request a channel with a specific ID to connect to or accept a connection from, and the broker handles the details of holding these channels open while they're being negotiated.

The Plugin interface has access to these for both Server and Client. The broker can be used by either (optionally) to reserve and connect to new streams. This is useful for complex args and return values, or anything else you might need a data stream for.

func (*GRPCBroker) Accept

func (b *GRPCBroker) Accept(id uint32) (net.Listener, error)

Accept accepts a connection by ID.

This should not be called multiple times with the same ID at one time.

func (*GRPCBroker) AcceptAndServe

func (b *GRPCBroker) AcceptAndServe(id uint32, s func([]grpc.ServerOption) *grpc.Server)

AcceptAndServe is used to accept a specific stream ID and immediately serve a gRPC server on that stream ID. This is used to easily serve complex arguments. Each AcceptAndServe call opens a new listener socket and sends the connection info down the stream to the dialer. Since a new connection is opened every call, these calls should be used sparingly. Multiple gRPC server implementations can be registered to a single AcceptAndServe call.

func (*GRPCBroker) Close

func (b *GRPCBroker) Close() error

Close closes the stream and all servers.

func (*GRPCBroker) Dial

func (b *GRPCBroker) Dial(id uint32) (conn *grpc.ClientConn, err error)

Dial opens a connection by ID.

func (*GRPCBroker) NextID

func (b *GRPCBroker) NextID() uint64

NextID returns a unique ID to use next.

It is possible for very long-running plugin hosts to wrap this value, though it would require a very large amount of calls. In practice we've never seen it happen.

func (*GRPCBroker) Run

func (b *GRPCBroker) Run()

Run starts the brokering and should be executed in a goroutine, since it blocks forever, or until the session closes.

Uses of GRPCBroker never need to call this. It is called internally by the plugin host/client.

type GRPCBrokerClient

type GRPCBrokerClient interface {
	StartStream(ctx context.Context, opts ...grpc.CallOption) (GRPCBroker_StartStreamClient, error)
}

func NewGRPCBrokerClient

func NewGRPCBrokerClient(cc *grpc.ClientConn) GRPCBrokerClient

type GRPCBrokerServer

type GRPCBrokerServer interface {
	StartStream(GRPCBroker_StartStreamServer) error
}

type GRPCBroker_StartStreamClient

type GRPCBroker_StartStreamClient interface {
	Send(*ConnInfo) error
	Recv() (*ConnInfo, error)
	grpc.ClientStream
}

type GRPCBroker_StartStreamServer

type GRPCBroker_StartStreamServer interface {
	Send(*ConnInfo) error
	Recv() (*ConnInfo, error)
	grpc.ServerStream
}

type GRPCClient

type GRPCClient struct {
	Conn    *grpc.ClientConn
	Plugins map[string]GRPCPlugin
	// contains filtered or unexported fields
}

GRPCClient connects to a GRPCServer over gRPC to dispense plugin types.

func (*GRPCClient) Close

func (c *GRPCClient) Close() error

Close the ClientProtocol impl.

func (*GRPCClient) Dispense

func (c *GRPCClient) Dispense(name string) (interface{}, error)

Dispense from ClientProtocol impl.

func (*GRPCClient) Ping

func (c *GRPCClient) Ping() error

Ping the ClientProtocol impl.

type GRPCPlugin

type GRPCPlugin interface {
	// GRPCServer should register this plugin for serving with the
	// given GRPCServer. Unlike Plugin.Server, this is only called once
	// since gRPC plugins serve singletons.
	GRPCServer(*GRPCBroker, *grpc.Server) error

	// GRPCClient should return the interface implementation for the plugin
	// you're serving via gRPC. The provided context will be canceled by
	// go-plugin in the event of the plugin process exiting.
	GRPCClient(context.Context, *GRPCBroker, *grpc.ClientConn) (interface{}, error)
}

GRPCPlugin is the interface that is implemented to serve/connect to a plugin over gRPC.

type GRPCServer

type GRPCServer struct {
	// Plugins are the list of plugins to serve.
	Plugins map[string]GRPCPlugin

	// Server is the actual server that will accept connections. This
	// will be used for plugin registration as well.
	Server func([]grpc.ServerOption) *grpc.Server

	// TLS should be the TLS configuration if available. If this is nil,
	// the connection will not have transport security.
	TLS *tls.Config

	// DoneCh is the channel that is closed when this server has exited.
	DoneCh chan struct{}

	// Stdout/StderrLis are the readers for stdout/stderr that will be copied
	// to the stdout/stderr connection that is output.
	Stdout io.Reader
	Stderr io.Reader
	// contains filtered or unexported fields
}

GRPCServer is a ServerType implementation that serves plugins over gRPC. This allows plugins to easily be written for other languages.

The GRPCServer outputs a custom configuration as a base64-encoded JSON structure represented by the GRPCServerConfig config structure.

func (*GRPCServer) Config

func (s *GRPCServer) Config() string

Config is the GRPCServerConfig encoded as JSON then base64.

func (*GRPCServer) GracefulStop

func (s *GRPCServer) GracefulStop()

GracefulStop calls GracefulStop on the underlying grpc.Server

func (*GRPCServer) Init

func (s *GRPCServer) Init() error

Init the ServerProtocol impl.

func (*GRPCServer) Serve

func (s *GRPCServer) Serve(lis net.Listener)

Serve the ServerProtocol impl.

func (*GRPCServer) Stop

func (s *GRPCServer) Stop()

Stop calls Stop on the underlying grpc.Server

type GRPCServerConfig

type GRPCServerConfig struct {
	StdoutAddr string `json:"stdout_addr"`
	StderrAddr string `json:"stderr_addr"`
}

GRPCServerConfig is the extra configuration passed along for consumers to facilitate using GRPC plugins.

type HandshakeConfig

type HandshakeConfig struct {
	// ProtocolVersion is the version that clients must match on to
	// agree they can communicate. This should match the ProtocolVersion
	// set on ClientConfig when using a plugin.
	ProtocolVersion uint

	// MagicCookieKey and value are used as a very basic verification
	// that a plugin is intended to be launched. This is not a security
	// measure, just a UX feature. If the magic cookie doesn't match,
	// we show human-friendly output.
	MagicCookieKey   string
	MagicCookieValue string
}

HandshakeConfig is the configuration used by client and servers to handshake before starting a plugin connection. This is embedded by both ServeConfig and ClientConfig.

In practice, the plugin host creates a HandshakeConfig that is exported and plugins then can easily consume it.

type ServeConfig

type ServeConfig struct {
	// HandshakeConfig is the configuration that must match clients.
	HandshakeConfig

	// TLSProvider is a function that returns a configured tls.Config.
	TLSProvider func() (*tls.Config, error)

	// Plugins are the plugins that are served.
	Plugins map[string]GRPCPlugin

	// GRPCServer should be non-nil to enable serving the plugins over
	// gRPC. This is a function to create the server when needed with the
	// given server options. The server options populated by go-plugin will
	// be for TLS if set. You may modify the input slice.
	//
	// Note that the grpc.Server will automatically be registered with
	// the gRPC health checking service. This is not optional since go-plugin
	// relies on this to implement Ping().
	GRPCServer func([]grpc.ServerOption) *grpc.Server

	// Logger is used to pass a logger into the server. If none is provided the
	// server will create a default logger.
	Logger hclog.Logger
}

ServeConfig configures what sorts of plugins are served.

type ServerProtocol

type ServerProtocol interface {
	// Init is called once to configure and initialize the protocol, but
	// not start listening. This is the point at which all validation should
	// be done and errors returned.
	Init() error

	// Config is extra configuration to be outputted to stdout. This will
	// be automatically base64 encoded to ensure it can be parsed properly.
	// This can be an empty string if additional configuration is not needed.
	Config() string

	// Serve is called to serve connections on the given listener. This should
	// continue until the listener is closed.
	Serve(net.Listener)
}

ServerProtocol is an interface that must be implemented for new plugin protocols to be servers.

type ServiceAddr

type ServiceAddr struct {
	Name      string
	Namespace string
	Port      int32
}

ServiceAddr implements the net.Addr interface

func (*ServiceAddr) Network

func (addr *ServiceAddr) Network() string

Network for the ServiceAddr - we only operate over tcp

func (*ServiceAddr) String

func (addr *ServiceAddr) String() string

String format of the ServiceAddr

Directories

Path Synopsis
examples
basic command
basic/plugin command
basic/proto
Package proto is a generated protocol buffer package.
Package proto is a generated protocol buffer package.
basic/shared
Package shared contains shared data between the host and plugins.
Package shared contains shared data between the host and plugins.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL