README
¶
racket
import "github.com/cognusion/go-racket"
Overview
Package racket is a manager for Jobs, Work, and Progress.
A Job is a repetitive task that uses a common Supervisor to ensure Work is properly distributed, that the correct number of workers are available to do the Work, and that those workers can send Progress along as-needed.
Work is a map of parameters that a worker can take to do its Job. It is important that Work contains at least all of the parameters a worker expects.
Progress is a typed communication construct that allows workers to send a predictable and actionable set of information. Some Progress may be ignored sometimes (e.g. if one is not tracking units of work (ala a progress bar) then the ProgressUpdate and ProgressEstimate types are simply discarded).
The interfaces are written such that one may use the existing Job system via NewJob() or implement their own. Likewise one might use Progress and Work and ignore the Job altogether.
var (
wCount atomic.Int64 // atomic counter
stdOut = log.New(os.Stdout, "", 0) // a logger to output to the terminal
wchan = make(chan Work) // a channel to put Work into
)
// a WorkerFunc to do work. In this case just push a ProgressMessage onto the
// progress channel, and atomically increment a number.
workerFunc := func(id any, work Work, pchan chan<- Progress) {
pchan <- PMessagef("I am %v! The work number is %d!\n", id, work.GetInt("the number"))
wCount.Add(1)
}
// Create a new Job, that will use our workerFunc
j := NewJob(workerFunc)
// Hire a Supervisor to oversee 2 workers, who will get work via our Work channel.
// The Supervisor will return a channel to receive Progress on, and a func to call
// when we are done putting Work onto the Work channel.
pchan, done := j.Supervisor(2, wchan)
defer close(pchan) // if we don't close pchan, the ProcessLogger (below) never exits cleanly.
// Spin up a ProgressLogger using our stdOut logger, logging messages,
// not especially handling errors, reading from pchan, not using a progress bar
go ProgressLogger(stdOut, true, nil, pchan, nil)
// Put 100 items of Work into the Work channel.
for i := range 100 {
wchan <- NewWork(map[string]any{
"the number": i,
})
}
done() // signal the Supervisor and any idle workers that we're done giving out Work.
// wait until all outstanding Work is accomplished. i.e. the Job is done.
<-j.IsDone()
Index
- func ProgressLogger(outLog *log.Logger, logMessages bool, errf ProgressErrorFunc, progressChan <-chan Progress, barChan chan Progress)
- type Job
- type Progress
- type ProgressErrorFunc
- type ProgressType
- type Work
- type WorkerFunc
Examples
Package files
func ProgressLogger
func ProgressLogger(outLog *log.Logger, logMessages bool, errf ProgressErrorFunc, progressChan <-chan Progress, barChan chan Progress)
ProgressLogger is a helper that can loop over a Progress channel and triage the items generically. If non-nil, the supplied ProgressErrorFunc will be called with the error after it is logged or printed: Panic'ing or Exit'ing is allowed. ProgressBar-related Progress will be sent to the barChan as-is.
type Job
type Job interface {
// Supervisor will ensure there are workers to do the Work, and a channel to receive that Work on,
// while also supplying a means to receive progress reports and how to report back when there is no
// more work to do.
Supervisor(maxWorkers int, workChan chan Work) (progressChan chan Progress, doneFunc func())
// NewWorker will ready a worker to do some Work, giving it an ID to reference it by. Calling this directly
// is generally unnecessary as Supervisor will handle it.
NewWorker(id any)
// IsDone will wait until all of the doled-out Work had been completed, and all of the workers have left.
// It's flexible enough to be used as a blocking inline "wait" or in a select{} so other things can occur whilst
// waiting.
IsDone() <-chan bool
}
Job is a repetitive task that uses a common Supervisor to ensure Work is properly distributed, that the correct number of workers are available to do the Work, and that those workers can send Progress along as-needed.
func NewJob
func NewJob(workerFunc WorkerFunc) Job
NewJob consumes a WorkerFunc to accomplish Work, and returns a Job.
type Progress
type Progress struct {
Type ProgressType
Data any
}
Progress is a tuple of a ProgressType and Data. It is also an error and a string.
func PErrorf
func PErrorf(format string, a ...any) Progress
PErrorf returns a ProgressError with a formatted error.
func PEstimate
func PEstimate(estimate int64) Progress
PEstimate returns a ProgressEstimate with the specified estimate.
func PMessagef
func PMessagef(format string, a ...any) Progress
PMessagef returns a ProgressMessage with a formatted string.
func PUpdate
func PUpdate(count int64) Progress
PUpdate returns a ProgressUpdate with the specified count.
func (*Progress) Error
func (p *Progress) Error() error
Error returns the Progress Data as an error if Progress is a ProgressError, or nil.
func (*Progress) String
func (p *Progress) String() string
String returns a formatted string representation of the ProgressType and the Data.
type ProgressErrorFunc
type ProgressErrorFunc func(error)
ProgressErrorFunc is a function that consumes an error.
type ProgressType
type ProgressType int
ProgressType is one of the constant types of Progress.
const (
ProgressError ProgressType = iota
ProgressUpdate
ProgressEstimate
ProgressMessage
ProgressOther
)
ProgressError is a ProgressType when the Data is an error. ProgressUpdate is a ProgressType when the Data is a numeric update (ala progress bar +/- math). ProgressEsimate is a ProgressType when the Data is a numeric [re]evaluation of how much work is to be performed. ProgressMessage is a ProgressType when the Data is a string message. ProgressOther is a ProgressType when Data is to be consumed elsewhere, and should not be interpretted outside of that elsewhere.
func (ProgressType) String
func (p ProgressType) String() string
String returns the stringified version of the type name
type Work
type Work struct {
// contains filtered or unexported fields
}
Work is a representation of specification to pass to a Worker doing a Job.
func NewWork
func NewWork(config map[string]any) Work
NewWork takes a map and returns a specified unit of Work.
func (*Work) Get
func (w *Work) Get(key string) any
Get returns the value associated with the key, or nil.
func (*Work) GetBool
func (w *Work) GetBool(key string) bool
GetBool returns the bool-ified value associated with the key.
func (*Work) GetInt
func (w *Work) GetInt(key string) int
GetInt returns the int-ifiied value associated with the key.
func (*Work) GetString
func (w *Work) GetString(key string) string
GetString returns the string-ified value associated with the key.
type WorkerFunc
type WorkerFunc func(id any, work Work, progressChan chan<- Progress)
WorkerFunc is a definition for how to accomplish Work! Each invocation can assume it has been giving a unique ID, has it's own unique Work, and it can send various Progress updates over the supplied channel.
Generated by godoc2md
Documentation
¶
Overview ¶
Package racket is a manager for Jobs, Work, and Progress.
A Job is a repetitive task that uses a common Supervisor to ensure Work is properly distributed, that the correct number of workers are available to do the Work, and that those workers can send Progress along as-needed.
Work is a map of parameters that a worker can take to do its Job. It is important that Work contains at least all of the parameters a worker expects.
Progress is a typed communication construct that allows workers to send a predictable and actionable set of information. Some Progress may be ignored sometimes (e.g. if one is not tracking units of work (ala a progress bar) then the ProgressUpdate and ProgressEstimate types are simply discarded).
The interfaces are written such that one may use the existing Job system via NewJob() or implement their own. Likewise one might use Progress and Work and ignore the Job altogether.
Example ¶
var (
wCount atomic.Int64 // atomic counter
stdOut = log.New(os.Stdout, "", 0) // a logger to output to the terminal
wchan = make(chan Work) // a channel to put Work into
)
// a WorkerFunc to do work. In this case just push a ProgressMessage onto the
// progress channel, and atomically increment a number.
workerFunc := func(id any, work Work, pchan chan<- Progress) {
pchan <- PMessagef("I am %v! The work number is %d!\n", id, work.GetInt("the number"))
wCount.Add(1)
}
// Create a new Job, that will use our workerFunc
j := NewJob(workerFunc)
// Hire a Supervisor to oversee 2 workers, who will get work via our Work channel.
// The Supervisor will return a channel to receive Progress on, and a func to call
// when we are done putting Work onto the Work channel.
pchan, done := j.Supervisor(2, wchan)
defer close(pchan) // if we don't close pchan, the ProcessLogger (below) never exits cleanly.
// Spin up a ProgressLogger using our stdOut logger, logging messages,
// not especially handling errors, reading from pchan, not using a progress bar
go ProgressLogger(stdOut, true, nil, pchan, nil)
// Put 100 items of Work into the Work channel.
for i := range 100 {
wchan <- NewWork(map[string]any{
"the number": i,
})
}
done() // signal the Supervisor and any idle workers that we're done giving out Work.
// wait until all outstanding Work is accomplished. i.e. the Job is done.
<-j.IsDone()
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ProgressLogger ¶
func ProgressLogger(outLog *log.Logger, logMessages bool, errf ProgressErrorFunc, progressChan <-chan Progress, barChan chan Progress)
ProgressLogger is a helper that can loop over a Progress channel and triage the items generically. If non-nil, the supplied ProgressErrorFunc will be called with the error after it is logged or printed: Panic'ing or Exit'ing is allowed. ProgressBar-related Progress will be sent to the barChan as-is.
Types ¶
type Job ¶
type Job interface {
// Supervisor will ensure there are workers to do the Work, and a channel to receive that Work on,
// while also supplying a means to receive progress reports and how to report back when there is no
// more work to do.
Supervisor(maxWorkers int, workChan chan Work) (progressChan chan Progress, doneFunc func())
// NewWorker will ready a worker to do some Work, giving it an ID to reference it by. Calling this directly
// is generally unnecessary as Supervisor will handle it.
NewWorker(id any)
// IsDone will wait until all of the doled-out Work had been completed, and all of the workers have left.
// It's flexible enough to be used as a blocking inline "wait" or in a select{} so other things can occur whilst
// waiting.
IsDone() <-chan bool
}
Job is a repetitive task that uses a common Supervisor to ensure Work is properly distributed, that the correct number of workers are available to do the Work, and that those workers can send Progress along as-needed.
func NewJob ¶
func NewJob(workerFunc WorkerFunc) Job
NewJob consumes a WorkerFunc to accomplish Work, and returns a Job.
type Progress ¶
type Progress struct {
Type ProgressType
Data any
}
Progress is a tuple of a ProgressType and Data. It is also an error and a string.
type ProgressErrorFunc ¶
type ProgressErrorFunc func(error)
ProgressErrorFunc is a function that consumes an error.
type ProgressType ¶
type ProgressType int
ProgressType is one of the constant types of Progress.
const ( ProgressError ProgressType = iota ProgressUpdate ProgressEstimate ProgressMessage ProgressOther )
ProgressError is a ProgressType when the Data is an error. ProgressUpdate is a ProgressType when the Data is a numeric update (ala progress bar +/- math). ProgressEsimate is a ProgressType when the Data is a numeric [re]evaluation of how much work is to be performed. ProgressMessage is a ProgressType when the Data is a string message. ProgressOther is a ProgressType when Data is to be consumed elsewhere, and should not be interpretted outside of that elsewhere.
func (ProgressType) String ¶
func (p ProgressType) String() string
String returns the stringified version of the type name
type Work ¶
type Work struct {
// contains filtered or unexported fields
}
Work is a representation of specification to pass to a Worker doing a Job.
type WorkerFunc ¶
WorkerFunc is a definition for how to accomplish Work! Each invocation can assume it has been giving a unique ID, has it's own unique Work, and it can send various Progress updates over the supplied channel.
Source Files