rangetripper

rangetripper

import "github.com/cognusion/go-rangetripper"

• Overview
• Index

Overview

Package rangetripper provides a performant http.RoundTripper that handles byte-range downloads if the resulting HTTP server claims to support them in a HEAD request for the file. RangeTripper will download 1/Nth of the file asynchronously with each of the “fileChunks“ specified in a New. N+1 actual downloaders are most likely as the +1 covers any gap from non-even division of content-length.

Index

• Constants
• type Client
• type RangeTripper
  • func New(fileChunks int, outputFilePath string) (*RangeTripper, error)
  • func NewWithLoggers(fileChunks int, outputFilePath string, timingLogger, debugLogger *log.Logger) (*RangeTripper, error)
  • func (rt *RangeTripper) Do(r *http.Request) (*http.Response, error)
  • func (rt *RangeTripper) RoundTrip(r *http.Request) (*http.Response, error)
  • func (rt *RangeTripper) SetChunkSize(chunkBytes int64)
  • func (rt *RangeTripper) SetClient(client Client)
  • func (rt *RangeTripper) SetMax(max int)
  • func (rt *RangeTripper) WithProgress() <-chan int64
• type RetryClient
  • func NewRetryClient(retries int, every, timeout time.Duration) *RetryClient
  • func NewRetryClientWithExponentialBackoff(retries int, initially, timeout time.Duration) *RetryClient
  • func (w *RetryClient) Do(req *http.Request) (*http.Response, error)

Package files

client.go retryclient.go rt.go

Constants

const (
    ContentLengthNumericError   = rtError("Content-Length value cannot be converted to a number")
    ContentLengthMismatchError  = rtError("downloaded file size does not match content-length")
    SingleRequestExhaustedError = rtError("one request has already been made with this RangeTripper")
)

Static errors to return

type Client

type Client interface {
    Do(*http.Request) (*http.Response, error)
}

Client is an interface that could refer to an http.Client or a rangetripper.RetryClient

var DefaultClient Client = NewRetryClient(10, 2*time.Second, 60*time.Second)

DefaultClient is what RangeTripper will use to actually make the individual GET requests. Change the values to change the outcome. Don't set the DefaultClient's Client.Transport to a RangeTripper, or :mindblown:. DefaultClient can be a lowly http.Client if you prefer

type RangeTripper

type RangeTripper struct {
    TimingsOut *log.Logger
    DebugOut   *log.Logger
    // contains filtered or unexported fields
}

RangeTripper is an http.RoundTripper to be used in an http.Client. This should not be used in its default state, instead by its New functions. A single RangeTripper must only be used for one request.

func New

func New(fileChunks int, outputFilePath string) (*RangeTripper, error)

New returns a RangeTripper or an error. Logged messages are discarded.

fileChunks is the number of pieces to divide the dowloaded file into (+/- 1). Overridden by SetMax.

outFilePath is the path to a filesystem location to save the file to, or "BUFFER". In the latter case, the contents will be returned in the final http.Response returned by Roundtrip, as the Response.Body.

func NewWithLoggers

func NewWithLoggers(fileChunks int, outputFilePath string, timingLogger, debugLogger *log.Logger) (*RangeTripper, error)

NewWithLoggers returns a RangeTripper or an error. Logged messages are sent to the specified Logger, or discarded if nil.

fileChunks is the number of pieces to divide the dowloaded file into (+/- 1). Overridden by SetMax.

outFilePath is the path to a filesystem location to save the file to, or "BUFFER". In the latter case, the contents will be returned in the final http.Response returned by Roundtrip, as the Response.Body.

timingLogger is a logger to send timing-related messages to.

debugLogger is a logger to send debug messages to.

func (*RangeTripper) Do

func (rt *RangeTripper) Do(r *http.Request) (*http.Response, error)

Do is a satisfier of the rangetripper.Client interface, and is identical to RoundTrip

func (*RangeTripper) RoundTrip

func (rt *RangeTripper) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip is called with a formed Request, writing the Body of the Response to to the specified output file. The Response should be ignored, but errors are important. Both the Request.Body and the RangeTripper.outFile will be closed when this function returns.

func (*RangeTripper) SetChunkSize

func (rt *RangeTripper) SetChunkSize(chunkBytes int64)

SetChunkSize overrides the “fileChunks“ and instead will divide the resulting Content-Length by this to determine the appropriate chunk count dynamically. “fileChunks“ will still be used to guide the maximum number of concurrent workers, unless “SetMax()“ is used.

func (*RangeTripper) SetClient

func (rt *RangeTripper) SetClient(client Client)

SetClient allows for overriding the Client used to make the requests.

func (*RangeTripper) SetMax

func (rt *RangeTripper) SetMax(max int)

SetMax allows for setting the maximum number of concurrently-running workers

func (*RangeTripper) WithProgress

func (rt *RangeTripper) WithProgress() <-chan int64

WithProgress returns a read-only chan that will first provide the total length of the content (in bytes), followed by a stream of completed byte-lengths. CAUTION: It is a generally bad idea to call this and then ignore the resulting channel.

type RetryClient

type RetryClient struct {
    // contains filtered or unexported fields
}

RetryClient contains variables and methods to use when making smarter HTTP requests

func NewRetryClient

func NewRetryClient(retries int, every, timeout time.Duration) *RetryClient

NewRetryClient returns a RetryClient that will retry failed requests retries times, every every, and use timeout as a timeout

func NewRetryClientWithExponentialBackoff

func NewRetryClientWithExponentialBackoff(retries int, initially, timeout time.Duration) *RetryClient

NewRetryClientWithExponentialBackoff returns a RetryClient that will retry failed requests retries times, first after initially and exponentially longer each time, and use timeout as a timeout

func (*RetryClient) Do

func (w *RetryClient) Do(req *http.Request) (*http.Response, error)

Do takes a Request, and returns a Response or an error, following the rules of the RetryClient

Generated by godoc2md