gosftp

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 18, 2026 License: MIT Imports: 22 Imported by: 0

README

gosftp

A Go library for synchronizing files over SSH/SFTP with connection pooling, retry logic, and bastion host support.

CI Go Version codecov License

Features

  • SSH client with SFTP support for file operations
  • Connection pooling for efficient connection reuse
  • Retry logic with exponential backoff for transient failures
  • Support for various authentication methods (private key, password, certificate)
  • Bastion/jump host support for multi-hop SSH connections
  • High-level sync API for file and directory synchronization

Installation

go get github.com/darshan-rambhia/gosftp

Usage

Basic Usage
package main

import (
    "context"
    "log"

    "github.com/darshan-rambhia/gosftp"
)

func main() {
    config := gosftp.Config{
        Host:    "example.com",
        Port:    22,
        User:    "deploy",
        KeyPath: "~/.ssh/id_ed25519",
    }

    client, err := gosftp.NewClient(config)
    if err != nil {
        log.Fatal(err)
    }
    defer client.Close()

    ctx := context.Background()
    err = client.UploadFile(ctx, "/local/path/file.txt", "/remote/path/file.txt")
    if err != nil {
        log.Fatal(err)
    }
}
Connection Pooling

For multiple operations to the same host, use connection pooling:

pool := gosftp.NewConnectionPool(5 * time.Minute)
defer pool.Close()

client, err := pool.GetOrCreate(config)
if err != nil {
    log.Fatal(err)
}
defer pool.Release(config)

// Use client...
High-Level Sync API

For common sync operations, use the Syncer API:

syncer, err := gosftp.NewSyncer(config)
if err != nil {
    log.Fatal(err)
}
defer syncer.Close()

// Sync a single file
result, err := syncer.SyncFile(ctx, "/local/file.txt", "/remote/file.txt", nil)

// Sync a directory
results, err := syncer.SyncDirectory(ctx, "/local/dir", "/remote/dir", nil)
Authentication Methods
Private Key (from file)
config := gosftp.Config{
    Host:    "example.com",
    Port:    22,
    User:    "deploy",
    KeyPath: "~/.ssh/id_ed25519",
}
Private Key (inline)
config := gosftp.Config{
    Host:       "example.com",
    Port:       22,
    User:       "deploy",
    PrivateKey: "-----BEGIN OPENSSH PRIVATE KEY-----\n...",
}
Password
config := gosftp.Config{
    Host:       "example.com",
    Port:       22,
    User:       "deploy",
    Password:   "secret",
    AuthMethod: gosftp.AuthMethodPassword,
}
Bastion/Jump Host
config := gosftp.Config{
    Host:           "internal-server",
    Port:           22,
    User:           "deploy",
    KeyPath:        "~/.ssh/id_ed25519",
    BastionHost:    "bastion.example.com",
    BastionPort:    22,
    BastionUser:    "jump",
    BastionKeyPath: "~/.ssh/bastion_key",
}

Testing

# Run unit tests
task test

# Run benchmarks (requires Docker)
task test:bench

# Run fuzz tests
task test:fuzz 

# Generate coverage report (requires Docker)
task test:coverage

License

MIT License - see LICENSE for details.

Documentation

Overview

Package gosftp provides a library for synchronizing files over SSH/SFTP.

This package provides:

  • SSH client with SFTP support for file operations
  • Connection pooling for efficient connection reuse
  • Retry logic with exponential backoff for transient failures
  • Support for various authentication methods (private key, password, certificate)
  • Bastion/jump host support for multi-hop SSH connections

Basic Usage

Create a client and upload a file:

config := gosftp.Config{
	Host:    "example.com",
	Port:    22,
	User:    "deploy",
	KeyPath: "~/.ssh/id_ed25519",
}

client, err := gosftp.NewClient(config)
if err != nil {
	log.Fatal(err)
}
defer client.Close()

err = client.UploadFile(ctx, "/local/path/file.txt", "/remote/path/file.txt")

Connection Pooling

For multiple operations to the same host, use connection pooling:

pool := gosftp.NewConnectionPool(5 * time.Minute)
defer pool.Close()

client, err := pool.GetOrCreate(config)
if err != nil {
	log.Fatal(err)
}
defer pool.Release(config)

// Use client...

High-Level API

For common sync operations, use the Syncer API:

syncer, err := gosftp.NewSyncer(config)
if err != nil {
	log.Fatal(err)
}
defer syncer.Close()

// Sync a single file
result, err := syncer.SyncFile(ctx, "/local/file.txt", "/remote/file.txt", nil)

// Sync a directory
results, err := syncer.SyncDirectory(ctx, "/local/dir", "/remote/dir", nil)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ComputeCombinedHash

func ComputeCombinedHash(files []FileInfo) string

ComputeCombinedHash computes a combined hash from multiple file hashes.

func ExpandPath

func ExpandPath(path string) string

ExpandPath expands ~ to home directory.

func HashFile

func HashFile(path string) (string, int64, error)

HashFile computes the SHA256 hash of a file.

func IsBinaryContent

func IsBinaryContent(content []byte) bool

IsBinaryContent checks if content appears to be binary.

func IsRetryableError

func IsRetryableError(err error) bool

IsRetryableError checks if an error is transient and worth retrying. It uses error type assertions for more reliable detection, with string matching as a fallback.

func Retry

func Retry(ctx context.Context, config RetryConfig, operation string, fn RetryableFunc) error

Retry executes the given function with exponential backoff retry logic.

func ValidateMode

func ValidateMode(mode string) error

ValidateMode checks if a file mode string is valid.

func WithRetry

func WithRetry(ctx context.Context, maxRetries int, operation string, fn RetryableFunc) error

WithRetry wraps an operation with retry logic.

Types

type AuthMethod

type AuthMethod string

AuthMethod represents the SSH authentication method to use.

const (
	// AuthMethodPrivateKey uses SSH private key authentication (default).
	AuthMethodPrivateKey AuthMethod = "private_key"
	// AuthMethodPassword uses password authentication.
	AuthMethodPassword AuthMethod = "password"
	// AuthMethodCertificate uses SSH certificate authentication.
	AuthMethodCertificate AuthMethod = "certificate"
)

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client wraps SSH and SFTP connections for file operations.

func NewClient

func NewClient(config Config) (*Client, error)

NewClient creates a new SSH/SFTP client.

func NewClientWithSFTP

func NewClientWithSFTP(sftpClient SFTPClientInterface, sshClient *ssh.Client) *Client

NewClientWithSFTP creates a Client with a custom SFTP client implementation. This is primarily used for testing with mock SFTP clients.

func (*Client) Close

func (c *Client) Close() error

Close closes SFTP, SSH, and bastion connections. Close is idempotent - calling it multiple times is safe. Returns an error if any close operation fails. Multiple errors are aggregated.

func (*Client) DeleteFile

func (c *Client) DeleteFile(ctx context.Context, remotePath string) error

DeleteFile removes a file from the remote host.

func (*Client) FileExists

func (c *Client) FileExists(ctx context.Context, remotePath string) (bool, error)

FileExists checks if a file exists on the remote host.

func (*Client) GetFileHash

func (c *Client) GetFileHash(ctx context.Context, remotePath string) (string, error)

GetFileHash returns the SHA256 hash of a remote file.

func (*Client) GetFileInfo

func (c *Client) GetFileInfo(ctx context.Context, remotePath string) (os.FileInfo, error)

GetFileInfo returns information about a remote file.

func (*Client) IsHealthy

func (c *Client) IsHealthy() bool

IsHealthy checks if the client's SSH connection is still valid.

func (*Client) ReadFileContent

func (c *Client) ReadFileContent(ctx context.Context, remotePath string, maxBytes int64) ([]byte, error)

ReadFileContent reads the content of a remote file.

func (*Client) SetFileAttributes

func (c *Client) SetFileAttributes(ctx context.Context, remotePath, owner, group, mode string) error

SetFileAttributes sets ownership and permissions on a remote file.

func (*Client) UploadFile

func (c *Client) UploadFile(ctx context.Context, localPath, remotePath string) error

UploadFile uploads a local file to the remote host.

type ClientInterface

type ClientInterface interface {
	// Close closes the SSH connection.
	Close() error
	// UploadFile uploads a local file to the remote host.
	UploadFile(ctx context.Context, localPath, remotePath string) error
	// GetFileHash returns the SHA256 hash of a remote file.
	GetFileHash(ctx context.Context, remotePath string) (string, error)
	// SetFileAttributes sets ownership and permissions on a remote file.
	SetFileAttributes(ctx context.Context, remotePath, owner, group, mode string) error
	// DeleteFile removes a file from the remote host.
	DeleteFile(ctx context.Context, remotePath string) error
	// FileExists checks if a file exists on the remote host.
	FileExists(ctx context.Context, remotePath string) (bool, error)
	// GetFileInfo returns information about a remote file.
	GetFileInfo(ctx context.Context, remotePath string) (os.FileInfo, error)
	// ReadFileContent reads the content of a remote file.
	ReadFileContent(ctx context.Context, remotePath string, maxBytes int64) ([]byte, error)
}

ClientInterface defines the interface for SSH/SFTP operations. This allows for mocking in tests.

type Config

type Config struct {
	// Host is the target SSH server hostname or IP address.
	Host string

	// Port is the SSH port (default 22).
	Port int

	// User is the SSH username.
	User string

	// AuthMethod specifies which authentication method to use.
	// If not set, it will be inferred from the provided credentials.
	AuthMethod AuthMethod

	// PrivateKey is the SSH private key content (PEM encoded).
	// Mutually exclusive with KeyPath.
	PrivateKey string

	// KeyPath is the path to the SSH private key file.
	// Mutually exclusive with PrivateKey.
	KeyPath string

	// Password is the SSH password for password authentication.
	Password string

	// Certificate is the SSH certificate content.
	// Used with PrivateKey or KeyPath for certificate authentication.
	Certificate string

	// CertificatePath is the path to the SSH certificate file.
	// Used with PrivateKey or KeyPath for certificate authentication.
	CertificatePath string

	// Timeout is the connection timeout (default 30s).
	Timeout time.Duration

	// KnownHostsFile is the path to a known_hosts file for host key verification.
	// If not set, defaults to ~/.ssh/known_hosts if it exists.
	KnownHostsFile string

	// InsecureIgnoreHostKey skips host key verification.
	// WARNING: This is insecure and should only be used for testing.
	// Deprecated: Use StrictHostKeyChecking = "no" instead.
	InsecureIgnoreHostKey bool

	// StrictHostKeyChecking controls host key verification behavior.
	// Valid values: "yes" (default), "no", "accept-new".
	// Takes precedence over InsecureIgnoreHostKey if set.
	StrictHostKeyChecking StrictHostKeyChecking

	// BastionHost is the hostname or IP of a bastion/jump host.
	BastionHost string

	// BastionPort is the SSH port of the bastion host (default 22).
	BastionPort int

	// BastionUser is the SSH username for the bastion host.
	// Falls back to User if not set.
	BastionUser string

	// BastionKey is the private key content for the bastion host.
	// Falls back to PrivateKey if not set.
	BastionKey string

	// BastionKeyPath is the path to the private key for the bastion host.
	// Falls back to KeyPath if not set.
	BastionKeyPath string

	// BastionPassword is the password for the bastion host.
	BastionPassword string

	// AgentForwarding enables SSH agent forwarding.
	AgentForwarding bool

	// ProxyURL is the URL of a SOCKS5 proxy to dial through (e.g. "socks5://127.0.0.1:1080").
	// When set, all SSH connections (direct and bastion) are established through the proxy.
	// If empty, connections are made directly.
	ProxyURL string

	// Logger is the logger to use for debug and info messages.
	// If not set, a no-op logger is used (all messages discarded).
	Logger Logger
}

Config holds SSH connection configuration.

func (Config) WithDefaults

func (c Config) WithDefaults() Config

WithDefaults returns a copy of the config with default values applied.

type ConnectionPool

type ConnectionPool struct {
	// contains filtered or unexported fields
}

ConnectionPool manages reusable SSH connections. It caches connections by a key derived from connection parameters, allowing connection reuse across multiple resource operations.

func NewConnectionPool

func NewConnectionPool(maxIdle time.Duration) *ConnectionPool

NewConnectionPool creates a new connection pool. maxIdle specifies how long idle connections are kept before being closed.

func (*ConnectionPool) Close

func (p *ConnectionPool) Close()

Close closes all connections in the pool and stops the cleanup goroutine.

func (*ConnectionPool) CloseIdle

func (p *ConnectionPool) CloseIdle()

CloseIdle closes connections that have been idle for longer than maxIdle.

func (*ConnectionPool) GetOrCreate

func (p *ConnectionPool) GetOrCreate(config Config) (*Client, error)

GetOrCreate gets an existing connection or creates a new one. The caller must call Release() when done with the connection.

func (*ConnectionPool) Release

func (p *ConnectionPool) Release(config Config)

Release returns a connection to the pool.

func (*ConnectionPool) Stats

func (p *ConnectionPool) Stats() PoolStats

Stats returns current pool statistics.

type DirectorySyncResult

type DirectorySyncResult struct {
	// Files contains the result for each file.
	Files []SyncResult

	// TotalSize is the total size of all synced files.
	TotalSize int64

	// CombinedHash is a combined hash of all file hashes.
	CombinedHash string

	// Uploaded is the number of files uploaded.
	Uploaded int

	// Skipped is the number of files skipped (unchanged).
	Skipped int

	// Deleted is the number of files deleted.
	Deleted int

	// Errors is the number of files that failed.
	Errors int
}

DirectorySyncResult represents the result of a directory sync operation.

type FileAttributes

type FileAttributes struct {
	// Owner is the file owner (username or UID).
	Owner string

	// Group is the file group (group name or GID).
	Group string

	// Mode is the file permissions in octal (e.g., "0644").
	Mode string
}

FileAttributes represents file ownership and permissions.

type FileInfo

type FileInfo struct {
	RelPath       string
	Hash          string
	Size          int64
	IsSymlink     bool
	SymlinkTarget string
}

FileInfo holds information about a file.

func ScanDirectory

func ScanDirectory(root string, excludePatterns []string, symlinkPolicy string) ([]FileInfo, error)

ScanDirectory walks a directory and returns information about all files.

type Logger

type Logger interface {
	// Debugf logs a debug message.
	Debugf(format string, args ...interface{})
	// Infof logs an info message.
	Infof(format string, args ...interface{})
	// Warnf logs a warning message.
	Warnf(format string, args ...interface{})
	// Errorf logs an error message.
	Errorf(format string, args ...interface{})
}

Logger defines the interface for logging in gosftp. Users can implement this interface to integrate with their logging system.

type NoOpLogger

type NoOpLogger struct{}

NoOpLogger is a logger that discards all messages.

func (*NoOpLogger) Debugf

func (n *NoOpLogger) Debugf(format string, args ...interface{})

func (*NoOpLogger) Errorf

func (n *NoOpLogger) Errorf(format string, args ...interface{})

func (*NoOpLogger) Infof

func (n *NoOpLogger) Infof(format string, args ...interface{})

func (*NoOpLogger) Warnf

func (n *NoOpLogger) Warnf(format string, args ...interface{})

type PoolStats

type PoolStats struct {
	Total int
	InUse int
	Idle  int
}

PoolStats contains pool statistics.

type RetryConfig

type RetryConfig struct {
	// MaxRetries is the maximum number of retry attempts (0 = no retries).
	MaxRetries int

	// InitialDelay is the initial delay between retries.
	InitialDelay time.Duration

	// MaxDelay is the maximum delay between retries.
	MaxDelay time.Duration

	// Multiplier is the backoff multiplier (e.g., 2.0 = double delay each retry).
	Multiplier float64

	// JitterFactor adds randomness to delay (0.0 = no jitter, 0.5 = ±50% jitter).
	JitterFactor float64

	// Logger is the logger to use for retry messages.
	// If not set, a no-op logger is used (all messages discarded).
	Logger Logger
}

RetryConfig configures retry behavior for SSH operations.

func DefaultRetryConfig

func DefaultRetryConfig() RetryConfig

DefaultRetryConfig returns sensible default retry configuration.

func NoRetryConfig

func NoRetryConfig() RetryConfig

NoRetryConfig returns a config with retries disabled.

type RetryableFunc

type RetryableFunc func() error

RetryableFunc is a function that can be retried.

type SFTPClientInterface

type SFTPClientInterface interface {
	Open(path string) (SFTPFile, error)
	Create(path string) (SFTPFile, error)
	Remove(path string) error
	Stat(path string) (os.FileInfo, error)
	Chmod(path string, mode os.FileMode) error
	MkdirAll(path string) error
	Close() error
}

SFTPClientInterface abstracts SFTP operations for testing.

type SFTPClientWrapper

type SFTPClientWrapper struct {
	// contains filtered or unexported fields
}

SFTPClientWrapper wraps the real sftp.Client to implement SFTPClientInterface.

func (*SFTPClientWrapper) Chmod

func (w *SFTPClientWrapper) Chmod(path string, mode os.FileMode) error

func (*SFTPClientWrapper) Close

func (w *SFTPClientWrapper) Close() error

func (*SFTPClientWrapper) Create

func (w *SFTPClientWrapper) Create(path string) (SFTPFile, error)

func (*SFTPClientWrapper) MkdirAll

func (w *SFTPClientWrapper) MkdirAll(path string) error

func (*SFTPClientWrapper) Open

func (w *SFTPClientWrapper) Open(path string) (SFTPFile, error)

func (*SFTPClientWrapper) Remove

func (w *SFTPClientWrapper) Remove(path string) error

func (*SFTPClientWrapper) Stat

func (w *SFTPClientWrapper) Stat(path string) (os.FileInfo, error)

type SFTPFile

type SFTPFile interface {
	io.Reader
	io.Writer
	io.Closer
}

SFTPFile abstracts file operations for testing.

type StrictHostKeyChecking added in v0.2.0

type StrictHostKeyChecking string

StrictHostKeyChecking controls SSH host key verification behavior. This mirrors OpenSSH's StrictHostKeyChecking option.

const (
	// StrictHostKeyCheckingYes requires the host key to be in known_hosts.
	// Unknown hosts are rejected. This is the default behavior.
	StrictHostKeyCheckingYes StrictHostKeyChecking = "yes"

	// StrictHostKeyCheckingNo disables host key verification entirely.
	// WARNING: This is insecure and vulnerable to MITM attacks.
	StrictHostKeyCheckingNo StrictHostKeyChecking = "no"

	// StrictHostKeyCheckingAcceptNew accepts and saves keys for new hosts,
	// but rejects connections if a known host's key has changed.
	// This is a good balance between security and convenience.
	StrictHostKeyCheckingAcceptNew StrictHostKeyChecking = "accept-new"
)

type SyncOptions

type SyncOptions struct {
	// Attributes specifies file ownership and permissions to set.
	Attributes *FileAttributes

	// ExcludePatterns is a list of glob patterns to exclude from sync.
	// Example: []string{"*.tmp", ".git", "node_modules"}
	ExcludePatterns []string

	// SymlinkPolicy specifies how to handle symlinks: "follow", "skip", or "preserve".
	// Default is "follow".
	SymlinkPolicy string

	// Parallelism is the number of concurrent uploads for directory sync.
	// Default is 4.
	Parallelism int

	// DryRun only reports what would be synced without making changes.
	DryRun bool
}

SyncOptions configures sync behavior.

func (SyncOptions) WithDefaults

func (o SyncOptions) WithDefaults() SyncOptions

WithDefaults returns a copy of the options with default values applied.

type SyncResult

type SyncResult struct {
	// LocalPath is the source file path.
	LocalPath string

	// RemotePath is the destination file path.
	RemotePath string

	// Hash is the SHA256 hash of the file content.
	Hash string

	// Size is the file size in bytes.
	Size int64

	// Changed indicates if the file was uploaded (true) or unchanged (false).
	Changed bool

	// Deleted indicates if the file was deleted from the remote.
	Deleted bool

	// Error contains any error that occurred during sync.
	Error error
}

SyncResult represents the result of a sync operation.

type Syncer

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer provides a high-level API for file synchronization operations.

func NewSyncer

func NewSyncer(config Config, opts ...SyncerOption) (*Syncer, error)

NewSyncer creates a new Syncer with the given configuration.

func (*Syncer) Client

func (s *Syncer) Client() *Client

Client returns the underlying SSH client.

func (*Syncer) Close

func (s *Syncer) Close() error

Close closes the syncer and releases resources.

func (*Syncer) DeleteFile

func (s *Syncer) DeleteFile(ctx context.Context, remotePath string) error

DeleteFile deletes a file from the remote host.

func (*Syncer) SyncDirectory

func (s *Syncer) SyncDirectory(ctx context.Context, localDir, remoteDir string, opts *SyncOptions) (*DirectorySyncResult, error)

SyncDirectory synchronizes a directory to the remote host.

func (*Syncer) SyncFile

func (s *Syncer) SyncFile(ctx context.Context, localPath, remotePath string, opts *SyncOptions) (*SyncResult, error)

SyncFile synchronizes a single file to the remote host.

type SyncerOption

type SyncerOption func(*Syncer)

SyncerOption configures a Syncer.

func WithConnectionPool

func WithConnectionPool(pool *ConnectionPool) SyncerOption

WithConnectionPool enables connection pooling.

func WithRetryConfig

func WithRetryConfig(config RetryConfig) SyncerOption

WithRetryConfig sets the retry configuration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL