iron

package module
v0.4.72 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 8, 2026 License: BSD-3-Clause Imports: 32 Imported by: 1

README

iRODS Native Interface in Go

The iron utility is both a golang library and a command line client, that provides a clean, simple and stable interface to iRODS.

Quality Gate Status

CLI Installation

This documentation page contains all technical information about the library and contains only generic installation instructions. KU Leuven or VSC users should consult the RDM documentation pages instead for installation instructions. Users from other institutes should contact their respective support staff.

The CLI binary can be downloaded directly from https://github.com/kuleuven/iron/releases/latest. E.g. on a Linux system or Windows WSL:

VERSION=$(curl -Ls -w %{url_effective} -o /dev/null https://github.com/kuleuven/iron/releases/latest | sed 's/.*\/v//')
mkdir -p .local/bin/
curl -L -s "https://github.com/kuleuven/iron/releases/download/v${VERSION}/iron_${VERSION}_linux_amd64.tar.gz" | tar zxvf - -C .local/bin/

Alternatively, it can be installed with go install:

go install github.com/kuleuven/iron/cmd/iron@latest

Note that the released windows binaries are not signed. Signed binaries are available at the KU Leuven RDM documentation pages.

CLI usage

The CLI expects a ~.irods/irods_environment.json file being present, with native or pam_password authentication. The password should either be given in this file under the pam_password key, or the irods authentication file .irods/.irodsA must be present.

$ iron
Golang client for iRODS

Usage:
  iron [command]

Available Commands:
  checksum    Compute or get the checksum of a file
  chmod       Change permissions
  completion  Generate the autocompletion script for the specified shell
  cp          Copy a data object
  create      Create a data object
  download    Sync a collection to a local directory
  get         Download a file
  help        Help about any command
  inherit     Change permission inheritance
  ls          List a collection
  mkdir       Create a collection
  mv          Move a data object or collection
  put         Upload a file
  rm          Remove a data object or collection
  rmdir       Remove a collection
  shell       Start an interactive shell.
  stat        Get information about an object or collection
  tree        Print the full tree structure beneath a collection
  upload      Sync a local directory to a collection

Flags:
      --admin            Enable admin access
  -v, --debug count      Enable debug output
  -h, --help             help for iron
      --native           Use native protocol
      --workdir string   Working directory

Use "iron [command] --help" for more information about a command.
$ iron shell
iron > /set > ls
rods  0 B  Jul 07 18:31    home/
rods  0 B  Nov 11  2022    projects/
iron > /set > ls home/
rods          0 B  Jul 07 09:50     coz/
rods          0 B  Aug 08  2024     public/
set_demo      0 B  Feb 02 09:38  +  set_demo/
set_pilot013  0 B  Jan 01 19:02     set_pilot013/
iron > /set > cd home/coz/
iron > /set/home/coz > ls
peter  0 B     Jul 07 11:11     peter/
iron > /set/home/coz > ls peter
peter  0 B     Jul 07 11:11     peter.txt
iron > /set/home/coz > local pwd
/home/peter
iron > /set/home/coz > local cd sub
iron > /set/home/coz > local pwd
/home/peter/sub
iron > /set/home/coz > download peter localdir
localdir/peter.txt
iron > /set/home/coz > exit
$ ls /home/peter/sub/localdir
peter.txt

GUI

A Fyne-based desktop GUI is available as a separate project https://github.com/kuleuven/iron-gui. It reuses the same ~/.irods/irods_environment.json, .irodsA and .irodsA.json files as the CLI, supports pam_interactive authentication (browser launch

  • dialog input), and lets you browse collections, transfer files edit metadata/permissions, and compute checksums.

Library usage

import (   
	"github.com/kuleuven/iron"
	"github.com/kuleuven/iron/api"
)

func example() error {
    var env iron.Env

    err := env.LoadFromFile(".irods/irods_environment.json")
    if err != nil {
        return err
    }

    env.Password = "my_password"

    ctx := context.Background()

    client, err := iron.New(ctx, env, iron.Option{
        ClientName:        "iron",
        Admin:             false, // Set to true to do all operations as admin, bypassing any ACLs
        MaxConns:          5,
    })
    if err != nil {
        return err
    }

    defer client.Close()

    objects, err := client.ListDataObjectsInCollection(ctx, "/path/to/data")
    if err != nil {
        return err
    }

    for _, object := range objects {
        fmt.Println(object.Path)
    }

    // Recursive walk through the tree, displaying access and metadata
    fn := func(path string, info api.Record, err error) error {
        if err != nil {
            return nil
        }

        fmt.Println(path)
        fmt.Printf("%v", info.Access())
        fmt.Printf("%v", info.Metadata())

        return nil
    }

    return client.Walk(ctx, "/path/to/more/data", fn, api.FetchAccess, api.FetchMetadata)
}

Implementation choices

  • The client currently requires 4.3.2 or later. It is relatively easy to extend support to 4.2.9 - 4.3.1, if the truncate and touch operations are not needed.
  • New features might require a recent version of irods, i.e. support for older versions is not guaranteed.
  • Breaking changes in further versions are avoided as much as possible, but the API might change in the future if it is beneficial.
  • Simplified communication code: types of messages are defined in msg/types.go, and are marshaled using the right format (xml, json or binary) by msg.Marshal. The binary part (Bs) of messages is not marshaled by msg.Marshal/msg.Unmarshal but directly read or written to the provided buffers in msg.Read/msg.Write.
  • Clients can choose between iron.Conn (one single connection) and iron.Client (a pool of connections) to use the provided API.
  • The Truncate and Touch methods are only available on open file handles, to help identifying the right replica to adjust. Because irods only supports those operations when the file is closed, the operations are actually done on the replica when the file is closed.
  • This client also attempts to support the native protocol, but this should be considered experimental.

Known issues

For the iron CLI utility, the following issues are known:

  • Windows: The iron shell command does not resize properly. The window size is now fixed to 80x25.

Documentation

Overview

Package iron provides an interface to IRODS.

Index

Constants

View Source
const (
	ClientServerRefuseTLS  = "CS_NEG_REFUSE"
	ClientServerRequireTLS = "CS_NEG_REQUIRE"
	ClientServerDontCare   = "CS_NEG_DONT_CARE"
)
View Source
const (
	SSLVerifyServerCert = "cert"
	SSLVerifyServerHost = "host"
	SSLVerifyServerNone = "none"
)

Variables

View Source
var AuthenticateURL = regexp.MustCompile(`Please authenticate at (https?://[^\s]+)`)
View Source
var DefaultEnv = Env{
	Port:                          1247,
	AuthScheme:                    native,
	EncryptionAlgorithm:           "AES-256-CBC",
	EncryptionSaltSize:            8,
	EncryptionKeySize:             32,
	EncryptionNumHashRounds:       8,
	Username:                      "rods",
	SSLVerifyServer:               SSLVerifyServerCert,
	ClientServerNegotiation:       "request_server_negotiation",
	ClientServerNegotiationPolicy: "CS_NEG_REQUIRE",
	DefaultResource:               "demoResc",
	DialTimeout:                   time.Minute,
	HandshakeTimeout:              time.Minute,
	GeneratedPasswordTimeout:      time.Hour,
}

DefaultEnv contains the default IRODS connection parameters. Use ApplyDefaults() to apply the default values to an environment.

View Source
var ErrNoConnectionsAvailable = errors.New("no connections available")
View Source
var ErrNotImplemented = fmt.Errorf("not implemented")
View Source
var ErrSSLNegotiationFailed = fmt.Errorf("SSL negotiation failed")
View Source
var ErrTLSRequired = fmt.Errorf("TLS is required for authentication but not enabled")
View Source
var ErrTransport = errors.New("transport error")
View Source
var ErrUnknownSSLVerifyPolicy = fmt.Errorf("unknown SSL verification policy")
View Source
var ErrUnsupportedPlatform = errors.New("unsupported platform")
View Source
var ErrUnsupportedVersion = fmt.Errorf("unsupported server version")
View Source
var HandshakeTimeout = time.Minute

Functions

func DefaultDialFunc

func DefaultDialFunc(ctx context.Context, env Env, clientName string) (net.Conn, error)

func SystemOpenBrowser added in v0.4.9

func SystemOpenBrowser(url string) error

Types

type Bot added in v0.4.4

type Bot map[string]string

func (Bot) Ask added in v0.4.4

func (b Bot) Ask(message string) (string, error)

func (Bot) Password added in v0.4.4

func (b Bot) Password(message string) (string, error)

func (Bot) Print added in v0.4.4

func (b Bot) Print(message string) error

type Client

type Client struct {
	*api.API
	// contains filtered or unexported fields
}

func New

func New(ctx context.Context, env Env, option Option) (*Client, error)

New creates a new Client instance with the provided environment settings, maximum connections, and options. The environment settings are used for dialing new connections. The maximum number of connections is the maximum number of connections that can be established at any given time. The options are used to customize the behavior of the client.

func (*Client) Close

func (c *Client) Close() error

Close closes all connections managed by the client, ensuring that any errors encountered during the closing process are aggregated and returned. The method is safe to call multiple times and locks the client during execution to prevent concurrent modifications to the connections.

func (*Client) ComputeChecksums added in v0.4.56

func (c *Client) ComputeChecksums(ctx context.Context, remote string, options transfer.Options) error

ComputeChecksums computes the checksums of a remote directory on the iRODS server using client recursion. The remote file refers to an iRODS path.

func (*Client) Connect

func (c *Client) Connect(ctx context.Context) (Conn, error)

Connect returns a new connection to the iRODS server. It will first try to reuse an available connection. If all connections are busy, it will create a new one up to the maximum number of connections. If the maximum number of connections has been reached, it will block until a connection becomes available, or reuse an existing connection in case AllowConcurrentUse is enabled.

func (*Client) ConnectAvailable added in v0.3.11

func (c *Client) ConnectAvailable(ctx context.Context, n int) ([]Conn, error)

ConnectAvailable returns a list of available connections to the iRODS server, up to the specified number. If no connections are available, it will return an empty list. Retrieved connections must be closed by the caller. If n is negative, it will return all available connections.

func (*Client) CopyDir added in v0.4.16

func (c *Client) CopyDir(ctx context.Context, remote1, remote2 string, options transfer.Options) error

CopyDir copies a remote directory to another remote directory from the iRODS server using client recursion. The remote files refers to an iRODS path.

func (*Client) Download added in v0.3.9

func (c *Client) Download(ctx context.Context, local, remote string, options transfer.Options) error

Download downloads a remote file from the iRODS server using parallel transfers. The local file refers to the local file system. The remote file refers to an iRODS path.

func (*Client) DownloadDir added in v0.3.11

func (c *Client) DownloadDir(ctx context.Context, local, remote string, options transfer.Options) error

DownloadDir downloads a remote directory from the iRODS server using parallel transfers. The local file refers to the local file system. The remote file refers to an iRODS path.

func (*Client) Env

func (c *Client) Env() Env

Env returns the client environment.

func (*Client) FromReader added in v0.4.37

func (c *Client) FromReader(ctx context.Context, r io.Reader, remote string, appendToFile bool, options transfer.Options) error

FromReader streams an io.Reader to a remote file on the iRODS server using parallel transfers. The remote file refers to an iRODS path.

func (*Client) Option

func (c *Client) Option() Option

Option returns the client options.

func (*Client) Pool added in v0.3.12

func (c *Client) Pool(size int) (*Pool, error)

Pool returns a subset of connections that can be used for dedicated API calls. The connections remain in the pool until the pool is Closed. The call will block until the requested number of connections are available.

func (*Client) RemoveDir added in v0.4.13

func (c *Client) RemoveDir(ctx context.Context, remote string, options transfer.Options) error

RemoveDir removes a remote directory from the iRODS server using client recursion. The remote file refers to an iRODS path.

func (*Client) ReplaceDefaultPool added in v0.4.37

func (c *Client) ReplaceDefaultPool(pool *Pool) *Pool

ReplaceDefaultPool sets the default pool, i.e. the pool from which connections are taken if client.Connect() is called. The previous default pool will be returned. Use with care, it will overwrite the default pool, but subpools of the existing pool will retain their original parent pool, and the caller is responsible for closing the returned original pool.

func (*Client) ToWriter added in v0.4.37

func (c *Client) ToWriter(ctx context.Context, w io.Writer, remote string, options transfer.Options) error

ToWriter streams a remote file from the iRODS server to an io.Writer using parallel transfers. The remote file refers to an iRODS path.

func (*Client) Upload added in v0.3.9

func (c *Client) Upload(ctx context.Context, local, remote string, options transfer.Options) error

Upload uploads a local file to the iRODS server using parallel transfers. The local file refers to the local file system. The remote file refers to an iRODS path.

func (*Client) UploadDir added in v0.3.11

func (c *Client) UploadDir(ctx context.Context, local, remote string, options transfer.Options) error

UploadDir uploads a local directory to the iRODS server using parallel transfers. The local file refers to the local file system. The remote file refers to an iRODS path.

func (*Client) Verify added in v0.3.9

func (c *Client) Verify(ctx context.Context, local, remote string) error

Verify checks the checksum of a local file against the checksum of a remote file

type Conn

type Conn interface {
	// Env returns the connection environment
	Env() Env

	// Transport returns the underlying net.Conn
	Transport() net.Conn

	// ServerVersion returns the version that the iRODS server reports
	// e.g. "4.3.2"
	ServerVersion() string

	// ClientSignature returns the client signature
	ClientSignature() string

	// NativePassword returns the native password
	// In case of PAM authentication, this is the generated password
	NativePassword() string

	// Request sends an API request for the given API number and expects a response.
	// Both request and response should represent a type such as in `msg/types.go`.
	// The request and response will be marshaled and unmarshaled automatically.
	// If a negative IntInfo is returned, an appropriate error will be returned.
	// This method is thread-safe.
	Request(ctx context.Context, apiNumber msg.APINumber, request, response any) error

	// RequestWithBuffers behaves as Request, with provided buffers for the request
	// and response binary data. Both requestBuf and responseBuf could be nil.
	// This method is thread-safe.
	RequestWithBuffers(ctx context.Context, apiNumber msg.APINumber, request, response any, requestBuf, responseBuf []byte) error

	// API returns an API using the current connection.
	API() *api.API

	// Close closes the connection.
	// It is safe to call Close multiple times.
	// This method is thread-safe but will obviously make future requests fail.
	Close() error

	// RegisterCloseHandler registers a function to be called when the connection is
	// about to closed. It is used to clean up state before the connection is closed.
	// The CloseHandler can be unregistered by calling the returned function.
	RegisterCloseHandler(handler func() error) context.CancelFunc

	// ConnectedAt returns the time when the connection was established.
	ConnectedAt() time.Time

	// TransportErrors returns the number of transport errors encountered on this connection.
	TransportErrors() int

	// SQLErrors returns the number of SQL errors encountered on this connection.
	SQLErrors() int
}

func Dial

func Dial(ctx context.Context, env Env, clientName string) (Conn, error)

Dial connects to an IRODS server and creates a new connection. The caller is responsible for closing the connection when it is no longer needed.

func NewConn

func NewConn(ctx context.Context, transport net.Conn, env Env, clientName string) (Conn, error)

NewConn initializes a new Conn instance with the provided network connection and environment settings. It performs a handshake as part of the initialization process and returns the constructed Conn instance. Returns an error if the handshake fails.

func NewPromptConn added in v0.4.4

func NewPromptConn(ctx context.Context, transport net.Conn, env Env, prompt Prompt, clientName string) (Conn, error)

NewPromptConn initializes a new Conn instance with the provided network connection and environment settings. It performs a handshake as part of the initialization process and returns the constructed Conn instance. For interactive authentication, the provided prompt will be used. Returns an error if the handshake fails.

func PromptDial added in v0.4.4

func PromptDial(ctx context.Context, env Env, prompt Prompt, clientName string) (Conn, error)

PromptDial connects to an IRODS server and creates a new connection. Prompt is used for possible interactive authentication. The caller is responsible for closing the connection when it is no longer needed.

type DialFunc

type DialFunc func(ctx context.Context, env Env, clientName string) (net.Conn, error)

DialFunc is used to connect to an IRODS server.

type Env

type Env struct {
	// Credentials and settings
	Host                          string `json:"irods_host"`
	Port                          int    `json:"irods_port"`
	Zone                          string `json:"irods_zone_name"`
	AuthScheme                    string `json:"irods_authentication_scheme"`
	EncryptionAlgorithm           string `json:"irods_encryption_algorithm"`
	EncryptionSaltSize            int    `json:"irods_encryption_salt_size"`
	EncryptionKeySize             int    `json:"irods_encryption_key_size"`
	EncryptionNumHashRounds       int    `json:"irods_encryption_num_hash_rounds"`
	Username                      string `json:"irods_user_name"`
	Password                      string `json:"irods_password,omitempty"`
	SSLCACertificateFile          string `json:"irods_ssl_ca_certificate_file"`
	SSLVerifyServer               string `json:"irods_ssl_verify_server"`
	ClientServerNegotiation       string `json:"irods_client_server_negotiation"`
	ClientServerNegotiationPolicy string `json:"irods_client_server_policy"`
	DefaultResource               string `json:"irods_default_resource"`
	ProxyUsername                 string `json:"irods_proxy_user"` // Authenticate with proxy credentials
	ProxyZone                     string `json:"irods_proxy_zone"` // Authenticate with proxy credentials
	IrodsAuthenticationUID        *int   `json:"irods_authentication_uid,omitempty"`

	// For pam authentication, request to generate a password that is valid for the given TTL.
	// The server will determine the actual TTL based on the server thresholds.
	// This value is rounded down to the nearest hour. If zero, the timeout will default to 2m1s.
	GeneratedPasswordTimeout time.Duration `json:"-"`

	// Persistent state for PAM interactive authentication
	PersistentState PersistentState `json:"-"` // If provided, this will be used for PAM authentication as persistent store

	// Advanced dial settings
	SSLServerName    string        `json:"-"` // If provided, this will be used for server verification, instead of the hostname
	DialTimeout      time.Duration `json:"-"` // If provided, this will be used for dial timeout
	HandshakeTimeout time.Duration `json:"-"` // If provided, this will be used for the handshake timeout
}

Env contains the IRODS connection parameters to establish a connection.

func (*Env) ApplyDefaults

func (env *Env) ApplyDefaults()

ApplyDefaults sets default values for the environment fields if they are not already set. It uses the values from DefaultEnv for most fields. If the ProxyUsername and ProxyZone are not specified, it uses the Username and Zone respectively. Additionally, if PamTTL is not set or is less than or equal to zero, it defaults to 60

func (*Env) LoadFromFile

func (env *Env) LoadFromFile(path string) error

LoadFromFile reads an environment configuration from a JSON file at the given path, overwriting the fields of the receiver.

type HandshakeFunc added in v0.4.37

type HandshakeFunc func(ctx context.Context) (Conn, error)

type Option

type Option struct {
	// ClientName is passed to the server as the client type
	ClientName string

	// DeferConnectionToFirstUse will defer the creation of the initial connection to
	// the first use of the Connect() method.
	DeferConnectionToFirstUse bool

	// AtFirstUse is an optional function that is called when the first connection is established,
	// before the connection is returned to the caller of Connect().
	AtFirstUse func(*api.API)

	// Maximum number of connections that can be established at any given time.
	MaxConns int

	// AllowConcurrentUse will allow multiple goroutines to use the same connection concurrently,
	// if the maximum number of connections has been reached and no connection is available.
	// Connect() will cycle through the existing connections.
	AllowConcurrentUse bool

	// Admin is a flag that indicates whether the client should act in admin mode.
	Admin bool

	// Experimental: UseNativeProtocol will force the use of the native protocol.
	// This is an experimental feature and may be removed in a future version.
	UseNativeProtocol bool

	// EnvCallback is an optional function that returns the environment settings for the connection
	// when a new connection is established. If not provided, the default environment settings are used.
	// This is useful in combination with the DeferConnectionToFirstUse option, to prepare the client
	// before the connection parameters are known. The returned time.Time is the time until which the
	// environment settings are valid, or zero if they are valid indefinitely.
	EnvCallback func() (Env, time.Time, error)

	// AuthenticationPrompt is an optional function that overrides the default pompt function.
	// It is used to prompt the user for information when authenticating with the server,
	// if the authentication scheme is pam_interactive.
	AuthenticationPrompt Prompt

	// DialFunc is an optional function that overrides the default dial function.
	DialFunc DialFunc

	// HandshakeFunc is an optional function that overrides the default handshake function.
	HandshakeFunc HandshakeFunc

	// DiscardConnectionAge is the maximum age of a connection before it is discarded.
	DiscardConnectionAge time.Duration
}

type PersistentState added in v0.4.10

type PersistentState interface {
	Load(m map[string]any) error // Load returns the current persistent state into the given map.
	Save(m map[string]any) error // Save persists the given state. Later calls to Load will reconstruct the same map contents.
}

PersistentState refers to the pstate used for PAM interactive authentication The functions must be thread-safe if multiple clients/connections are used concurrently.

type Pool added in v0.3.11

type Pool struct {
	*api.API
	// contains filtered or unexported fields
}

Pool returns a subset of connections that can be used for dedicated API calls. The connections remain in the pool until the pool is Closed.

func (*Pool) Close added in v0.3.11

func (p *Pool) Close() error

Close returns all connections managed by the pool to the parent pool. Connections that were not returned to the pool yet, will be returned to the parent pool later.

func (*Pool) Connect added in v0.3.11

func (p *Pool) Connect(ctx context.Context) (Conn, error)

Connect returns a new connection to the iRODS server. It will first try to reuse an available connection. If all connections are busy, it will create a new one up to the maximum number of connections. If the maximum number of connections has been reached, it will block until a connection becomes available, or reuse an existing connection in case AllowConcurrentUse is enabled.

func (*Pool) ConnectAvailable added in v0.3.11

func (p *Pool) ConnectAvailable(ctx context.Context, n int) ([]Conn, error)

ConnectAvailable returns a list of available connections to the iRODS server, up to the specified number. If no connections are available, it will return an empty list. Retrieved connections must be closed by the caller. If n is negative, it will return all available connections.

func (*Pool) Pool added in v0.3.11

func (p *Pool) Pool(size int) (*Pool, error)

Pool returns a subpool of connections It will block until the requested number of connections are available.

type Prompt added in v0.4.4

type Prompt interface {
	Print(message string) error
	Ask(message string) (string, error)
	Password(message string) (string, error)
}
var StdPrompt Prompt = &prompt{
	r: os.Stdin,
	w: os.Stdout,
}

Directories

Path Synopsis
cmd
iron command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL