ffstools

Command-line tools for FFS

See also https://github.com/creachadair/ffs.

Overview

• The ffs tool supports running and communicating with a blob storage service, and provides commands to manipulate the contents of the store as FFS specific messages.

  # To install:
  go install github.com/creachadair/ffstools/ffs@latest
  

  When installing, you may want to specify build --tags to enable other storage backends. See Storage Backends.

• The file2json tool decodes wire-format node messages and translates them to JSON for easier reading by humans.

  # To install:
  go install github.com/creachadair/ffstools/file2json@latest
  

Installation and Usage

Install ffs as noted above, then the following example should work as-written:

# Start up a storage server using local files as storage, and run a
# shell with the server addressed via the FFS_STORE environment.
# THe storage server stops after the shell exits.
ffs storage --store file:test.db --listen demo.sock --exec bash -s <<EOF
  # Create a root pointer to anchor some data.
  ffs root create example --desc 'Example root pointer'

  # Create some files to import into storage.
  mkdir -p files/sub
  echo "This is my file." > files/sub/f1.txt
  echo "Many others are like it" > files/sub/f2.txt
  echo "But this one is mine." > files/f3.txt

  # Copy the files directory into the store.
  ffs import --into example/test1 files

  # List the contents we just wrote...
  ffs file list -long example/test1
  ffs file list -long example/test1/sub

  # Move some files around...
  echo "That was your file." > files/sub/f1.txt
  mv files/sub/f2.txt files/f4.txt
  rm files/f3.txt

  # Add another copy of the structure.
  ffs import -into example/test2 files

  # List the revised contents...
  ffs file list -long example/test1
  ffs file list -long example/test2
  ffs file list -long example/test2/sub

  # List the stuff reachable from the root.
  ffs file list -long -key example

  # GC unreachable data in the store.
  ffs gc
EOF

Encryption

The ffs storage subcommand supports encrypting the underlying store, using the --key flag, which accepts the path of a file containing the encryption key. In this mode, each object in the store is encrypted with an AEAD over ChaCha20poly1305.

To generate a random keyring file, for example:

% ffs storage keygen example.keyring
Passphrase: ****
(confirm) Passphrase: ****
Wrote a new 32-byte keyring to "example.keyring"

This will prompt you to enter a passphrase to unlock the keyring, and initialize it with a randomly-generated key. You will be prompted again when starting up the store:

% ffs storage --store file:test.db --listen db.sock --key example.keyring
Passphrase for "example.keyring": ****

Storage Backends

The following storage backends are currently supported by default:

The following storage backends can be enabled by building with the specified tags:

For local storage, I generally use BadgerDB unless I want a single file, in which case I currently prefer BuntDB.

[!WARNING] Some of the storage engines used here have begun accepting commits largely authored by LLM coding tools. It is unclear whether the maintainers are doing appropiate diligence on the results. Use caution in applying them for production uses.