A w3up client in golang. ⚠️ Heavily WIP.

go get github.com/storacha/guppy

⚠️ Heavily WIP. At time of writing the client/CLI does not yet upload arbitrary files or directories, only prepared CAR files.

There are two ways to use the client library: you can get a user to interactively log in, or bring a prepared, authorized identity.

To have the user log in, use (*client.Client) RequestAccess() to have the service ask the user to authenticate, and (*client.Client) PollClaim() to notice when they do. (Example)

To bring your own pre-authorized identity, instantiate the client with the option client.WithPrincipal(signer). (Example) You'll first need to generate a DID and then delegate capabilities to that identity.

The CLI will automatically generate an identity for you and store it in ~/.guppy/config. Like the library, there are two ways to authenticate the CLI client: interactively, or by authorizing in advance.

To authorize interactively, use go run ./cmd login and follow the prompts.

To authorize in advance, use go run ./cmd whoami to see the client's DID and then delegate capabilities to that identity. Then, pass the proofs you create on the command line whenever you use the CLI.

NAME:
   guppy - interact with the Storacha Network

USAGE:
   guppy [global options] command [command options] [arguments...]

COMMANDS:
   whoami      Print information about the current agent.
   login       Authenticate this agent with your email address to gain access to all capabilities that have been delegated to it.
   up, upload  Store a file(s) to the service and register an upload.
   ls, list    List uploads in the current space.
   help, h     Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h  show help

While both Guppy and the Storacha JS CLI provide access to the Storacha Network, they serve different use cases and are optimized for different scenarios:

Storacha JS CLI: Designed for easy integration in web applications and straightforward file uploads. Perfect for developers who need simple, quick uploads and standard web-based workflows.

Guppy: Built specifically for enterprise-scale, large data uploads with a focus on efficiency, reliability, and incremental updates. Ideal for organizations managing substantial datasets that change over time.

Guppy provides several enterprise-focused features that set it apart:

Resumable Uploads: Guppy can resume interrupted uploads, ensuring that large datasets don't need to be re-uploaded from scratch if something goes wrong.

Multiple Data Source Support: Pull data from various sources including:

• Local filesystems
• Remote data sources
• Cloud storage providers
• Other network locations

Incremental & Updatable Uploads: Rather than treating each upload as independent, Guppy supports incremental uploads that only transfer what has changed since the last upload, dramatically reducing bandwidth and time for large, evolving datasets.

Parallel Processing: Built-in parallel processing and concurrent data pulling capabilities to maximize throughput for large uploads.

In Guppy, a "space" represents more than just a collection of uploads. Each space is configured with a set of data sources, and Guppy treats uploads intelligently:

• Initial Upload: The first upload from a data source captures the complete dataset
• Subsequent Uploads: All following uploads are treated as "updates" that only include changes (additions, modifications, deletions) since the last upload
• Change Detection: Guppy automatically detects what has changed and only processes the delta, making updates extremely efficient even for massive datasets

This approach makes Guppy particularly well-suited for scenarios like:

• Regular backups of large databases
• Synchronizing evolving datasets
• Enterprise content management systems
• Any situation where large amounts of data change incrementally over time

You can use ucan-key to generate a private key and DID for use with the library. Install Node.js and then use the ucan-key module:

npx ucan-key ed

Output should look something like:

# did:key:z6Mkh9TtUbFJcUHhMmS9dEbqpBbHPbL9oxg1zziWn1CYCNZ2
MgCb+bRGl02JqlWMPUxCyntxlYj0T/zLtR2tn8LFvw6+Yke0BKAP/OUu2tXpd+tniEoOzB3pxqxHZpRhrZl1UYUeraT0=

You can use the private key (the line starting Mg...) in the CLI by setting the environment variable GUPPY_PRIVATE_KEY. Alternatively you can use it programmatically after parsing it:

package main

import "github.com/web3-storage/go-ucanto/principal/ed25519/signer"

signer, _ := signer.Parse("MgCb+bRGl02JqlWMPUxCyntxlYj0T/zLtR2tn8LFvw6+Yke0BKAP/OUu2tXpd+tniEoOzB3pxqxHZpRhrZl1UYUeraT0=")

Proofs are delegations to your DID enabling it to perform tasks. Currently the best way to obtain proofs that will allow you to interact with the Storacha Network is to use the Storacha JS CLI:

1. Generate a DID and make a note of it (the string starting with did:key:...)
2. Install w3 CLI:

   npm install -g @storacha/cli

3. Create a space:

   storacha space create <NAME>

4. Delegate capabilities to your DID:

   storacha delegation create -c 'store/*' -c 'upload/*' <DID>`

pkg.go.dev Reference

Feel free to join in. All welcome. Please open an issue!

Dual-licensed under MIT + Apache 2.0