S3 API Usage

ZSoftly object storage is S3-compatible. Any tool or SDK that supports S3 works with it.

Endpoints:

Region	Endpoint
YUL (Montreal)	https://objects.yul.zcp.zsoftly.ca
YOW (Ottawa)	https://objects.yow.zcp.zsoftly.ca

Examples below use the YUL endpoint. Substitute YOW if your bucket is in Ottawa.

Find your bucket name

Before you write an endpoint URL, confirm the bucket’s real name. The platform appends a numeric suffix to the name you asked for when the bucket is created. A bucket you requested as my-bucket exists as something like my-bucket-001024. That suffixed value is the name the S3 API answers to. Using the name you typed gives NoSuchBucket even though the bucket exists.

zcp object-storage bucket list <storage-slug> --region os-yul --project <project-slug>

SLUG                NAME                OBJECTS  SIZE (GB)  STATUS
my-bucket-001024    my-bucket-001024    0                   Inactive

Use that value everywhere below in place of my-bucket. A bucket showing Inactive with zero objects is normal until something is written to it.

Note

The CLI’s own bucket subcommands are split. Commands that go through the S3 gateway (object …, bucket versioning, bucket policy, bucket lifecycle, bucket cors, bucket tag, bucket empty) take the bucket name, even though their help text labels the argument <bucket-slug>. Only bucket delete, which goes through the control plane, takes the slug. In practice the name and the slug are usually the same suffixed string.

Connect and list buckets

Configure your client with the regional endpoint and your access keys, then list your buckets. Pick your language:

AWS CLI

# Store your credentials once
aws configure set aws_access_key_id <access-key>
aws configure set aws_secret_access_key <secret-key>

# List buckets
aws s3 ls --endpoint-url https://objects.yul.zcp.zsoftly.ca

Python

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="https://objects.yul.zcp.zsoftly.ca",
    aws_access_key_id="<access-key>",
    aws_secret_access_key="<secret-key>",
)

for bucket in s3.list_buckets()["Buckets"]:
    print(bucket["Name"])

JavaScript

import { S3Client, ListBucketsCommand } from '@aws-sdk/client-s3';

const client = new S3Client({
  endpoint: 'https://objects.yul.zcp.zsoftly.ca',
  region: 'us-east-1',
  credentials: { accessKeyId: '<access-key>', secretAccessKey: '<secret-key>' },
  forcePathStyle: true,
});

const { Buckets } = await client.send(new ListBucketsCommand({}));
for (const b of Buckets ?? []) console.log(b.Name);

TypeScript

import { S3Client, ListBucketsCommand, type Bucket } from '@aws-sdk/client-s3';

const client = new S3Client({
  endpoint: 'https://objects.yul.zcp.zsoftly.ca',
  region: 'us-east-1',
  credentials: { accessKeyId: '<access-key>', secretAccessKey: '<secret-key>' },
  forcePathStyle: true,
});

const { Buckets } = await client.send(new ListBucketsCommand({}));
Buckets?.forEach((b: Bucket) => console.log(b.Name));

Go

package main

import (
  "context"
  "fmt"

  "github.com/aws/aws-sdk-go-v2/aws"
  "github.com/aws/aws-sdk-go-v2/credentials"
  "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
  client := s3.New(s3.Options{
    BaseEndpoint: aws.String("https://objects.yul.zcp.zsoftly.ca"),
    Region:       "us-east-1",
    Credentials:  credentials.NewStaticCredentialsProvider("<access-key>", "<secret-key>", ""),
    UsePathStyle: true,
  })

  out, err := client.ListBuckets(context.TODO(), &s3.ListBucketsInput{})
  if err != nil {
    panic(err)
  }
  for _, b := range out.Buckets {
    fmt.Println(*b.Name)
  }
}

C#

using Amazon.S3;

var config = new AmazonS3Config
{
    ServiceURL = "https://objects.yul.zcp.zsoftly.ca",
    ForcePathStyle = true,
};
using var client = new AmazonS3Client("<access-key>", "<secret-key>", config);

var response = await client.ListBucketsAsync();
foreach (var bucket in response.Buckets)
{
    Console.WriteLine(bucket.BucketName);
}

Java

import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3Configuration;

import java.net.URI;

public class ListBuckets {
    public static void main(String[] args) {
        S3Client s3 = S3Client.builder()
            .endpointOverride(URI.create("https://objects.yul.zcp.zsoftly.ca"))
            .region(Region.US_EAST_1)
            .credentialsProvider(StaticCredentialsProvider.create(
                AwsBasicCredentials.create("<access-key>", "<secret-key>")))
            .serviceConfiguration(S3Configuration.builder()
                .pathStyleAccessEnabled(true).build())
            .build();

        s3.listBuckets().buckets().forEach(b -> System.out.println(b.name()));
    }
}

Ruby

require "aws-sdk-s3"

s3 = Aws::S3::Client.new(
  endpoint: "https://objects.yul.zcp.zsoftly.ca",
  region: "us-east-1",
  access_key_id: "<access-key>",
  secret_access_key: "<secret-key>",
  force_path_style: true
)

s3.list_buckets.buckets.each { |b| puts b.name }

PHP

<?php
require "vendor/autoload.php";

use Aws\S3\S3Client;

$s3 = new S3Client([
    "version" => "latest",
    "region" => "us-east-1",
    "endpoint" => "https://objects.yul.zcp.zsoftly.ca",
    "use_path_style_endpoint" => true,
    "credentials" => [
        "key" => "<access-key>",
        "secret" => "<secret-key>",
    ],
]);

foreach ($s3->listBuckets()["Buckets"] as $bucket) {
    echo $bucket["Name"] . "\n";
}

Tip

Always enable path-style addressing: forcePathStyle (JS/TS), UsePathStyle (Go), ForcePathStyle (C#), pathStyleAccessEnabled (Java), force_path_style (Ruby), or use_path_style_endpoint (PHP). ZSoftly object storage requires it. Virtual-hosted style is not supported.

Danger

Do not set default bucket encryption (SSE-S3). The storage gateway has no encryption key backend yet, and a bucket with default encryption configured rejects every subsequent upload with InvalidArgument until the setting is removed. This applies to PutBucketEncryption from any SDK or aws s3api put-bucket-encryption.

Clear the setting on a bucket that already has it:

zcp object-storage bucket encryption status <storage-slug> <bucket-name>
zcp object-storage bucket encryption disable <storage-slug> <bucket-name>

To encrypt what you store today, encrypt on the client before upload. See Back Up Vaultwarden to Object Storage with restic for a worked example.

Common operations

Once your client (or the AWS CLI) is configured, all standard S3 operations work. Using the AWS CLI:

# Create a bucket
aws s3 mb s3://my-bucket --endpoint-url https://objects.yul.zcp.zsoftly.ca

# Upload a file
aws s3 cp ./file.txt s3://my-bucket/ --endpoint-url https://objects.yul.zcp.zsoftly.ca

# Download a file
aws s3 cp s3://my-bucket/file.txt ./file.txt --endpoint-url https://objects.yul.zcp.zsoftly.ca

# List objects in a bucket
aws s3 ls s3://my-bucket/ --endpoint-url https://objects.yul.zcp.zsoftly.ca

# Delete an object
aws s3 rm s3://my-bucket/file.txt --endpoint-url https://objects.yul.zcp.zsoftly.ca

s3cmd

s3cmd is a standalone command-line client for S3-compatible storage. Install it, then create ~/.s3cfg:

[default]
access_key = <access-key>
secret_key = <secret-key>
host_base = objects.yul.zcp.zsoftly.ca
host_bucket = objects.yul.zcp.zsoftly.ca
use_https = True

This file holds your secret key, so restrict its permissions:

chmod 600 ~/.s3cfg

Setting host_bucket to the same value as host_base, with no bucket placeholder, forces the path-style addressing the service requires. For a bucket in Ottawa, use objects.yow.zcp.zsoftly.ca.

Run the common commands:

s3cmd ls                                    # list your buckets
s3cmd mb s3://my-bucket                      # create a bucket
s3cmd put ./file.txt s3://my-bucket/         # upload a file
s3cmd get s3://my-bucket/file.txt .          # download a file
s3cmd sync ./local-dir/ s3://my-bucket/dir/  # sync a folder
s3cmd rm s3://my-bucket/file.txt             # delete an object

See also: Access Keys, Create Bucket