Storage

The Storage service provides S3-compatible object storage backed by MinIO with erasure coding across multiple hosts. The Storage Agent manages cluster deployment, health monitoring, and configuration.

Features

• S3-compatible API — use any S3 SDK or tool (aws-cli, boto3, mc) to interact with your storage
• Erasure coding — data is encoded and distributed across 4 or more hosts for durability, surviving up to half the drives failing
• Bucket policies — fine-grained access control using S3-compatible bucket policies
• Lifecycle rules — automatically transition or expire objects based on age
• Versioning — keep multiple versions of objects for rollback and audit purposes

Create a Bucket

agentmetal bucket create --name assets --region fsn1

Configure Lifecycle Rules

agentmetal bucket lifecycle set assets \
  --rule "expire-old:prefix=logs/:expiry=30d" \
  --rule "archive:prefix=backups/:transition=90d:tier=glacier"

Enable Versioning

agentmetal bucket versioning enable assets

Access Your Storage

After creating a bucket, retrieve the S3 endpoint and credentials:

agentmetal bucket get assets --credentials

Use with the AWS CLI:

aws s3 ls s3://assets/ --endpoint-url https://s3.fsn1.agentmetal.local

Or from your application using any S3 SDK by pointing the endpoint to your AgentMetal storage URL.

What the Agent Manages

The Storage Agent performs these operations:

1. MinIO cluster deployment — installs and configures MinIO across multiple bare-metal hosts with erasure coding
2. Erasure coding configuration — sets up EC profiles to distribute data across available drives, ensuring durability even if multiple drives fail simultaneously
3. Health monitoring — continuously checks disk health, node availability, and replication status
4. Capacity management — tracks usage per bucket and alerts when approaching capacity limits
5. Self-healing — replaces degraded drives and rebalances data automatically

Bucket Policies

Apply an S3-compatible bucket policy for access control:

agentmetal bucket policy set assets --policy policy.json

Example policy allowing public read:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::assets/public/*"
    }
  ]
}

Architecture

MinIO is deployed with a minimum of 4 nodes and 4 drives per node. Data is erasure-coded with a default parity of 4, meaning the cluster can lose up to 4 drives and still serve reads and writes without data loss.