Getting Started

Installation01

Install the core Trove package:

go get github.com/xraph/trove

Then install one or more storage drivers:

bash

go get github.com/xraph/trove/drivers/localdriver

Create a Driver02

Every Trove instance requires a storage driver. Drivers implement driver.Driver and handle the actual backend I/O.

import "github.com/xraph/trove/drivers/localdriver"

drv := localdriver.New()
err := drv.Open(ctx, "file:///var/data/storage")

Open a Trove Instance03

Pass the driver to trove.Open to create a Trove handle. All subsequent operations flow through this handle.

import "github.com/xraph/trove"

t, err := trove.Open(drv)
if err != nil {
    log.Fatal(err)
}
defer t.Close(ctx)

Open accepts functional options for configuration:

t, err := trove.Open(drv,
    trove.WithDefaultBucket("primary"),
    trove.WithChunkSize(8 * 1024 * 1024),  // 8 MB streaming chunks
    trove.WithPoolSize(32),                 // max concurrent streams
)

Create a Bucket04

Buckets are top-level namespaces that group objects. Create one before storing objects:

err := t.CreateBucket(ctx, "uploads")

Note

If you set a default bucket with trove.WithDefaultBucket("uploads"), you can pass an empty string for the bucket parameter in subsequent operations and Trove will use the default.

Store an Object05

Put stores an object from any io.Reader. It returns *driver.ObjectInfo with the stored object's metadata.

import (
    "strings"
    "github.com/xraph/trove/driver"
)

info, err := t.Put(ctx, "uploads", "photos/cat.jpg", file,
    driver.WithContentType("image/jpeg"),
    driver.WithMetadata(map[string]string{
        "author":      "alice",
        "uploaded-by": "web-app",
    }),
)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("stored %s (%d bytes, etag=%s)\n", info.Key, info.Size, info.ETag)

For simple text content:

info, err := t.Put(ctx, "uploads", "notes/hello.txt",
    strings.NewReader("Hello, Trove!"),
    driver.WithContentType("text/plain"),
)

Retrieve an Object06

Get returns an *driver.ObjectReader that combines a ReadCloser for the content with an Info field containing metadata.

obj, err := t.Get(ctx, "uploads", "photos/cat.jpg")
if err != nil {
    log.Fatal(err)
}
defer obj.Close()

// Access metadata via obj.Info
fmt.Printf("content-type: %s, size: %d\n", obj.Info.ContentType, obj.Info.Size)

// Read content
data, err := io.ReadAll(obj)
if err != nil {
    log.Fatal(err)
}

List Objects07

List returns an *driver.ObjectIterator for cursor-based iteration. Use WithPrefix to filter by key prefix.

Collect all results at once:

iter, err := t.List(ctx, "uploads", driver.WithPrefix("photos/"))
if err != nil {
    log.Fatal(err)
}
defer iter.Close()

objects, err := iter.All(ctx)
if err != nil {
    log.Fatal(err)
}

for _, obj := range objects {
    fmt.Printf("  %s  %d bytes  %s\n", obj.Key, obj.Size, obj.ContentType)
}

Iterate one at a time:

iter, err := t.List(ctx, "uploads", driver.WithPrefix("photos/"))
if err != nil {
    log.Fatal(err)
}
defer iter.Close()

for {
    obj, err := iter.Next(ctx)
    if errors.Is(err, io.EOF) {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("  %s  %d bytes\n", obj.Key, obj.Size)
}

Additional list options:

Option	Description
`driver.WithPrefix(p)`	Filter objects by key prefix
`driver.WithDelimiter(d)`	Group keys by delimiter (e.g. `"/"`)
`driver.WithMaxKeys(n)`	Maximum objects per page (default: 1000)
`driver.WithCursor(c)`	Pagination cursor from a previous call

Delete an Object08

err := t.Delete(ctx, "uploads", "photos/cat.jpg")

Check Metadata09

Head returns object metadata without downloading the content. This is useful for checking existence, size, or content type before fetching.

info, err := t.Head(ctx, "uploads", "photos/cat.jpg")
if err != nil {
    log.Fatal(err)
}

fmt.Printf("key:          %s\n", info.Key)
fmt.Printf("size:         %d\n", info.Size)
fmt.Printf("content-type: %s\n", info.ContentType)
fmt.Printf("etag:         %s\n", info.ETag)
fmt.Printf("modified:     %s\n", info.LastModified)
fmt.Printf("metadata:     %v\n", info.Metadata)

The ObjectInfo struct contains:

Field	Type	Description
`Key`	`string`	Object key
`Size`	`int64`	Content length in bytes
`ContentType`	`string`	MIME type
`ETag`	`string`	Entity tag for conditional requests
`LastModified`	`time.Time`	Last modification timestamp
`Metadata`	`map[string]string`	User-defined metadata
`VersionID`	`string`	Version identifier (if versioning enabled)
`StorageClass`	`string`	Storage tier (e.g. `"STANDARD"`, `"GLACIER"`)

Copy an Object10

Copy duplicates an object within the same bucket or across buckets. The source is not modified.

info, err := t.Copy(ctx,
    "uploads", "photos/cat.jpg",       // source
    "archive", "2024/photos/cat.jpg",  // destination
)

Multi-Backend Routing11

Trove can route objects to different storage backends based on bucket, key pattern, or custom logic.

import (
    "github.com/xraph/trove"
    "github.com/xraph/trove/drivers/localdriver"
    "github.com/xraph/trove/drivers/s3driver"
)

primary := localdriver.New()
primary.Open(ctx, "file:///var/data/hot")

archive := s3driver.New()
archive.Open(ctx, "s3://my-archive-bucket?region=us-east-1")

t, err := trove.Open(primary,
    // Register a named backend.
    trove.WithBackend("archive", archive),

    // Route by key pattern (filepath.Match syntax).
    trove.WithRoute("*.log", "archive"),
    trove.WithRoute("backups/*", "archive"),

    // Route by custom function.
    trove.WithRouteFunc(func(bucket, key string) string {
        if bucket == "compliance" {
            return "archive"
        }
        return "" // fall through to default
    }),
)

Route resolution priority:

Route functions (highest)
Pattern routes
Default driver (fallback)

Add Middleware12

Middleware intercepts the data stream on read and/or write. Register middleware for encryption, compression, or custom processing.

import (
    "github.com/xraph/trove/middleware"
    "github.com/xraph/trove/middleware/encrypt"
    "github.com/xraph/trove/middleware/compress"
)

// Encrypt all objects globally.
t.UseMiddleware(middleware.Registration{
    Middleware: encrypt.New(encrypt.WithKeyProvider(
        encrypt.NewStaticKeyProvider(myAES256Key),
    )),
})

// Compress only text files in the "logs" bucket.
t.UseMiddleware(middleware.Registration{
    Middleware: compress.New(),
    Scope:     middleware.And(
        middleware.ForBuckets("logs"),
        middleware.ForKeys("*.txt", "*.log", "*.json"),
    ),
    Priority:  10,
})

Next Steps13

Architecture -- Understand the storage pipeline, routing, and middleware architecture
Drivers -- Full reference for all storage backend drivers
Middleware -- Encryption, compression, deduplication, scanning, and custom middleware
Streaming -- Chunked transfers, backpressure, resumable uploads, and progress hooks
Content-Addressable Storage -- Hash-based deduplication and garbage collection
Virtual Filesystem -- io/fs-compatible filesystem abstraction over object storage

← Previous

Trove: Multi-Backend Object Storage for Go

Architecture