GCS Driver

The GCS driver stores objects in Google Cloud Storage. It implements the core driver.Driver interface plus MultipartDriver, PresignDriver, and RangeDriver capability interfaces.

Installation01

The GCS driver has its own Go module to isolate the Google Cloud SDK dependency:

go get github.com/xraph/trove/drivers/gcsdriver

Usage02

import (
    "context"
    "github.com/xraph/trove"
    "github.com/xraph/trove/drivers/gcsdriver"
)

ctx := context.Background()

// Create and open the driver.
drv := gcsdriver.New()
err := drv.Open(ctx, "gcs://my-project/my-bucket")
if err != nil {
    log.Fatal(err)
}

// Use with Trove.
t, err := trove.Open(drv)

Custom Endpoint (Emulator)

drv := gcsdriver.New()
err := drv.Open(ctx, "gcs://my-project/my-bucket?endpoint=http://localhost:4443")

With Credentials File

drv := gcsdriver.New()
err := drv.Open(ctx, "gcs://my-project/my-bucket?credentials=/path/to/service-account.json")

DSN Format03

gcs://PROJECT_ID/BUCKET
gcs://PROJECT_ID/BUCKET?credentials=/path/to/key.json&endpoint=http://localhost:4443

Component	Description
`PROJECT_ID`	Google Cloud project ID
`BUCKET`	Default bucket name
`credentials`	Path to service account JSON key file. Omit to use Application Default Credentials
`endpoint`	Override GCS endpoint URL (for emulators like `fake-gcs-server`)

Driver options can also override DSN values:

drv.Open(ctx, "gcs://project/bucket",
    driver.WithEndpoint("http://localhost:4443"),
)

Capabilities04

The GCS driver implements these capability interfaces beyond the core driver.Driver:

MultipartDriver

Upload large objects using GCS compose operations:

if mp, ok := t.Driver().(driver.MultipartDriver); ok {
    uploadID, _ := mp.InitiateMultipart(ctx, "bucket", "large-file.zip",
        driver.WithContentType("application/zip"),
    )

    part1, _ := mp.UploadPart(ctx, "bucket", "large-file.zip", uploadID, 1, chunk1Reader)
    part2, _ := mp.UploadPart(ctx, "bucket", "large-file.zip", uploadID, 2, chunk2Reader)

    info, _ := mp.CompleteMultipart(ctx, "bucket", "large-file.zip", uploadID,
        []driver.PartInfo{*part1, *part2},
    )
}

GCS multipart uploads work by uploading parts as temporary objects and composing them into the final object.

PresignDriver

Generate signed URLs for direct client uploads/downloads:

if ps, ok := t.Driver().(driver.PresignDriver); ok {
    downloadURL, _ := ps.PresignGet(ctx, "bucket", "file.pdf", 15*time.Minute)
    uploadURL, _ := ps.PresignPut(ctx, "bucket", "upload.zip", time.Hour)
}

RangeDriver

Read specific byte ranges:

if rd, ok := t.Driver().(driver.RangeDriver); ok {
    reader, _ := rd.GetRange(ctx, "bucket", "video.mp4", 1000, 1000)
    defer reader.Close()
}

API05

Constructor

func New() *GCSDriver

Creates a new GCS driver instance.

Client

func (d *GCSDriver) Client() *storage.Client

Returns the underlying *storage.Client for advanced GCS operations.

Unwrap

func Unwrap(accessor interface{ Driver() driver.Driver }) *GCSDriver

Extract the *GCSDriver from a Trove handle:

gcsdrv := gcsdriver.Unwrap(troveInstance)
if gcsdrv != nil {
    raw := gcsdrv.Client()
}

Driver Registration

The GCS driver auto-registers via init():

factory, ok := driver.Lookup("gcs")
drv := factory()

Emulator Setup for Development06

Run fake-gcs-server locally with Docker:

docker run -d --name fake-gcs \
  -p 4443:4443 \
  fsouza/fake-gcs-server -scheme http

Then connect:

drv := gcsdriver.New()
drv.Open(ctx, "gcs://test-project/test-bucket?endpoint=http://localhost:4443")

Integration Tests07

# Start fake-gcs-server
docker run -d -p 4443:4443 fsouza/fake-gcs-server -scheme http

# Run integration tests
cd drivers/gcsdriver
go test -tags integration -v ./...

Environment variables:

Variable	Default	Description
`GCS_ENDPOINT`	`http://localhost:4443/storage/v1/`	GCS endpoint
`GCS_PROJECT_ID`	`test-project`	Google Cloud project ID

Limitations08

Multipart uploads use GCS compose, which is limited to 32 components per compose call
Signed URLs require appropriate service account credentials
When using a custom endpoint, authentication is disabled automatically

← Previous

S3 Driver

Azure Driver