Storage

FastSchema provides a powerful filesystem abstraction thanks to the wonderful rclone package.

The FastSchema rclone integration provides straightforward drivers for various storage systems, making it easy to implement and switch between them.

The rclonefs package offers a filesystem abstraction that allows reading and writing files across different storage systems.

Supported storage systems include:

• Local filesystem
• S3 compatible storage

Disk interface

A Disk is a filesystem that can be used to read and write files to and from a specific storage system.

type Disk interface {
	Name() string
	Root() string
	URL(filepath string) string
	Delete(c context.Context, filepath string) error
	Put(c context.Context, file *File) (*File, error)
	PutReader(c context.Context, in io.Reader, size uint64, mime, dst string) (*File, error)
	PutMultipart(c context.Context, m *multipart.FileHeader, dsts ...string) (*File, error)
	LocalPublicPath() string
}

Disk Configuration

type StorageConfig struct {
	DefaultDisk string        `json:"default_disk"`
	Disks       []*DiskConfig `json:"disks"`
}

type DiskConfig struct {
	Name            string        `json:"name"`
	Driver          string        `json:"driver"`
	Root            string        `json:"root"`
	BaseURL         string        `json:"base_url"`
	PublicPath      string        `json:"public_path"`
	Provider        string        `json:"provider"`
	Endpoint        string        `json:"endpoint"`
	Region          string        `json:"region"`
	Bucket          string        `json:"bucket"`
	AccessKeyID     string        `json:"access_key_id"`
	SecretAccessKey string        `json:"secret_access_key"`
	ACL             string        `json:"acl"`
}

A FastSchema application may have multiple disks, each with a unique name and configuration.

A default disk is used for general file storage, while additional disks can be configured for specific purposes.

Defaukt disk configuration:

{
  "name": "public",
  "driver": "local",
  "root": "./public",
  "public_path": "/files",
  "base_url": "http://localhost:8000/files"
}

There are two methods for configuring disks:

• Using environment variables:
  • STORAGE: A string representing a JSON object with disk configurations.
• Using application configuration:
  • app.Config.StorageConfig: A StorageConfig object with disk configurations.

If there is no configuration for the default disk, the first disk in the configuration will be used as the default disk.

Serving files at the root path

public_path decides the URL prefix a local disk is served under. It defaults to /files, but any prefix is accepted, including /.

API routes are always matched before static files, so a file on disk can never take over an API URL, even when public_path is /. Serving a disk at the root is still a broad choice: every request that does not match an API route is looked up on disk first. FastSchema logs a warning at startup when it sees public_path: "/", and the request log keeps recording those requests either way.

Static mounts match on whole path segments. A disk published at /files serves /files and /files/photo.png, and leaves /filesystem alone.

Usage

Creating a new disk

disks, err = rclonefs.NewFromConfig(
  storageDisksConfig, // []*DiskConfig
  localRoot, // string representing the local root path
)

Using a disk

A storage disk is accessed using the app.Disk() method.

Disk(names ...string) Disk

If no disk name is provided, the default disk will be used.

// Get the default disk
disk := app.Disk()

// Get a specific disk
awsS3Disk := app.Disk("awss3")

// Get disk name
diskName := disk.Name()

// Get disk root
diskRoot := disk.Root()

// Get disk URL
diskURL := disk.URL("path/to/file")

// Put a file
file, err := disk.Put(c, &fs.File{
  Name: "file.txt",
  Path: "path/to/file.txt",
  Type: "text/plain",
  Size: 1024,
  Reader: strings.NewReader("Hello, World!"),
})

Registering an external file by URL

A file record whose path is an absolute http:// or https:// URL points to an externally hosted file. FastSchema stores the URL as-is and serves it verbatim: the disk base URL is never prepended, and no bytes are downloaded. Such a record owns no storage object, so deleting it removes only the database row (the remote file is left untouched).

You can register one through the content API without uploading a file:

curl -X POST http://localhost:8000/api/content/file \
  -H "Content-Type: application/json" \
  -d '{
    "name": "logo.png",
    "path": "https://cdn.example.com/logo.png",
    "type": "image/png",
    "size": 0,
    "disk": ""
  }'

The dashboard exposes the same feature through the "From URL" button in the media library. With the SDK, use file().registerURL():

await fs.file().registerURL({
  url: "https://cdn.example.com/logo.png",
  name: "logo.png", // optional, defaults to the file name in the URL
});