Formats and running it
S3 and Azure Blob storage for the cache
Settings, Storage keeps the cache in a bucket instead of on the box. Requests are signed on the box with the bucket's keys. No cloud SDK is installed.
Fill in the endpoint, region, bucket, an optional folder and the keys, then Save and test the bucket: it puts a small file in, reads it back and deletes it. Switching to the bucket tests it again and changes nothing if that fails. Keys can come from the environment instead, which keeps them out of the database.
New files still land on local disk first, so a download never waits on the bucket, and an uploader sends them up every minute. Every upload carries the file's SHA-256 (or MD5 for Azure, which Azure checks), and the bucket refuses bytes that do not match. Local copies to keep caps the disk, dropping the least recently used copies of files the bucket already holds. A file fetched back from the bucket is checked against its SHA-256 before any of it is served.
If the bucket cannot be reached, installs carry on from local copies and new files wait on disk until it is back.
In short
- AWS S3 and anything S3 compatible: MinIO, R2, Wasabi
- Azure Blob with Shared Key signing
- Hash checked uploads and downloads
- Local copy cap with least recently used eviction
- Keeps working from disk when the bucket does not
In the documentation
- Cache, storage and retention Administrator Guide
- Upgrades, health and high availability Administrator Guide
Goes well with
- More than one node Point two nodes at one MariaDB, RDS included, and put a load balancer in front.
- Artifacts, SBOMs and lifecycle Every file stored once by SHA-256, CycloneDX and SPDX out, properties and promotion stages.
- Lockdown and degraded modes For the week an ecosystem is on fire. Stop fetching anything new, keep building from what you hold.
One container, about two minutes
A Linux box with Docker, or one without it, and a reverse proxy for TLS. The installer does the rest and it is safe to run twice. Free, MIT licensed, nothing to sign up for.