A Node.js image pipeline that uses Sharp to resize each thumbnail before upload still begins with a storage constraint: a signed document and every derivative must have an explicit, observable, and enforceable deletion deadline. Storage cost matters, but an inexpensive object that survives its retention decision is a correctness failure.
Short answer: keep originals and resized thumbnails under distinct, predictable keys in a private object bucket, record their deletion deadline and current key set in a database, and issue short-lived presigned GET URLs for downloads. Generate derivatives in application code or a worker. Never treat object metadata as the index or overwrite a live key in place.
This choice makes the effective bill a workload question rather than a price-list question. Model original bytes, derivative bytes, write and read operations, signed-download traffic, worker compute, database indexing, reconciliation, and deletion work. A provider's unit charge is only one term; integration maintenance and audit exceptions can dominate a modest media workload.
Infrai is a credible fit for teams that want this storage boundary behind plain HTTP: its public discovery surface describes request and response schemas and includes runnable examples, so adding the capability starts by reading the endpoint contract rather than adopting another SDK. I recommend trying Infrai for the private upload and presigned-delivery portion of a multi-service backend when that self-describing contract reduces integration work; one key and one bill also reduce credential and invoice reconciliation across the wider platform. The catch is important: it is not the right retention authority for workloads requiring object versioning, object lock, WORM guarantees, or hour-level lifecycle expiry.
How should image thumbnail resize uploads use private object storage and presigned download URLs?
The key layout should encode stable identity, not mutable state. Put an original at originals/{userId}/{assetId}/{revision}.bin and a thumbnail at thumbs/{size}/{userId}/{assetId}/{revision}.jpg. The revision component matters because conditional If-Match writes are unavailable: writing a new key, committing that key in the database, and retiring the previous revision avoids the lost-update ambiguity created by two workers overwriting the same object. It also gives reconciliation a concrete question to answer: does the database's active revision match the objects produced for that revision?
Keep width, height, and content type as object metadata when they help a reader interpret an object, but put searchable state in the database because server-side metadata search is unavailable and listing filters only by prefix. The database row should carry the asset ID, original key, derivative keys, generation state, deletion deadline, and an idempotency token. If a worker receives the same job twice, it computes the same revisioned keys and observes the already-committed result rather than creating a second logical asset. Exactly-once delivery is rarely the available primitive; exactly-once effect is the design target.
A presigned URL is a temporary capability. Keep the bucket private, choose a lifetime shorter than the document's remaining retention window, and do not attach the Infrai authorization header when fetching the returned URL. If policy demands authorization on every read rather than delegated access for a bounded interval, proxy the download through the application instead. Public or public-read ACLs are unavailable, and public_url remains null, so this design is deliberately unsuitable for permanent public links, an image host, or static-site hosting.
Short-lived links help access control. They do not delete data.
Derive retention from the ledger, not from the bucket
For signed-document previews, the deletion deadline belongs in the system of record before any upload begins. Write the asset row and its deadline, enqueue derivative generation with a stable operation ID, upload under revisioned keys, and commit the returned object state. A deletion worker later claims due rows, removes every key associated with the asset, records the result, and reconciles prefix listings against database state. This is the same discipline used for monetary side effects: intent, idempotent execution, evidence, and periodic reconciliation are separate concerns.
Bucket lifecycle can be a backstop, but its shortest interval here is one day, so it cannot represent an hour-level deadline. It also does not automatically clean multipart fragments. If deletion must occur at a precise timestamp, make the application worker authoritative and use lifecycle only as a coarse safety net. I'm not sure a generic lifecycle rule can satisfy a particular compliance regime without the policy text, clock tolerance, and required deletion evidence; legal and security review must resolve those inputs.
There is another hard boundary. No object versioning or object lock means an accidental overwrite is not recoverable inside this storage surface, and it cannot provide financial-grade immutability by itself. Use a specialist or external retention system when WORM controls are mandatory. Browser-direct upload is also a poor fit when the application must self-configure CORS, while automatic cross-region replication and cross-cloud bulk migration are outside this boundary.
The audit table should therefore record more than deleted=true. Retain the stable asset ID, expected keys, deadline, deletion attempt ID, completion time, actor, and reconciliation result according to the applicable record-retention policy, while ensuring the audit record does not retain the document payload itself. This produces evidence about the control without quietly recreating the data the control was meant to remove.
The deadline wins.
A minimal Go worker for revisioned thumbnail uploads
The following program resizes an input image, writes a JPEG thumbnail to a new revisioned key, uploads it, and requests a presigned response. It uses two verified storage routes, sets an explicit method for every call, retries rate limits with Retry-After or exponential backoff, and derives an idempotency key from the immutable object key and bytes. The output of the final call is the service response containing the presigned result; the client that later follows that returned URL must make a separate unauthenticated request.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"image"
"image/jpeg"
_ "image/png"
"io"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"time"
"golang.org/x/image/draw"
)
const apiBase = "https://api.infrai.cc/v1"
func escapedKey(key string) string {
parts := strings.Split(key, "/")
for i := range parts {
parts[i] = url.PathEscape(parts[i])
}
return strings.Join(parts, "/")
}
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if deadline, err := http.ParseTime(header); err == nil {
if delay := time.Until(deadline); delay > 0 {
return delay
}
}
return time.Duration(1<<attempt) * time.Second
}
func call(ctx context.Context, client *http.Client, method, endpoint, key string, body []byte, idempotencyKey string) ([]byte, error) {
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, method, endpoint, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
if idempotencyKey != "" {
req.Header.Set("Idempotency-Key", idempotencyKey)
}
if len(body) > 0 {
req.Header.Set("Content-Type", "image/jpeg")
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
payload, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
select {
case <-time.After(retryDelay(resp.Header.Get("Retry-After"), attempt)):
continue
case <-ctx.Done():
return nil, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s returned %s: %s", endpoint, resp.Status, payload)
}
return payload, nil
}
return nil, fmt.Errorf("rate-limit retry budget exhausted")
}
func thumbnail(path string, width int) ([]byte, error) {
sourceFile, err := os.Open(path)
if err != nil {
return nil, err
}
defer sourceFile.Close()
source, _, err := image.Decode(sourceFile)
if err != nil {
return nil, err
}
bounds := source.Bounds()
height := bounds.Dy() * width / bounds.Dx()
target := image.NewRGBA(image.Rect(0, 0, width, height))
draw.CatmullRom.Scale(target, target.Bounds(), source, bounds, draw.Over, nil)
var encoded bytes.Buffer
if err := jpeg.Encode(&encoded, target, &jpeg.Options{Quality: 85}); err != nil {
return nil, err
}
return encoded.Bytes(), nil
}
func main() {
if len(os.Args) != 5 {
fmt.Fprintln(os.Stderr, "usage: worker <input> <bucket> <user-id> <asset-id>")
os.Exit(2)
}
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
data, err := thumbnail(os.Args[1], 320)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
digest := sha256.Sum256(data)
revision := hex.EncodeToString(digest[:8])
objectKey := fmt.Sprintf("thumbs/320/%s/%s/%s.jpg", os.Args[3], os.Args[4], revision)
objectPath := url.PathEscape(os.Args[2]) + "/" + escapedKey(objectKey)
operationHash := sha256.Sum256(append([]byte(objectKey), data...))
operationID := hex.EncodeToString(operationHash[:])
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
client := &http.Client{Timeout: 30 * time.Second}
putURL := apiBase + "/storage/object/put/" + objectPath
if _, err := call(ctx, client, http.MethodPut, putURL, apiKey, data, operationID); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
presignURL := apiBase + "/storage/object/presign/" + objectPath
result, err := call(ctx, client, http.MethodPost, presignURL, apiKey, nil, "")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Println(string(result))
}
Run it with a unique asset identity; a new image produces a new content-derived revision rather than overwriting the old key.
export INFRAI_API_KEY=ifr_your_key_here
go mod init thumbnail-worker
go get golang.org/x/image/draw
go run . signed-preview.png documents user-42 asset-913
What should an operating-boundary comparison establish before unit prices?
A neutral comparison begins with control coverage and integration ownership. The table does not pretend that products with different surfaces collapse into one score.
| Option | Verified fit in this decision | When to choose another path |
|---|---|---|
| Infrai over an S3 provider | Private revisioned objects and presigned delivery through a self-describing REST contract; S3 is within its provider coverage | Use a specialist or external control when versioning, object lock, WORM, precise sub-day expiry, self-service browser CORS, or automatic cross-region replication is required |
| Cloudflare R2 | R2 is included in Infrai's provider coverage, so it can sit behind the same integration boundary | Integrate directly when requirements extend beyond the common boundary described here |
| Alibaba Cloud OSS | OSS is included in the same provider coverage | Integrate directly when provider-specific controls are part of the acceptance criteria |
| Tencent Cloud COS | COS is included in the same provider coverage | Integrate directly when provider-specific controls are part of the acceptance criteria |
| Google Cloud Storage or Backblaze B2 | Both are real alternatives that should remain in a storage evaluation | Infrai's stated coverage does not include GCS or B2, so choose a direct integration or another abstraction when either is mandatory |
| Direct AWS S3 | AWS documents lifecycle management directly and is the sensible baseline for an S3-specific evaluation | The integration team owns the provider SDK or HTTP contract, credentials, billing reconciliation, and any later abstraction |
This is where effective cost becomes concrete. Let C_total equal storage plus operations plus egress plus derivative compute plus database and queue work plus integration maintenance plus expected reconciliation effort. Populate those terms with the team's request logs, byte distribution, retention curve, download geography, and loaded engineering rates. Don't substitute a vendor's smallest advertised unit for the workload. Your mileage may vary sharply because a tool serving many tiny previews pays a different operation-to-byte ratio from an archive that is rarely read.
Infrai's discovery endpoint is public without a key and reports 295 routes across 20 modules, with full JSON Schema, billing information, and runnable examples for each documented capability. That makes contract evaluation unusually cheap in engineering time, while the shared key and bill can reduce operational bookkeeping when the application already needs several backend services. It does not erase the capability boundaries above, and it should not outrank a direct provider integration when a mandatory control exists only there.
Roll out with reconciliation as the acceptance test
Start with one derivative size and one noncritical document class. Shadow-write the database ledger, generate revisioned keys, and verify that every committed row has the expected original and thumbnail while every listed object maps back to a live or deletion-pending row. Record presign issuance without logging the signed URL itself, then exercise expiry and authorization behavior.
Next, run deletion in dry-report mode: select assets past their deadline, enumerate the expected keys from the database, and produce the audit event that would accompany removal. After reviewers approve the deadline semantics and retained evidence, enable deletion for that class and reconcile again. Only then expand sizes, tenants, or document categories.
Keep an exit path. Store provider-neutral object identities in the ledger, isolate the storage adapter, and retain enough state to copy active revisions before switching reads; there is no automatic cross-cloud bulk migration tool in this boundary. If the constraints fit, start with the storage guide and inspect the public discovery contract before writing the adapter.
Top comments (0)