Self-Hosted Storage for High-Throughput App File Uploads — Retention Deadlines

Short answer: For a B2B SaaS app comparing MinIO self-hosted with DigitalOcean Spaces for file uploads, decide large-document throughput and deletion evidence as separate contracts; the right boundary depends on whether the team operates storage infrastructure or a managed endpoint.

Large signed documents change the storage decision in an awkward way: the fastest upload path is not the same thing as a trustworthy deletion policy. For an app retaining signed documents until an explicit deadline, I would model those as two contracts. The upload contract covers multipart transfer, retries, checksums, and concurrency. The retention contract covers timestamps, legal holds, lifecycle evaluation, and evidence that deletion happened.

The practical answer is to keep the application on an S3-compatible object interface, push large files through multipart upload, and make the deletion deadline application data as well as object metadata. That preserves a useful comparison between MinIO, DigitalOcean Spaces, and Amazon S3 without pretending that API compatibility removes operational differences.

What does the large-file path actually require?

A signed PDF is still an object, but a scanned agreement can be hundreds of megabytes or larger. Sending it through one request makes a transient network failure expensive: the retry repeats work that already succeeded. Multipart upload changes the unit of failure. The application can upload parts concurrently, retry one failed part, then complete the object after the required parts are present.

The S3 multipart contract has concrete limits: a part must be at least 5 MiB except for the final part, an upload may contain up to 10,000 parts, and a completed object may be up to 5 TiB. Those numbers are design inputs, not trivia. At 64 MiB per part, a 5 TiB object would need 80,000 parts, so that part size cannot support the maximum object size. Most document systems will never approach 5 TiB, but the arithmetic is a useful check against copying a sample configuration blindly.

A minimal upload record should outlive the request:

from dataclasses import dataclass
from datetime import datetime

@dataclass
class DocumentUpload:
    document_id: str
    object_key: str
    content_length: int
    checksum: str
    retention_deadline: datetime
    upload_id: str | None = None
    state: str = "pending"

The object key is not the deletion policy. Store the deadline in a database row that can be queried, audited, and joined to the tenant and document record. Store the checksum and expected length before accepting the document as complete.

Which failure modes should decide the architecture?

The first trap is an abandoned multipart upload. A client can disappear after uploading 37 parts, leaving storage occupied even though no readable document exists. Lifecycle rules for incomplete multipart uploads are therefore part of the baseline configuration, alongside rules for expired objects.

The second trap is a clock disagreement. “Delete 30 days after signature” is not the same as “delete 30 days after upload,” and neither is safely implemented by adding 30 days in a browser. Write an explicit UTC deadline at the business event, then have a server-side reconciler select records whose deadline has passed. Object lifecycle expiry can be a second control, but it should not be the only record of what the application intended.

The third trap is assuming that a successful delete request proves the whole workflow is complete. The worker needs an idempotent state transition, a retry policy, and an audit event containing the object key, deadline, deletion attempt time, and result. A missing object during a retry should be treated as a converged state only after the application can establish that the document record is no longer supposed to point to it.

I would test these cases before comparing monthly bills: a part upload that times out, completion after a duplicate retry, a worker crash after deleting the object but before updating the database, and a deadline changed by an authorized retention action. Those tests expose the architecture more clearly than a feature checklist.

Should a self-hosted app choose managed storage for file uploads?

S3-compatible does not mean operationally identical. The useful comparison is the boundary each system leaves with the application team.

That table is deliberately unexciting. The trade-off is explicit: throughput depends on client concurrency, part size, network path, disk layout, and request limits; a product label is not a benchmark. A self-hosted cluster can have excellent local bandwidth and poor tail latency during a disk rebuild. A managed service can remove cluster maintenance while leaving the application exposed to slow client uploads and uncollected multipart state. Measure the path your users take.

The interface only narrows the application change.

What should the rollout prove?

Start with one document class and a bounded object-size distribution. Record upload duration, retry count, completed multipart uploads, abandoned uploads, checksum failures, delete lag, and bytes retained after the deadline. Alert on the last two; a fast upload that never disappears is a retention failure.

That is the part teams tend to discover late.

Keep the storage adapter narrow: initiate multipart upload, sign or proxy part transfers, complete, abort, inspect metadata, and delete. Do not spread provider-specific calls through request handlers. Run the reconciler from the database deadline, and use storage lifecycle rules as defense in depth with a documented relationship to that deadline.

During migration, dual-write metadata first, then copy a sample of documents with checksums and lengths, and compare read-after-copy results before changing the read path. Abort the migration when verification differs. The decision is complete only when throughput, failure recovery, and deletion evidence all pass the same test suite.

The durable design is a contract split: multipart upload handles large-file throughput; an explicit, audited deadline handles retention. The choice between a self-operated service and a managed endpoint changes who carries the operational work, not who is accountable for the document lifecycle.

Sources

• https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html
• https://docs.min.io/community/minio-object-store/administration/object-management/object-lifecycle-management.html
• https://docs.digitalocean.com/products/spaces/how-to/