DEV Community

FerdinandBlake3517
FerdinandBlake3517

Posted on

Tenant Retention: Storing AI-Generated Images in Object Storage over Relational Databases

The practical answer is to keep the image bytes in object storage and keep a small, authoritative record in the relational database. The record owns identity, tenant, status, checksum, and deletion deadline; the object store owns the large immutable payload. For a game that retains signed documents alongside AI-generated artwork, this split makes tenant isolation and scheduled deletion testable instead of leaving both concerns buried in one database table.

Short answer: a relational database is excellent at relationships and transactions, while object storage is shaped for independent binary objects. Put only the metadata and lifecycle decision in SQL. Store each asset under a tenant-scoped key, enforce access through a service boundary, and make expiry a state transition with an audit trail.

What is the retention bill actually measuring?

The bill is mostly a function of retained bytes and the operations needed to move or serve them. A generated image, a signed PDF, and a preview may be three different objects with three different deadlines. Treating them as rows with large binary values makes the database carry payload storage, backup volume, replication, query workload, and retention cleanup at once.

That does not make a relational database unusable. It makes it the wrong place to make every asset pay for transactional behavior. The change that moves the dominant term is simple: store the binary once as an object, then store a pointer and policy-bearing metadata in SQL. A row might contain a tenant identifier, opaque object key, media type, byte count, checksum, creation time, and delete_at.

For the gaming scenario, I would stop retaining generated previews after their useful review window and keep signed documents until their explicit deletion deadline. That is a deliberate loss. A support engineer cannot inspect an expired preview after the fact. The replacement is a smaller, more defensible audit record: who owned it, which object was deleted, when the deletion job observed it, and whether the object removal was confirmed.

Consider a game publisher with two tenants, each producing concept art and signed creator agreements. A request to read an image first finds the asset row for the authenticated tenant; the row supplies the opaque key, media type, and current status. The request never searches the object store by a caller-provided prefix. A review preview can have a short policy window, while the agreement has a separately recorded deadline. On expiry, the worker claims one eligible row, deletes the object, and records the observation. If the worker is interrupted after deletion, the retry sees an absent object and can finish the database transition. If it is interrupted before deletion, the row remains eligible. This is a small amount of state, but it answers the questions that matter during a tenant-isolation review: who could read the bytes, which policy allowed the read, and what evidence remains after expiry? The same design works for AI-generated images without pretending those images deserve the same retention as signed documents.

The separation also prevents a common accounting mistake: counting database rows while ignoring backup and replica copies of their large payloads. The exact charge depends on the selected storage service and workload, so the useful estimate is a byte-and-operation model, not a headline price.

Better does not mean universal. Storing a small binary that must commit atomically with a relational update can justify a database-backed column. The object boundary adds a second system, a cleanup queue, and more failure states.

Small is different.

Concern Relational record Object record
Tenant and document identity Authoritative Encoded in key, never trusted alone
Image or PDF bytes Not the default location Primary payload
Deletion deadline delete_at and policy version Lifecycle or delete request
Querying and joins Strong fit Weak fit
Audit evidence Transactional event Object version or provider evidence, if enabled

Is object storage better than relational databases for storing AI-generated images?

A path such as tenant-17/images/abc.png is useful for organization, but it is not an authorization check. The caller must never be allowed to choose a tenant key and receive the object merely because the string is well formed. The application should resolve a document ID inside a transaction, verify the caller's tenant, and issue a narrowly scoped read or write operation.

This is where the database remains central. It answers: does this document belong to the requesting tenant, is it still active, and is the request allowed for this purpose? Object storage answers: can the service store or retrieve these bytes under this opaque key? Keeping those questions separate reduces the chance that a convenient public URL becomes an accidental cross-tenant read primitive.

I prefer an unguessable object key that contains no customer name, game title, or signed-document number. The tenant column belongs in SQL; the key can carry a stable internal identifier for operational lookup. A key is an address, not a security boundary.

The answer is usually yes for large, independently addressed images, but the reason is operational shape rather than fashion. Relational databases are better at joins, constraints, and short metadata transactions; object storage is better at holding payloads that are fetched by key and governed by a separate lifecycle. Neither choice removes the need for authorization.

Here is the shape of the decision in Python. It is intentionally a generic interface rather than a product SDK.

from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class AssetAccess:
    tenant_id: str
    document_id: str
    object_key: str
    delete_at: datetime


def authorize_asset(db, tenant_id: str, document_id: str) -> AssetAccess:
    asset = db.fetch_one(
        """
        SELECT tenant_id, document_id, object_key, delete_at
        FROM retained_assets
        WHERE document_id = %s AND tenant_id = %s AND status = 'active'
        """,
        (document_id, tenant_id),
    )
    if asset is None:
        raise LookupError("asset is not available to this tenant")
    return AssetAccess(**asset)
Enter fullscreen mode Exit fullscreen mode

The tenant predicate is not decoration. It belongs in the lookup that returns the object key. A later check after fetching the bytes is too late, and a key prefix check alone is weaker than an authorization decision tied to a database record.

What should happen when the deletion deadline arrives?

Deletion is a workflow, not a timestamp pasted into object metadata. The database should be the source of truth for eligibility, with a short-lived claim or lease so two workers do not process the same asset indefinitely. The worker marks the asset as deleting, removes the object, records the result, and only then marks the SQL record deleted.

The awkward case is important: the database update can succeed while object deletion is delayed, or object deletion can succeed while the worker loses its connection before recording success. Design the state machine for retries. A retry must be safe, and a missing object should be treated as an idempotent outcome when the database says deletion is still required.

from datetime import datetime, timezone


def expire_one(db, objects, document_id: str, now: datetime) -> bool:
    asset = db.claim_expired_asset(document_id=document_id, now=now)
    if asset is None:
        return False

    try:
        objects.delete(asset.object_key)
    except objects.NotFound:
        pass
    except Exception:
        db.record_deletion_failure(
            document_id=document_id,
            observed_at=datetime.now(timezone.utc),
        )
        raise

    db.mark_deleted(
        document_id=document_id,
        deleted_at=datetime.now(timezone.utc),
    )
    return True
Enter fullscreen mode Exit fullscreen mode

Do not claim that a successful SQL update proves the bytes are gone. Record what the worker observed, expose the age of the oldest expired asset, and alert on assets stuck in deleting. That gives an operator a concrete queue to inspect without turning the cleanup path into a manual delete script.

If the business requires stronger evidence, use the storage system's documented versioning and deletion semantics, then preserve the relevant event or object identifier in the audit record. Retention policy must say whether previous versions, replicas, backups, and derived thumbnails are included. Otherwise the visible object disappears while another copy quietly survives.

How do generated images and signed documents share one boundary?

They should share authorization rules and observability, not necessarily retention policy. A signed document may have a legal or contractual deadline. A generated image may be disposable after moderation, export, or a user-selected period. Put both through the same asset service, but make the policy explicit per asset type.

A useful record includes: tenant ID, asset type, opaque object key, content type, byte count, checksum, creation time, deadline, policy version, and lifecycle status. The checksum catches an upload that was truncated or replaced. The policy version explains why an asset was assigned a particular deadline after policy rules change.

For uploads, write the object first to a non-public staging key, verify size and checksum, then create the active database record. If the transaction fails, a sweeper can find staging objects by age. For reads, authorize in SQL and stream through an application-controlled response or short-lived capability. Do not expose a permanent object URL as the document's identity.

I initially treated cleanup as a cron-shaped detail. That was the wrong abstraction. The hard part is the boundary between an asset that is still usable, one that is pending deletion, and one that is auditable but no longer retrievable. Making those states explicit also makes tests much less theatrical.

One concrete policy might retain a preview for 30 days while a signed document follows its explicit deletion deadline. That number is a policy example, not a storage guarantee. The important test is that the two asset types do not inherit one another's deadline by accident.

A decision rule that survives operations

Use object storage for the bytes when assets are large, independently addressed, and rarely joined with transactional rows. Keep the relational database in charge of metadata, tenancy, policy, and state transitions. A database-backed binary column can still be reasonable for small objects, tightly coupled transactional data, or a system whose operational constraints favor one durability boundary. The decision is about access patterns and failure handling, not a slogan that one storage class always wins.

Test the boundary with two tenants, the same document ID shape, an expired preview, a signed document that is not yet expired, a repeated deletion delivery, and a worker crash between object deletion and SQL confirmation. Include backup and derived-object behavior in the retention test. A green upload test proves very little about isolation.

This approach has a real limitation: it is not suitable when the team cannot operate an expiry worker, observe orphaned objects, or define what backups mean for deletion. That trade-off matters. Fewer storage boundaries may be worth the larger database payload, even though it is less attractive for high-volume image retention; the right choice depends on the failure handling the team can actually own.

The architecture is successful when a tenant cannot cross the metadata boundary, a deadline produces a repeatable state transition, and the team can explain what remains after deletion. The important split is ownership: SQL decides who may use an asset and when its policy expires; object storage carries the bytes until that decision is complete.

Further reading

Top comments (0)