Store bytes by key; keep application relationships elsewhere#
An object store maps a key to object bytes plus service metadata inside a bucket/container. It fits images, video, backups, logs, datasets, and downloadable files. A database usually owns application metadata such as owner, authorization, processing state, and relationships.
Move large bytes directly to object storage#
Presigned access lets the API authorize a bounded upload without proxying every byte. The application must still validate identity, key scope, content policy, final object state, and abandoned uploads. Review large-file uploads.
Multipart upload makes large transfers recoverable#
Split a large object into numbered parts, upload parts independently - potentially in parallel - and complete the upload from those parts. Failed parts can be retried without retransmitting the entire object. Track upload identity, integrity, completion, and cleanup rather than treating multipart as automatic application resumability.
Consistency, durability, and availability answer different questions#
Current Amazon S3 provides strong read-after-write consistency for successful object PUT and DELETE operations, including subsequent GET and LIST behavior. That corrects old interview folklore. Strong consistency does not mean every surrounding workflow is atomic: the metadata database, CDN, event consumer, or authorization cache may still lag or fail.
CDN delivery and private access#
Object storage → CDN edge → users
Private object:
authorize user → issue bounded signed access → fetch bytesA CDN lowers repeated origin work and geographic latency. Private media needs an authorization boundary, expiration, and cache-key policy; an unguessable object key is not sufficient authorization.
When to choose it - and when not to#
| Good fit | Poor fit |
|---|---|
| Large immutable or versioned bytes, archives, media, logs | Relational joins, row constraints, low-latency record mutation |
| Direct HTTP transfer and lifecycle retention | Queue semantics or arbitrary record queries |
| Origin for CDN and asynchronous processing | Assuming object upload and metadata commit form one transaction |
- Repeating the obsolete claim that all S3 reads are eventually consistent.
- Putting large media bytes in the primary relational row by default.
- Proxying all uploads through application servers without a requirement.
- Trusting object keys as authorization.
- Assuming one event notification or an atomic object-plus-database workflow.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- Who authorizes upload and download?
- Where is object metadata and processing status stored?
- How are retries, checksums, and abandoned parts handled?
- What if the event is delivered twice?
- How does CDN invalidation affect freshness?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Amazon S3 and its data consistency modelAWS Documentation
- Multipart upload overviewAWS Documentation
- Amazon S3 Event NotificationsAWS Documentation