The API server should not become an expensive pipe#
Naive:
Client → API server → Object storageProxying a 20 GB upload consumes API bandwidth, ties up a long connection, and can create memory, disk, and timeout pressure. The application still authorizes and tracks the upload without necessarily carrying the bytes.
Retry the failed part, not the entire file#
Split the file into numbered parts, upload independently, record integrity information, and finalize only when required parts are present. Resuming needs an upload identifier and part inventory; completion should be safe to retry.
Upload-state model
pending → uploading → uploaded → processing → ready
↘ failed / expiredTrack expected object key, owner, size/type limits, completed parts, checksum where appropriate, timestamps, and processing status. Abort or expire abandoned multipart uploads so unfinished parts do not accumulate forever.
- Trusting a client-supplied filename as the storage key.
- Returning an unrestricted or excessively long-lived upload credential.
- Marking metadata ready before upload integrity and completion are established.
- Making “complete upload” create duplicate processing jobs on retry.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- How does a 20 GB upload resume?
- How are part checksums verified?
- Who cleans abandoned uploads?
- How is duplicate completion handled?
- How is processing retried?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Multipart upload overviewAWS Documentation
- Upload with presigned URLsAWS Documentation