Skip to content

Scenario: file and image upload pipeline

Let storage take the bytes into quarantine, and let a worker that trusts nothing but the bytes promote them.

Predict first

Before you read: predict

ExamHall's candidates upload an ID photo before each exam, and a scan checks every photo. The proctor console shows a candidate's tile, with the photo at photos/<candidate>.jpg if one exists, when she checks in, 2 seconds after her upload. Storage's event for a new object reaches a worker anywhere from seconds to a minute later. The API must carry no photo bytes, and the console must never show a photo that has not passed the scan.

Which design meets both?

Pick one

The 30-second version

Two deep dives decide an upload design. Bytes: the browser sends them straight to storage, into a quarantine prefix nothing serves. Processing: storage's event queues a job; a worker checks the bytes and promotes what passes. Events come at least once, so a job is named by its file's version; a status row tells the UI.

Let storage take the bytes into quarantine, and let a worker that trusts nothing but the bytes promote them.

How it works

Step 1 of 12

Two deep dives. Getting the bytes in without your servers, then processing them after they land. A status the UI reads joins the two.

Analogy: A charity shop's donations back room

Donors leave bags at the back door, and volunteers sort them before anything reaches the shop floor.

In the charity shop In the pipeline
bags left at the back door, never carried through the till bytes straight to storage (step 2)
the back room, where nothing is on sale incoming/, read by nobody (step 3)
a bell rings once for each bag left, however many items it holds one event per object (step 4)
priced stock goes to the floor, never back on the sorting shelf notify on incoming/ only (step 5)
an electrical item is opened and tested, whatever its box says the bytes, not the label (step 6)
only a passed item is priced and put out; a failed one goes to recycling promote or reject (step 8)
each bag ticket tells one drop-off from another, even from the same donor what names one file's version (step 9)

Breaks down when: a volunteer can see that a bag was already sorted, while a duplicate event looks exactly like the first (step 9).

Contrasting case: streaming the file through the API lets it check every byte before storing, at the price of carrying every byte (step 2).

In real terms: storage takes the bytes; the pipeline decides what may be served.

Examples, run for real

4.1 ExamHall EXH-3910: photos on the console before their scan

ExamHall's browsers PUT ID photos straight to photos/, and a worker scanned each when its event arrived. Candidates check in two seconds after uploading, before any event does.

4.1 before: ID photos PUT straight to photos/, scanned when the event arrives · at check-in the console showed 40 of 40 photos unchecked, the flagged one among them
4.1 MR !4210: PUT to incoming/, which the console never reads; the worker copies a clean photo to photos/ · unchecked photos shown 0 · photos shown after the scan 39 · the flagged one never shown

MR !4210 moves the upload to incoming/, which the console never reads; only the worker writes photos/, after the scan (steps 3, 8).

4.2 Chat: which objects raise events

A chat app's users send three images and one 30-part video.

4.2 chat: 3 images and a 30-part video under incoming/ · queue messages before the video's Complete 3, after 4 · jobs 4 · objects the worker wrote under media/ 4

The video's 30 parts raised nothing; the object it became at Complete raised one message, like each image (step 4).

4.3 What the API would carry

4.3 2,000,000 attachments a day x 3 MB through the API: 12 TB a day (in from browsers, out to storage), 1.11 Gbit/s on average · straight to storage: 0 bytes, 2,000,000 signing calls

Every byte routed through the API is a byte in and a byte out; signing calls are small (step 2).

Try it

Answer first, then check. Nothing is saved.

Question 1 of 6

A chat app sends image attachments without the bytes passing through its API. The recipient's app polls the attachment's status and fetches the image from media/ as soon as it reads ready.

In what order does the app's pipeline run these steps?

Your order

  1. 1empty
  2. 2empty
  3. 3empty
  4. 4empty
  5. 5empty

Tap the lines in the order you expect.

Question 2 of 6

A chat app's API signs each avatar upload for incoming/avatars/u-7.jpg, the same key every time for user u-7. Storage's event for each new object puts a job on a queue, and a worker resizes the photo to media/avatars/u-7-256.jpg; each resize is a paid call. At 10:00 Mia uploads a photo, and its event is delivered twice. At 10:05 she uploads a new photo to the same key.

Which rule resizes each photo once and leaves her profile showing the 10:05 photo?

Pick one

Question 3 of 6

A streaming service's creators upload cover images to incoming/covers/. A worker takes each queued job, reads the object, and writes a 320-pixel copy of it as covers/<name>-320.jpg in the same bucket. Three creators upload one cover each.

Which notification setup runs three resize jobs and leaves three copies?

Pick one

Question 4 of 6

A marketplace's sellers upload product photos to incoming/<seller>/<file name>, keeping the name their phone gave the file. Storage's event for each new object puts a job on a queue, and a worker writes a 640-pixel copy of the photo under media/; it skips no job, since a copy is cheap. Each job writes the copy's key into the upload's status row, and the product page shows the image at that key. Ana uploads IMG_0001.jpg and IMG_0002.jpg for two products, and the first one's event is delivered twice. Ben uploads his own IMG_0001.jpg for a third product.

Which name for the copy leaves exactly one copy per upload and shows every product page its own photo?

Pick one

Question 5 of 6

A payments app's API signs a PUT for each expense receipt under incoming/receipts/, with Content-Type: image/jpeg among its signed headers, and the app always sends that header. A worker passes each new receipt to an OCR service. The interviewer's next follow-up: "Two uploads arrive: a JPEG photo of a receipt, and a plain-text note the user renamed receipt.jpg.

Which check in your worker sends the photo to OCR and keeps the note out?

Pick one

Question 6 of 6

A ticket resale site's sellers upload a photo of each ticket, and the listing page polls the upload's status. Its worker runs this for each queued job:

A seller's photo passes its checks. Its storage event is delivered twice, and the second copy is taken after the first job has finished.

What does the listing page show afterwards?

Code for this question
async function onJob(job) {  await db.setStatus(job.uploadId, 'processing');  if (await db.seen(job.key, job.etag)) return;  await db.markSeen(job.key, job.etag);  const ok = await resizeAndCheck(job);  await db.setStatus(job.uploadId, ok ? 'ready' : 'rejected');}
Pick one

Go deeper

Pro tips
  1. Put one number on the bytes before you draw the pipeline. When: the first minutes of an upload round, when the interviewer asks why the API should not take the file (step 2). Verify:
    tip 30,000 candidates x 200 MB of proctoring video = 6,000 GB a day; through the API it crosses the network twice: 12,000 GB
    Status: RAN.
Source

quote-check.mjs finds every passage below verbatim in its saved text. Valid as of 2026-10-13.

  • AWS, Uploading objects with presigned URLs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html (aws-presigned-upload.txt). The row's first source. Quoted: "If an object with the same key that is specified in the presigned URL already exists in the bucket, Amazon S3 replaces the existing object with the uploaded object."
  • AWS, Uploading and copying objects using multipart upload: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html (aws-mpu-overview.txt). The row's first source. Quoted: "Upon receiving the complete multipart upload request, Amazon S3 constructs the object from the uploaded parts"; "If you stop the multipart upload, Amazon S3 deletes upload artifacts and all parts that you uploaded." Supports step 4 and t2.
  • OWASP, File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html (owasp-file-upload.txt). The row's second source. Quoted: "Validate the file type, don't trust the Content-Type header as it can be spoofed"; "Set a file size limit"; "Store the files on a different server. If that's not possible, store them outside of the webroot"; "Run the file through an antivirus or a sandbox if available to validate that it doesn't contain malicious data"; "Rewriting does not guarantee that all malicious content is removed, and the image processor itself handles untrusted input."
  • AWS, Amazon S3 Event Notifications: https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html (aws-event-notifications.txt). An S3 docs page the store model's rules quote (the row names "Amazon S3 docs"). Quoted: "Amazon S3 event notifications are designed to be delivered at least once."; "Typically, event notifications are delivered in seconds but can sometimes take a minute or longer."; "If your notification writes to the same bucket that triggers the notification, it could cause an execution loop."; "To avoid this, use two buckets, or configure the trigger to only apply to a prefix used for incoming objects."
  • AWS, Event notification types and destinations: https://docs.aws.amazon.com/AmazonS3/latest/userguide/notification-how-to-event-types-and-destinations.html (aws-event-types.txt). Quoted: "s3:ObjectCreated:CompleteMultipartUpload"; "Prefix and suffix filters apply to object keys only"; "On rare occasions, Amazon S3's retry mechanism might cause duplicate S3 Event Notifications for the same object event."

Steps 1, 10–12 are the composition the scripts run, not quoted rules; 4.3 and the tip are arithmetic.

Related lessons

Checked against the tools and docs as of 2026-10-13.