Lesson 6 of 17Intermediate15 min

Storage

By the end of this lesson

Use object and file storage appropriately.

Object storage is the default place to put files in a cloud system. You give it a key — a string like photos/4821/portrait.jpg — and it stores the bytes under that key. You read them back by key over HTTP. There are no folders, although keys containing slashes are displayed as though there were.

It is not a filesystem, and the differences matter. You cannot open an object and write to the middle of it. You cannot rename cheaply; a rename is a copy and a delete. There are no file locks. What you get instead is durability, effectively unlimited capacity, and a price low enough that keeping every employee photo ever uploaded is not a budget conversation.

The terms, with both providers' names:

Object: One stored item: the bytes plus some metadata such as content type. Azure calls it a blob; AWS calls it an object.
Container or bucket: The named grouping objects live in. Azure Blob Storage has containers inside a storage account; Amazon S3 has buckets. Access settings are applied here, so the grouping is a security boundary as well as an organisational one.
Key: The full name of the object within the container. Design it deliberately — it is the only way you will find the object again, and it is visible to anyone who can read the container listing.
Storage tier: How ready the provider keeps the data. Hotter tiers cost more to store and less to read; colder tiers reverse that.
Presigned URL: A time-limited URL that grants one specific operation on one specific object, signed by your credentials. Azure calls this a shared access signature, or SAS; AWS calls it a presigned URL.
File share: Network storage that behaves like a mounted drive, with directories, partial writes and locks. Azure Files, Amazon EFS or FSx.

Watch out

A file saved inside a container is already gone

A container or function instance has a writable local filesystem, and writing an upload to it appears to work. It works in development, it works in a smoke test, and it fails in production in a way that is hard to trace.

The instance is replaced on every deployment, on scale-in, on a crash, and whenever the platform moves your workload. Everything written locally goes with it. Worse, while several replicas are running, a photo written to replica two is invisible to a later request served by replica one, so the file appears to exist intermittently.

Uploads, generated reports, exported files and anything else a user expects to find later go to object storage. The local disk is for temporary working space within a single request, and even then you delete it when you are done.

Object storage covers most needs. A file share is the answer to a narrower question.

	Object storage	File share
Called	Azure Blob Storage, Amazon S3	Azure Files, Amazon EFS
Accessed by	HTTP, by key, through an SDK or a signed URL	A mounted path, using ordinary file APIs
Partial writes	No. You replace the whole object	Yes, like a local file
Directories and locks	Simulated by key prefixes. No locking	Real directories and file locks
Cost	Lowest per gigabyte, with tiers to go lower	Noticeably higher
Serving directly to a browser	Yes, with a signed URL or through a CDN	No. Your application has to read and stream it
Use for	Employee photos, uploads, exports, backups, static assets	Legacy software that insists on a filesystem path, or a shared working directory between processes

Tiers, described by behaviour rather than by each provider's product names, which differ and change:

Hot or standard — read any time, highest storage price, lowest read price. Employee photos the mobile app displays daily belong here
Cool or infrequent access — cheaper to store, more expensive per read, sometimes with a minimum storage duration before you can move or delete without a charge. Suits last year's uploads
Archive or deep archive — cheapest storage by a wide margin, and retrieval takes minutes to hours and costs real money. Suits records you keep for a legal retention period and expect never to read
Lifecycle rules move objects between tiers automatically after a number of days, which is where the saving actually comes from. Doing it by hand does not happen
The trap is retrieval cost. A colder tier is only cheaper if your guess about read frequency is right. A monthly report that scans every archived object can cost more than storing it all in the hot tier would have

Direct upload with a presigned URL. The file never passes through the employees API, which keeps large uploads off your compute.

The client asks your API for permission
The mobile app calls an endpoint on the employees API. Your code authenticates the user and checks they are allowed to change this employee's photo. This is the step that must not be skipped — the signed URL itself carries no user identity.
Your API generates a narrow signed URL
One operation — write — on one key, expiring in minutes. Your credential does the signing; the client never sees it.
The client uploads straight to storage
A single HTTP PUT to the signed URL. Your compute is not in the path, so a 30MB photo on a slow connection does not occupy a request thread for two minutes.
The client tells your API the upload finished
Your code verifies the object exists, checks its size and content type, and only then records the key against the employee. Until that point the database knows nothing about it, so an abandoned upload leaves no broken reference.
Downloads work the same way in reverse
Generate a short-lived read URL rather than making the container public. The container stays private and each link stops working on its own.

Employees API — issuing an upload URL

// POST /employees/{id}/photo-upload-url
app.MapPost("/employees/{id}/photo-upload-url", async (
    string id,
    ClaimsPrincipal user,
    IPhotoStorage storage,
    CancellationToken ct) =>
{
    if (!await storage.UserMayEditPhotoAsync(user, id, ct))
        return Results.Forbid();

    // A new key each time. Never trust a client-supplied filename as a key.
    var key = "photos/" + id + "/" + Guid.NewGuid().ToString("n") + ".jpg";

    var url = await storage.CreateWriteUrlAsync(
        key,
        expiresIn: TimeSpan.FromMinutes(10),
        contentType: "image/jpeg",
        ct);

    return Results.Ok(new { uploadUrl = url, key });
})
.RequireAuthorization();

The authorisation check comes first and does the real work. Once the URL is signed, anyone holding it can perform that write, so the decision about who is allowed has to happen before signing.
The key is generated server-side. A client-supplied filename invites path characters, collisions and overwrites of another employee's photo, and it is one of the more common upload vulnerabilities.
The expiry is short. Ten minutes is generous for one upload and limits how long a leaked link stays useful. There is no way to know whether a signed URL has been copied, so a short window is the only control you have.
Pinning the content type means a caller cannot substitute an executable for the image the API expects. Verify it again on completion, because a content type header is a claim, not proof.
The provider SDKs sign these URLs for you — a shared access signature on Azure, a presigned URL on AWS — behind an interface like this one. Keeping the provider behind an interface also keeps your endpoint testable without a storage account.

Common mistake

Common mistakes when starting out

Making the container public so images load in the browser. Public means the whole internet, including anyone who guesses or scrapes a key, and employee photos are personal data. Signed URLs or a CDN with restricted origin access do the same job without the exposure.

Streaming every upload and download through the API. It works, and it consumes a request thread for the whole transfer, so a few large uploads can starve ordinary traffic.

Storing the storage URL in the database instead of the key. Account names, regions and domains change; keys do not. Keep the key and build the URL when you need it.

Recording the database row before confirming the upload. An abandoned upload then leaves a reference to an object that does not exist, and the failure surfaces much later as a broken image.

Using object storage as a queue by polling for new files. Listing is slow and eventually consistent for some operations. Use the platform's storage event notification instead, which is exactly what the photo function in the previous lesson consumes.

Summary

Object storage maps a key to bytes over HTTP — durable, cheap and not a filesystem
Never keep uploads on a container's local disk; it dies with the instance and is invisible to other replicas
Tiers trade storage cost against retrieval cost and delay, and lifecycle rules are what make the saving real
Presigned URLs let clients upload and download directly while the container stays private
Authorise the user before signing a URL, generate keys server-side, and confirm the upload before recording it

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Try it yourself

Design the key layout

Write the key structure for employee photos, including the original upload and a generated thumbnail. Then answer: how would you find every photo for one employee, and how would a lifecycle rule move photos older than a year to a cheaper tier?

Show solution

A structure that works: photos/{employeeId}/{uploadId}.jpg for originals and thumbnails/{employeeId}/{uploadId}.jpg for derived images. Listing by the prefix photos/{employeeId}/ finds one employee's uploads, which is the only listing query most applications need.

Separating originals from thumbnails by prefix rather than by suffix is what makes the lifecycle rule and the access policy possible, because both operate on prefixes. Mixing them under one prefix means any rule you write applies to both.

For age-based tiering, prefer the object's own creation date in the lifecycle rule over a date in the key. A date-based key looks convenient and then locks you into the layout when requirements change.

One defensible alternative is putting the date in the key when you know you will always query by time range and never by employee. Choose based on how you will read the data, because the key layout is expensive to change once objects exist.

Think about it

The photo that vanishes

Field staff report that a photo they uploaded appears for a while, then disappears, and sometimes comes back. The API writes uploads to ./wwwroot/uploads and runs on three replicas.

Explain the behaviour and what you change.

Show solution

Each replica has its own local disk. The upload landed on one of them, so requests routed to that replica find the file and requests routed to the other two do not. That is the appearing and disappearing.

It comes back because load balancing sometimes sends the user to the replica holding the file. Nothing is corrupted and nothing is intermittent in the usual sense — the state is simply in the wrong place.

The permanent version of the same bug is a deployment. Replacing the replicas deletes every upload, and there is no recovery because nothing was ever copied anywhere else.

The fix is object storage for the bytes and the key in the database, so every replica resolves the same photo. Sticky sessions would mask the symptom for one user and would still lose everything on the next deploy, which is why it is not a fix.

Knowledge check

Nothing is recorded and there is no score. The explanation appears either way.

Why is a presigned URL preferable to making the photo container publicly readable?

It downloads fasterIt grants one operation on one object for a short time, instead of granting everyone permanent read accessIt compresses the object automaticallyIt removes the need to authenticate the user

When is moving old employee photos to an archive tier a poor decision?

When the photos are smallWhen a monthly process reads all of them, because retrieval cost and delay outweigh the storage savingWhen there are fewer than a thousand objectsArchive tiers are always the cheaper option

Saved in this browser only.