Messaging and Queues

A queue is a durable list. One process appends work to it, another takes work off it, and neither has to be running at the same moment. The producer's obligation ends when the broker has accepted the message; whether the consumer is busy, restarting or scaled to zero is not the producer's problem.

Two things come out of that. The first is decoupling in time: a web request can hand over a job that takes ninety seconds and answer the user immediately. The second is absorbing spikes. Invoicing runs on the first of the month and two thousand invoices need rendering; the renderer manages about twenty a second. Without a queue, two thousand requests arrive at once and something falls over. With one, the work becomes a hundred seconds of steady load and nothing is lost. The queue does not make the work faster. It makes the arrival rate somebody else's problem.

// Producer — the web request returns as soon as the broker has the message
public sealed class InvoiceRenderingQueue(ServiceBusSender sender)
{
    public Task EnqueueAsync(int invoiceId, string correlationId, CancellationToken token)
    {
        var message = new ServiceBusMessage(
            JsonSerializer.SerializeToUtf8Bytes(new RenderInvoice(invoiceId)))
        {
            MessageId = "render-invoice-" + invoiceId,   // duplicate-detection key
            CorrelationId = correlationId,
            SessionId = invoiceId.ToString(),           // ordering, per invoice only
        };

        return sender.SendMessageAsync(message, token);
    }
}

// Consumer — a background service, four messages at a time
protected override async Task ExecuteAsync(CancellationToken stopping)
{
    ServiceBusProcessor processor = client.CreateProcessor(
        "invoice-rendering",
        new ServiceBusProcessorOptions
        {
            MaxConcurrentCalls = 4,
            AutoCompleteMessages = false,   // completing means the work is finished
        });

    processor.ProcessMessageAsync += HandleAsync;
    processor.ProcessErrorAsync += args =>
    {
        log.LogError(args.Exception, "Processor error on {Entity}.", args.EntityPath);
        return Task.CompletedTask;
    };

    await processor.StartProcessingAsync(stopping);

    try { await Task.Delay(Timeout.Infinite, stopping); }
    finally { await processor.StopProcessingAsync(CancellationToken.None); }
}

• The producer is finished once the broker acknowledges the message. Nothing in this method knows whether a consumer exists, which is the decoupling you are buying.
• MessageId gives brokers with duplicate detection something to compare. It suppresses a repeat within a configured window — a window, not a promise, so the consumer still has to tolerate duplicates.
• SessionId groups messages so everything for one invoice is handled in order, by one consumer, one at a time. Note what it does not give you: any ordering between different invoices. Per-key ordering is usually what the business actually needs, and it is much cheaper than global ordering.
• Turning off automatic completion is the important line. With it on, the message is marked done before your handler finishes, so a crash mid-render loses the work and the queue looks healthy. Completing it yourself means completion states that the work happened.
• Each message needs its own dependency injection scope, exactly as a web request does. Resolving a DbContext once for the worker's lifetime gives four concurrent messages one shared context, and the errors that produces are intermittent and hard to read.
• Stopping deliberately on shutdown lets messages in flight finish instead of being abandoned. Abandoned work is not lost — it is redelivered — but redelivery is where duplicates come from, so reducing it is worth two lines.

private const int MaxAttempts = 5;

private async Task HandleAsync(ProcessMessageEventArgs args)
{
    RenderInvoice work = args.Message.Body.ToObjectFromJson<RenderInvoice>();

    await using AsyncServiceScope scope = scopes.CreateAsyncScope();
    var renderer = scope.ServiceProvider.GetRequiredService<IInvoiceRenderer>();

    try
    {
        // Contains the idempotency guard: rendering the same invoice twice
        // must not produce two documents or two emails.
        await renderer.RenderAsync(work.InvoiceId, args.CancellationToken);
        await args.CompleteMessageAsync(args.Message, args.CancellationToken);
    }
    catch (InvoiceNotFoundException ex)
    {
        // No number of retries will make this invoice exist.
        await args.DeadLetterMessageAsync(
            args.Message, "UnknownInvoice", ex.Message, args.CancellationToken);
    }
    catch (Exception ex) when (args.Message.DeliveryCount >= MaxAttempts)
    {
        log.LogError(ex, "Giving up on invoice {InvoiceId} after {Attempts} attempts.",
            work.InvoiceId, args.Message.DeliveryCount);

        await args.DeadLetterMessageAsync(
            args.Message, "AttemptLimitReached", ex.Message, args.CancellationToken);
    }

    // Anything else: leave the message uncompleted. The broker redelivers it
    // when the lock expires, and DeliveryCount goes up by one.
}

• Two kinds of failure need two responses. A message referring to an invoice that does not exist will fail identically five times, so it goes straight to the dead letter queue and stops occupying a consumer.
• A transient failure — a timeout, a deadlock, a restarting dependency — is worth another attempt, and the broker's DeliveryCount is the counter you use rather than one of your own.
• The absence of code at the end is the mechanism. Not completing a message is what causes redelivery. That is invisible in review, which is why the comment is there.
• Without an attempt limit, a message that always fails is redelivered forever. On a session-based queue it blocks every later message for the same invoice; on a plain queue it burns consumer capacity and fills your logs with the same stack trace.
• Dead lettering is half a solution. A dead letter queue with no alarm is where work goes to be forgotten, and the symptom is a customer asking about an invoice nobody can find. Alarm on a depth above zero, give it an owner, and make sure a fixed message can be replayed.
• The render itself still needs the idempotency guard from the event-driven lesson. Redelivery is normal operation, not an edge case, so producing a second document has to be impossible rather than unlikely.