Webhook-driven PDF workflows: async rendering at scale

The first PDF endpoint you write is almost always synchronous. A user hits POST /invoices/:id/pdf, your server renders a document, and you stream bytes back. It works fine — until it doesn't. A 40-page report takes 12 seconds to render, your HTTP timeout is 30 seconds, and suddenly a burst of concurrent requests is holding open connections, exhausting your worker pool, and triggering 502s at the load balancer.

Async rendering with webhooks solves this. Instead of blocking on the render, you accept the job, return a job ID immediately, and notify the caller when the PDF is ready. This post walks through the pattern, the failure modes, and how to make it reliable.

Why sync rendering breaks

A few realities of PDF generation:

• Render times are bimodal. Simple receipts render in 200ms. Multi-page reports with charts and embedded fonts can take 10–30 seconds.
• Chromium-based renderers (which most modern PDF services use under the hood) are memory-heavy. Each concurrent render can consume 200–500MB.
• HTTP clients timeout. Load balancers timeout. Your users' browsers timeout.

If you serve renders inline, your API's throughput is bounded by your slowest document. Async decouples request acceptance from work completion.

The basic shape

The pattern has three moving parts:

1. A POST endpoint that enqueues a job and returns a job ID.
2. A worker that renders the PDF and uploads the result to object storage.
3. A webhook delivery step that notifies the caller with a URL to download the file.

Here's the client side using Kamy, which supports webhook callbacks natively:

ts
import { kamy } from "@kamydev/sdk";

const job = await kamy.documents.renderAsync({
  template: "quarterly-report",
  data: { quarter: "Q3-2024", accountId: "acc_123" },
  webhook: {
    url: "https://api.yourapp.com/hooks/pdf-ready",
    secret: process.env.KAMY_WEBHOOK_SECRET,
  },
});

console.log(job.id); // "job_01HXYZ..."

The response comes back in milliseconds. Your API is free to accept the next request.

Receiving the webhook

When rendering completes, Kamy sends a signed POST to your webhook URL:

json
{
  "event": "document.rendered",
  "job_id": "job_01HXYZ...",
  "status": "succeeded",
  "document": {
    "url": "https://cdn.kamy.dev/d/abc123.pdf",
    "expires_at": "2024-11-15T18:00:00Z",
    "size_bytes": 184203,
    "pages": 12
  }
}

Your handler needs to do three things: verify the signature, look up the job, and trigger whatever comes next (email the user, mark an order complete, etc.).

ts
import crypto from "node:crypto";
import express from "express";

const app = express();

app.post(
  "/hooks/pdf-ready",
  express.raw({ type: "application/json" }),
  async (req, res) => {
    const signature = req.header("X-Kamy-Signature");
    const expected = crypto
      .createHmac("sha256", process.env.KAMY_WEBHOOK_SECRET!)
      .update(req.body)
      .digest("hex");

    if (signature !== expected) {
      return res.status(401).send("invalid signature");
    }

    const payload = JSON.parse(req.body.toString());
    await handleRenderComplete(payload);

    res.status(200).send("ok");
  }
);

Return 2xx fast. Do the actual downstream work — sending email, updating your database, posting to Slack — in a background job. Webhook senders retry on non-2xx responses, and you don't want a slow email provider to cause duplicate deliveries.

Idempotency

Webhooks will be delivered more than once. Network hiccups, timeouts on your side, and retry policies all conspire to send the same event twice. Track processed job_ids:

ts
async function handleRenderComplete(payload: RenderedPayload) {
  const alreadyProcessed = await db.webhookEvents.findUnique({
    where: { jobId: payload.job_id },
  });

  if (alreadyProcessed) return;

  await db.$transaction([
    db.webhookEvents.create({ data: { jobId: payload.job_id } }),
    db.invoices.update({
      where: { pdfJobId: payload.job_id },
      data: { pdfUrl: payload.document.url, status: "ready" },
    }),
  ]);

  await queue.enqueue("send-invoice-email", { jobId: payload.job_id });
}

The unique constraint on jobId gives you idempotency at the database level — even if two webhook deliveries race, one transaction will fail cleanly.

Handling failures

Renders can fail: bad template data, oversized images, upstream timeouts. Your webhook handler should treat failure events as first-class:

ts
if (payload.status === "failed") {
  await db.invoices.update({
    where: { pdfJobId: payload.job_id },
    data: {
      status: "render_failed",
      failureReason: payload.error.message,
    },
  });
  await alerting.notify(`PDF render failed: ${payload.error.code}`);
  return;
}

Don't swallow failures silently. If 2% of your renders fail with TEMPLATE_DATA_INVALID, you want to know before your customers do.

Scaling considerations

Once async is in place, a few things get easier and a few get harder.

Easier: You can absorb traffic sp