Building a Document Automation Pipeline with a REST API

Most backend systems eventually need to produce documents: invoices, contracts, shipping labels, reports, receipts. The first version is usually a one-off script that glues together a templating library and a headless browser. It works until volume grows, templates multiply, and someone asks for retries, audit logs, and versioning. At that point you stop having a script and start needing a pipeline.

This post walks through how to build a document automation pipeline backed by a REST API. The goal: turn document generation into a predictable, observable service that the rest of your stack can call without thinking about Chromium processes or font caches.

The shape of the pipeline

A practical pipeline has four stages:

1. Trigger — something happens (an order is placed, a contract is signed).
2. Compose — your service gathers data and picks a template.
3. Render — a REST API turns template + data into a PDF.
4. Deliver — the file is stored, emailed, or returned to the caller.

Keeping these stages separate makes each one independently retryable and testable. Don't render inside your HTTP request handler. Don't email inside your renderer. Each stage should accept input, produce output, and emit an event.

Step 1: Define document jobs as data

Treat every document as a job record before it becomes a file. A minimal schema looks like this:

json
{
  "id": "doc_01HRX9...",
  "template": "invoice_v3",
  "data": { "invoiceNumber": "INV-1042", "total": 199.0 },
  "status": "pending",
  "attempts": 0,
  "createdAt": "2025-01-14T10:00:00Z",
  "resultUrl": null
}

Persisting the job before calling the renderer gives you idempotency, retries, and an audit trail for free. If the renderer crashes, a worker can pick the job up again. If a customer complains about an invoice, you have the exact input that produced it.

Step 2: Call the rendering API

This is where a hosted REST API earns its keep. Running headless Chrome in production means managing memory leaks, font installation, sandboxing, and concurrency limits. A service like Kamy exposes rendering as a single HTTP call:

http
POST /v1/documents/render
Authorization: Bearer sk_live_...
Content-Type: application/json

{
  "template": "invoice_v3",
  "data": {
    "invoiceNumber": "INV-1042",
    "items": [{ "name": "Pro plan", "price": 199.0 }]
  }
}

The response returns a signed URL pointing to the generated PDF:

json
{
  "id": "doc_01HRX9...",
  "url": "https://cdn.kamy.dev/r/abc123.pdf",
  "expiresAt": "2025-01-14T11:00:00Z"
}

From your worker code, the call is straightforward:

ts
import { kamy } from "@kamydev/sdk";

export async function renderInvoice(job: InvoiceJob) {
  const pdf = await kamy.documents.render({
    template: "invoice_v3",
    data: job.data,
  });

  await db.jobs.update(job.id, {
    status: "rendered",
    resultUrl: pdf.url,
  });

  return pdf.url;
}

Whether you use Kamy, a self-hosted renderer, or something else, keep the call behind an interface. You'll thank yourself when you swap providers or add a fallback.

Step 3: Run renders through a queue

Rendering is I/O-bound and occasionally slow. Don't block your API. Push jobs onto a queue and process them with a worker pool.

ts
// producer (HTTP handler)
app.post("/invoices/:id/generate", async (req, res) => {
  const job = await db.jobs.create({
    template: "invoice_v3",
    data: await buildInvoiceData(req.params.id),
    status: "pending",
  });

  await queue.add("render", { jobId: job.id });
  res.status(202).json({ jobId: job.id });
});

ts
// consumer (worker)
queue.process("render", async ({ data }) => {
  const job = await db.jobs.findById(data.jobId);
  try {
    await renderInvoice(job);
  } catch (err) {
    await db.jobs.update(job.id, {
      status: "failed",
      attempts: job.attempts + 1,
    });
    throw err; // let the queue retry
  }
});

Return 202 Accepted and a job ID. Let the client poll or subscribe to a webhook. This pattern scales linearly: add more workers, render more documents.

Step 4: Store and deliver

A signed URL from the rendering API is fine for short-lived downloads, but for anything you need to keep — invoices, signed contracts — copy the file to your own storage immediately:

ts
const response = await fetch(pdf.url);
const buffer = Buffer.from(await response.arrayBuffer());

await s3.putObject({
  Bucket: "documents",
  Key: `invoices/${job.id}.pdf`,
  Body: buffer,
  ContentType: "application/pdf",
});

Now the document lives in your bucket with your retention policy. Delivery becomes a separate step: email it, attach it to a CRM record, expose a download endpoint with your own auth.

Step 5: Make it observable

A pipeline you can't see into is a pipeline you can't trust. At minimum, track:

• Job state transitions (pending → rendered → delivered → failed)
• Render duration per template
• Failure reasons grouped by template version

A simple metrics emit point:

ts
metrics.histogram("document.render.duration_ms", duration, {
  template: job.template,
  status: "success",
});