Workly

Task queue

Your background job is an endpoint.

Enqueue a task with a URL and a payload, and Workly delivers it to your app as a signed HTTP request, retries it until it succeeds, and keeps every attempt. There are no workers or broker to run, and any language that answers HTTP can handle a task, serverless included.

Enqueue, deliver, retry, record.

01

Enqueue

A URL, a JSON payload and a queue name, from the Python or TypeScript SDK, the CLI or plain HTTP. The queue is created the first time you use it.

02

Deliver

Workly sends the request, signed, within milliseconds of it being due, at the queue's rate and concurrency.

03

Retry

A 2xx and the task is done. A 5xx, 408, 429, timeout or dropped connection is retried with backoff until it succeeds or runs out of attempts.

04

Record

Every attempt is kept with its status code, error, response body and timing, and a failed task stays until you retry it or it ages out.

enqueue.py
from workly import Workly

workly = Workly()  # WORKLY_URL, WORKLY_API_KEY
workly.tasks.enqueue(
    "emails",
    url="https://api.example.com/tasks/send-email",
    payload={"user_id": 123},
    idempotency_key="welcome-123",  # enqueue twice, get one task
    delay_seconds=600,              # or run_at
)
route.ts
import { SignatureVerificationError, verifySignature } from "@worklydev/sdk";

// app/tasks/send-email/route.ts, or any Fetch API handler
export async function POST(req: Request) {
  const body = await req.text();
  try {
    await verifySignature(body, req.headers);
  } catch (err) {
    if (err instanceof SignatureVerificationError) return new Response(null, { status: 401 }); // not retried
    throw err;
  }
  await sendEmail(JSON.parse(body).user_id);
  return new Response(null, { status: 204 }); // 2xx: done
}
workly.yaml
# workly.yaml, applied with `workly deploy`
version: 1
queues:
  emails:
    max_attempts: 5
    timeout_seconds: 30
    rate_per_second: 5      # the email API's limit
    max_concurrency: 10

Retries, limits and timeouts are set per queue.

Retries, limits and timeouts belong to the queue, not to every call that enqueues into it, so they live in one place and change without a deploy of your app.

Retries that behave

Five attempts by default, with exponential backoff and jitter, and Retry-After honoured. A 4xx fails at once, since sending it again won't help, unless the queue sets retry_on_4xx.

Rate and concurrency limits

rate_per_second and max_concurrency per queue, across every delivery, so a backlog never floods the API behind it. Busy queues don't hold up quiet ones.

Timeouts

Each attempt gets the queue's timeout_seconds, up to 30 minutes, and a task can override it. No handler hangs forever.

Delayed tasks

Send a task in ten minutes with delay_seconds, or at a set time with run_at. Pause a queue during an incident; enqueueing keeps working. Or let it pause itself (below).

Idempotency keys

Enqueue with the same idempotency_key and get the existing task back, so retrying your own request after a timeout is safe.

Signed deliveries

Every request carries an HMAC-SHA256 signature with a timestamp. The SDKs check it in one call, and in any other language it's a few lines.

Deduplicate in the handler

Delivery is at least once. Workly-Task-Id is the same on every attempt, so a handler can tell a retry from new work.

Callbacks

Give a task an on_complete URL and Workly calls it with the outcome when the task finishes, so your app reacts without polling.

Traces that connect

A task carries the traceparent of the request that enqueued it, so your handler's spans join the same trace.

Every attempt ends one of five ways.

Each delivery is recorded with its start, duration, status code, response headers and the first 4 KB of the body, and how it ended. That's what decides a retry, and what you read when a task fails.

OutcomeMeansRetried
successA 2xx response. The task is done.—
http_errorAny other response.5xx, 408 and 429; other 4xx only with retry_on_4xx
timeoutNo response within the timeout.Yes
network_errorDNS, connection refused or reset, TLS.Yes
lostWorkly stopped mid-delivery, so the result is unknown.Yes
A queue in the Workly dashboard: its backlog, recent activity and failed tasks grouped by error.

When something fails, the dashboard groups it by error and lets you retry the group. How debugging works →

When an endpoint goes down, the queue waits for it.

Retries spend each task's attempts during an outage, and a long one leaves every task failed, to retry by hand. Give a queue auto_pause and it pauses itself after a run of failures, tries one task now and then, and resumes on its own once the endpoint answers.

Trips on outages, not bad requests

5xx responses, timeouts and network errors in a row count. A 4xx about one task, a 429 or a lost attempt doesn't, and any success resets the count.

Tasks keep their attempts

While the queue is paused, its tasks wait as pending. Nothing fails because the endpoint was down, so there's nothing to retry by hand afterwards.

One probe at a time

Workly sends one task 30 seconds after the pause, then twice as long after each one that fails, up to every 10 minutes. The first success resumes the queue, and the backlog drains at its limits.

You hear about it

An alert when the queue pauses and another when it resumes. The dashboard says why and when the next probe goes, with Resume now for when you know it's back.

workly.yaml
# workly.yaml
version: 1
queues:
  billing:
    auto_pause:
      # 5xx, timeouts or network errors in a row
      after_failures: 20

For a queue whose tasks go to one service. In a queue that delivers to many endpoints, one being down would hold up the rest. Auto-pause in the docs →

Where a task queue isn't the answer.

No strict ordering

Tasks start roughly in the order they're due, but with retries and more than one delivery at a time they can finish in any order. max_concurrency: 1 gets you close.

A request at a time

An attempt can take up to 30 minutes. Work that runs for hours belongs in a worker, or split into tasks.

Not a message bus

Each task goes to one URL. For streaming or pub/sub to many consumers, use Kafka, NATS or similar.

Questions

What is an HTTP task queue?
A queue that delivers each job as an HTTP request to your app, instead of handing it to a worker process that pulls from a broker. Your background job is a route in the app you already run, and the queue takes care of retries, timeouts, rate limits and keeping a record of every attempt. Google Cloud Tasks works this way; Workly is an open-source one you can run anywhere.
Do I need to run workers or Redis?
No. Workly pushes tasks to your endpoints, so there are no workers to deploy or scale, and no broker. Self-hosted, Workly is one server plus Postgres (or SQLite on your laptop). Hosted, there is nothing to run.
Does it work with serverless functions?
Yes. A task is an HTTP request, so any endpoint that answers HTTP can handle it: a route in Next.js, a Lambda or Cloud Run function, a FastAPI or Express app. Set the queue's timeout to fit how long your function may run.
What happens when my endpoint is down?
The task is retried with backoff, by default five attempts over a growing wait, and Retry-After is honoured. If it runs out of attempts, it stays as failed with every attempt's response, grouped with other tasks that failed the same way, and you can retry them all once the endpoint is back. For a longer outage, give the queue auto_pause: after a run of failures it pauses itself, its tasks wait without using attempts, and it resumes on its own when the endpoint answers again.
Is delivery exactly once?
At least once. Workly never drops a task, but a handler can see one twice, for example when the server stops after sending a request and before recording the answer. Use the Workly-Task-Id header, the same on every attempt, to make the handler safe to run twice.
How fast is it?
One server sustains 1,000 tasks a second end to end, with deliveries starting within milliseconds of the enqueue, on Postgres or SQLite.

Try it in five minutes.

workly dev gives you a server and dashboard on your laptop with nothing else to start, and the Python and TypeScript SDKs let your app enqueue to it. Or sign in to hosted Workly and get a workspace and an API key in a minute, free on the Hobby plan.