Skip to content

Workly for Python

The Python SDK for Workly, the open-source HTTP task queue. Enqueue a task, and Workly delivers it to your endpoint over HTTP, retrying with backoff until it succeeds.

pip install workly # the SDK
pip install "workly[cli]" # the SDK plus the workly binary, for `workly dev`

Requires Python 3.11+. The only dependency is httpx. The cli extra adds workly-cli, the workly binary (about 7 MB): the dev server (workly dev) and the CLI, including workly validate, plan and deploy for configuration as code. Install it on dev machines and in CI; production images that only enqueue tasks don’t need it.

Start a local server with workly dev (SQLite, no auth, on :7337), then:

from workly import Workly
client = Workly() # WORKLY_URL, else http://localhost:7337; WORKLY_API_KEY
task = client.tasks.enqueue(
"emails",
url="https://api.example.com/send-email",
payload={"user_id": 123},
)
print(task.id, task.state)

Your endpoint receives a POST with the JSON payload. Any 2xx marks the task succeeded. A 5xx, 408, 429, timeout or connection error is retried with backoff. Delivery is at-least-once, so make handlers safe to run twice. Workly-Task-Id is stable across attempts and works as a deduplication key.

Every delivery is signed: with the server’s WORKLY_SIGNING_SECRET when you run Workly yourself, or with your workspace’s secret on workly.run. Check the signature against the raw body:

from workly import SignatureVerificationError, verify_signature
# Flask
@app.post("/send-email")
def send_email():
try:
verify_signature(request.get_data(), request.headers) # secret from WORKLY_SIGNING_SECRET
except SignatureVerificationError:
abort(401)
...
# FastAPI / Starlette
@app.post("/send-email")
async def send_email(request: Request):
verify_signature(await request.body(), request.headers, secret)
...

By default, signatures older than 5 minutes are rejected (tolerance=). workly dev signs with workly-dev-secret. To test your handler without a server, use workly.sign(body, secret), which returns the headers Workly would send.

On workly.run, point the client at the hosted API and use the keys from the console:

export WORKLY_URL=https://api.workly.run
export WORKLY_API_KEY=... # API keys page
export WORKLY_SIGNING_SECRET=whsec_... # same page; each workspace has its own

Nothing else changes. Hosted workspaces have a request rate limit and a monthly task quota: requests over the rate limit are retried for you, and an enqueue over the quota raises PermissionDeniedError with the code quota_exceeded.

from datetime import datetime, timedelta, timezone
client.tasks.enqueue(
"emails",
url="https://api.example.com/send-email",
payload={"user_id": 123}, # JSON; or body="raw text", content_type="text/csv"
method="POST", # default
headers={"X-Tenant": "acme"}, # sent with every delivery
delay_seconds=60, # or run_at=datetime(..., tzinfo=timezone.utc)
idempotency_key="welcome-123", # an existing task with this key is returned instead
max_attempts=5, # overrides the queue's setting
timeout_seconds=10, # likewise
)
# Up to 100 at once, all or nothing.
client.tasks.enqueue_many(
"emails",
[{"url": "https://api.example.com/send-email", "payload": {"user_id": i}} for i in range(100)],
)
task = client.tasks.get("task_01J9...")
for attempt in client.tasks.attempts(task.id):
print(attempt.number, attempt.outcome, attempt.status_code, attempt.duration_ms)
client.tasks.retry(task.id) # run now; replays a finished task
delivery = client.tasks.delivery(task.id) # the signed request, to replay by hand
client.tasks.cancel(task.id)
# Newest first, across pages. All filters are optional.
for task in client.tasks.list(
queue="emails", state="failed", created_after=datetime.now(timezone.utc) - timedelta(days=1)
):
...

Queues are created automatically on first enqueue. put sets a queue’s configuration declaratively, which makes it easy to keep in code: settings you leave out are reset to their defaults.

client.queues.put(
"emails",
max_attempts=10,
min_backoff_seconds=1,
max_backoff_seconds=3600,
backoff_multiplier=2,
timeout_seconds=30,
rate_per_second=50,
max_concurrency=20,
auto_pause_after=20, # pause after 20 failures in a row, until a probe succeeds
)
queue = client.queues.get("emails")
print(queue.stats.due, queue.stats.oldest_due_at)
client.queues.pause("emails") # stop delivering; enqueueing still works
client.queues.resume("emails")
client.queues.cancel_pending("emails") # cancel pending tasks
client.queues.retry_failed("emails") # replay failed tasks, e.g. after an outage
for group in client.queues.failures("emails"): # failed tasks grouped by error
print(group.count, group.error)
client.queues.retry_failed("emails", last_error="HTTP 503 Service Unavailable") # replay one group
client.queues.activity("emails") # succeeded/failed/cancelled in the last 7 days
client.queues.delete("emails") # delete the queue and all its tasks

A schedule calls a URL on a cron timetable; each call is a run, retried and recorded like a task. Schedules are separate from queues. put is declarative, like queues.put. See Schedules.

schedule = client.schedules.put(
"nightly-report",
cron="0 3 * * *",
timezone="Europe/Stockholm",
url="https://api.example.com/reports/nightly",
payload={"kind": "daily"},
max_attempts=3,
overlap="skip", # skip a slot while the previous run is still going
)
print(schedule.next_runs) # the next five times
run = client.schedules.trigger("nightly-report") # run it now
for run in client.runs.list(schedule="nightly-report", state="failed"):
print(run.id, run.scheduled_for, run.last_error)
client.runs.attempts(run.id)
client.runs.retry(run.id) # deliver it again, e.g. a failed or skipped run
client.schedules.pause("nightly-report")
client.schedules.resume("nightly-report") # from the next scheduled time
client.schedules.delete("nightly-report") # with all its runs

Runs are delivered with Workly-Run-Id, Workly-Schedule and Workly-Scheduled-For headers and the same signature, so verify_signature works unchanged.

A group tracks a set of tasks and enqueues a callback task once it’s closed and they’ve all finished: a POST of the group’s status and counts. See Task groups. Like every callback, it goes to Workly’s _callbacks queue.

groups.enqueue does the whole job in one call: it creates the group, enqueues the tasks into it in batches of 100 and closes it, and it’s safe to retry with the same idempotency_key.

from workly import Callback
group = client.groups.enqueue(
"stock-prices",
(
{"url": "https://api.example.com/jobs/update-stock-price", "payload": {"symbol": c.symbol}}
for c in companies
),
name="daily-stock-prices",
idempotency_key="stock-prices-2026-10-19", # a retry finishes this group rather than start another
# or a dict: {"url": "https://api.example.com/jobs/refresh-views"}
on_complete=Callback(url="https://api.example.com/jobs/refresh-views"),
)

Or step by step: client.groups.create(...), then client.tasks.enqueue(..., group_id=group.id) or enqueue_many for each task, then client.groups.close(group.id), after which it completes once its tasks have finished.

group = client.groups.get(group.id)
print(group.status, group.total, group.succeeded, group.failed, group.remaining)
for task in client.tasks.list(group_id=group.id, state="failed"):
print(task.id, task.last_error)
for group in client.groups.list(name="daily-stock-prices", status="failed"):
print(group.id, group.finished_at)

Tasks and schedules take the same on_complete: each time the task, or a run of the schedule, succeeds, fails or is cancelled, Workly POSTs a summary of it to the URL. See Callbacks.

client.tasks.enqueue(
"exports",
url="https://api.example.com/jobs/export",
payload={"export_id": 42},
on_complete=Callback(url="https://api.example.com/jobs/export-done"),
)
client.schedules.put(
"nightly-report",
cron="0 3 * * *",
url="https://api.example.com/reports/nightly",
on_complete={"url": "https://api.example.com/reports/nightly-done"},
)

The summary arrives as the callback task’s JSON body: task_id, queue, state, attempt_count, last_status, last_error and finished_at for a task; run_id, schedule and the same outcome fields for a run.

A workflow is a DAG of HTTP steps: each step is delivered as a task once the steps in its depends_on have succeeded, and a failed step stops the run. Most workflows live in a workly.yaml and are applied with workly deploy; put does the same from code, declaratively. See Workflows.

client.workflows.put(
"daily-refresh",
steps=[
{"name": "prices", "url": "https://api.example.com/jobs/prices"},
{"name": "fx", "url": "https://api.example.com/jobs/fx"},
{
"name": "analytics",
"url": "https://api.example.com/jobs/analytics",
"depends_on": ["prices", "fx"],
},
],
)
run = client.workflows.trigger(
"daily-refresh", input={"date": "2026-11-13"}, idempotency_key="refresh-2026-11-13"
)
run = client.workflow_runs.get(run.id)
for step in run.steps:
print(step.name, step.status, step.blocked_by, step.error)
for run in client.workflow_runs.list(workflow="daily-refresh", status="failed"):
client.workflow_runs.retry(run.id) # its failed and skipped steps run again
client.workflow_runs.cancel(run.id) # steps not yet started are skipped

A step without payload or body is sent the run’s context as JSON, with input. A step can also wait for a task group: its endpoint answers 2xx with Workly-Wait-For: group/<id>, and the step finishes when the group does.

Alert channels get an email or a Slack message when tasks or schedule runs fail: within about a minute, then at most one every 30 minutes per queue or schedule. See Alerts.

channel = client.alerts.create(
"slack", "https://hooks.slack.com/services/..."
) # or "email", "ops@example.com"
client.alerts.test(channel.id) # send a test alert now
print(client.alerts.list().channels)
client.alerts.delete(channel.id)

AsyncWorkly has the same methods, as coroutines:

from workly import AsyncWorkly
async with AsyncWorkly() as client:
task = await client.tasks.enqueue("emails", url="...", payload={"user_id": 123})
async for task in client.tasks.list(queue="emails"):
...
Exception When
BadRequestError 400, invalid request
AuthenticationError 401, missing or wrong API key
PermissionDeniedError 403, not allowed, or a hosted limit was reached (e.g. quota_exceeded)
NotFoundError 404, no such queue or task
ConflictError 409, e.g. retrying a running task
PayloadTooLargeError 413
RateLimitError 429, still rate limited after the retries
APIError base class of the above; has .status, .code (stable) and .message
APIConnectionError the server couldn’t be reached

Requests are retried up to max_retries times (2 by default), honouring Retry-After. A 429 is always retried, since the server refused the request without doing anything. Connection errors and 502–504 are retried for requests that are safe to repeat: an enqueue only when it has an idempotency_key, or when the connection failed before the request was sent.

Apache-2.0