This is the abridged developer documentation for Workly
# Workly documentation
> Workly is an open-source HTTP task queue and cron scheduler: enqueue a task with a URL, and Workly delivers it with retries, rate limits and full history.
Workly is an open-source HTTP task queue and scheduler. You enqueue a task with a URL and a payload; Workly delivers it to your endpoint and retries with backoff until the endpoint answers 2xx. There are no workers to run and no broker to operate. It also runs [schedules](https://workly.run/docs/schedules/), which call an endpoint on a cron timetable; [task groups](https://workly.run/docs/groups/), which call you back once a set of tasks has finished; and [workflows](https://workly.run/docs/workflows/), which run HTTP steps in dependency order. The same binary runs everywhere: `workly dev` on your laptop (SQLite, no configuration), `workly serve` with Postgres in your own infrastructure, or hosted at [workly.run](https://workly.run). Moving between them means changing environment variables, not code. ## Start here * [Quickstart](https://workly.run/docs/quickstart/): run Workly locally, enqueue a task and handle it. * [Concepts](https://workly.run/docs/concepts/): queues, tasks, attempts, retries, delivery semantics and signing, including verifying signatures without an SDK. * [Schedules](https://workly.run/docs/schedules/): call an endpoint on a cron timetable. * [Task groups](https://workly.run/docs/groups/): fan out tasks and get one callback when they’ve all finished. * [Workflows](https://workly.run/docs/workflows/): run HTTP steps in dependency order, each delivered as a task, waiting for groups where they fan out. * [Callbacks](https://workly.run/docs/callbacks/): be called with the outcome each time a task or a schedule’s run finishes. * [Alerts](https://workly.run/docs/alerts/): hear about failed tasks and runs by email or in Slack. * [Configuration as code](https://workly.run/docs/deploy/): keep queues, workflows and schedules in a `workly.yaml` and deploy it with `workly deploy`. * [Self-hosting](https://workly.run/docs/self-hosting/): run it in production on Docker, Compose or Kubernetes. * [MCP](https://workly.run/docs/mcp/): let coding agents see inside your queues and act on them. ## How delivery works * **Tasks** belong to a **queue**, created on first use. The queue holds the delivery settings: retries, timeout, rate and concurrency limits. A task can override its retries and timeout. * Each delivery is an **attempt**. A 2xx response marks the task succeeded. A 5xx, 408 or 429 response, a timeout or a connection error is retried with exponential backoff and jitter, honouring `Retry-After`. Other 4xx responses fail the task straight away, unless the queue sets `retry_on_4xx`. * By default a queue delivers up to 3 tasks at once, makes up to 5 attempts, waits between 2 seconds and 1 hour between them, and times attempts out after 30 seconds. Raise `max_concurrency` (up to 10,000) for endpoints that can take more. * A task that runs out of attempts is **failed**. It keeps every attempt’s status, error and response body, and once the endpoint is fixed you can retry it, or all of a queue’s failures at once. * Delivery is **at least once**: if the server stops mid-delivery, the attempt is recorded as `lost` and the task is delivered again. Make handlers safe to run twice, using the `Workly-Task-Id` header, which is the same on every attempt, as a deduplication key. * Every delivery is **signed** with `Workly-Signature`. The SDKs check it in one call, and [Concepts](https://workly.run/docs/concepts/#verifying-a-signature-without-an-sdk) shows how in any language. ## Reference * SDKs: [Python](https://workly.run/docs/sdks/python/) and [TypeScript](https://workly.run/docs/sdks/typescript/), with every method. * [CLI](https://workly.run/docs/cli/): every `workly` command and flag. * [API reference](https://workly.run/docs/api/): the HTTP API, generated from its OpenAPI spec. * [Configuration](https://workly.run/docs/configuration/): every `WORKLY_*` setting. * [Metrics](https://workly.run/docs/metrics/): Prometheus metrics and example alerts. * [Performance](https://workly.run/docs/performance/): load test results. ## For AI agents * [workly.run/docs/llms-full.txt](https://workly.run/docs/llms-full.txt) has all of these docs in one file, SDKs and CLI included. [llms-small.txt](https://workly.run/docs/llms-small.txt) leaves out the reference pages and self-hosting. * Each page is also Markdown at its address plus `.md`, such as [workly.run/docs/quickstart.md](https://workly.run/docs/quickstart.md). * The API is described by [openapi.yaml](https://workly.run/docs/api/openapi.yaml). * An agent connected over [MCP](https://workly.run/docs/mcp/) can read these pages with the `get_docs` tool.
# Quickstart
> Run Workly locally with workly dev, enqueue a task from Python, TypeScript or curl, and handle it in your own endpoint. About five minutes.
Run Workly on your machine, enqueue a task and handle it in your own code. ## Install
```sh
pip install "workly[cli]" # the Python SDK plus the workly binary
# or just the binary:
curl -fsSL https://raw.githubusercontent.com/worklydev/workly/main/install.sh | sh
```
Binaries for Linux, macOS and Windows are also on the [releases page](https://github.com/worklydev/workly/releases). With Docker, `docker run -p 7337:7337 ghcr.io/worklydev/workly dev`. SDKs, the CLI and the server work together when their first two version numbers match: any 0.4.x SDK or CLI works with any 0.4.x server. Each is released on its own, so the last number can differ. workly.run runs the latest server; `workly --version` prints the CLI’s. ## Run it
```sh
workly dev
```
This starts a server and the dashboard on , with SQLite storage in `.workly/` and no API key. It prints its settings, including the signing secret (`workly-dev-secret`). ## Enqueue a task `workly dev` has a built-in echo target at `/_dev/echo`. With `fail_rate=0.5` it fails half the time, so you can watch retries:
```sh
curl -X POST localhost:7337/v1/queues/demo/tasks \
-d '{"url": "http://localhost:7337/_dev/echo?fail_rate=0.5", "payload": {"hello": "world"}}'
```
The queue `demo` is created on first use. Queue names are lowercase letters, digits, `-` and `_`; names starting with `_` are reserved for Workly. See the task’s attempts in the dashboard, or with the API:
```sh
curl localhost:7337/v1/tasks/
curl localhost:7337/v1/tasks//attempts
```
## Enqueue from your code Python (`pip install workly`):
```python
from workly import Workly
client = Workly() # WORKLY_URL, else http://localhost:7337; WORKLY_API_KEY
task = client.tasks.enqueue(
"emails",
url="http://localhost:8000/send-email",
payload={"user_id": 123},
)
```
TypeScript (`npm install @worklydev/sdk`):
```ts
import { Workly } from '@worklydev/sdk';
const workly = new Workly(); // WORKLY_URL, else http://localhost:7337; WORKLY_API_KEY
const task = await workly.tasks.enqueue('emails', {
url: 'http://localhost:8000/send-email',
payload: { user_id: 123 },
});
```
Tasks can also be delayed, deduplicated with an idempotency key, and given their own retries and timeout: the [Python](https://workly.run/docs/sdks/python/#tasks) and [TypeScript](https://workly.run/docs/sdks/typescript/#tasks) SDK pages cover every option. ## Handle it Your endpoint receives a `POST` with the JSON payload. Answer 2xx when the work is done; anything else is retried. Check the signature against the raw body first:
```python
from flask import Flask, abort, request
from workly import SignatureVerificationError, verify_signature
app = Flask(__name__)
@app.post("/send-email")
def send_email():
try:
verify_signature(request.get_data(), request.headers) # secret from WORKLY_SIGNING_SECRET
except SignatureVerificationError:
abort(401)
user_id = request.get_json()["user_id"]
...
return "", 204
```
```ts
import { SignatureVerificationError, verifySignature } from '@worklydev/sdk';
export async function POST(request: Request) {
const body = await request.text();
try {
await verifySignature(body, request.headers); // secret from WORKLY_SIGNING_SECRET
} catch (err) {
if (err instanceof SignatureVerificationError) return new Response(null, { status: 401 });
throw err;
}
const { user_id } = JSON.parse(body);
// ...
return new Response(null, { status: 204 });
}
```
Run your handler with `WORKLY_SIGNING_SECRET=workly-dev-secret`. Delivery is at least once, so make the handler safe to run twice; `Workly-Task-Id` is the same on every attempt. ## Shape the queue A new queue delivers up to 3 tasks at once, with no rate limit:
```sh
workly queues ls
curl -X PUT localhost:7337/v1/queues/emails -d '{"rate_per_second": 5, "max_concurrency": 2}'
workly queues pause emails
```
When a handler has been failing, see its failures grouped by error, fix it, and retry them:
```sh
workly queues failures emails
workly queues retry-failed emails
```
## Run something on a schedule [Schedules](https://workly.run/docs/schedules/) call an endpoint on a cron timetable, separately from queues. This one calls the echo target every minute:
```sh
workly schedules put heartbeat http://localhost:7337/_dev/echo --cron '* * * * *'
workly schedules trigger heartbeat # run it now, too
workly schedules get heartbeat # its next times and recent runs
```
## Next * Connect your coding agent over [MCP](https://workly.run/docs/mcp/) to enqueue test tasks and read failures while you work on a handler. * Go to production with [self-hosting](https://workly.run/docs/self-hosting/), or point `WORKLY_URL` at a hosted server.
# Concepts
> How Workly queues, tasks and attempts work: retries and backoff, at-least-once delivery, delivery headers, and verifying signatures without an SDK.
Queues, tasks and attempts, what happens on each delivery, and how your endpoint checks that a request came from Workly. [Schedules](https://workly.run/docs/schedules/) and [task groups](https://workly.run/docs/groups/) build on the same pieces. ## Queues A **queue** holds tasks and the policy for delivering them. It’s created with the defaults below the first time you enqueue into it. Change its policy with `PUT /v1/queues/{queue}`, `workly queues put` or a [`workly.yaml`](https://workly.run/docs/deploy/). | Setting | Default | | | ------------------------------------------------------------------ | ---------- | -------------------------------------------------------------------------------------------------------------- | | `max_attempts` | 5 | Total attempts before a task fails. | | `min_backoff_seconds`, `max_backoff_seconds`, `backoff_multiplier` | 2, 3600, 2 | The wait between attempts (see [Retries](#retries)). | | `timeout_seconds` | 30 | How long one attempt may take. | | `max_concurrency` | 3 | Deliveries in flight at once, up to 10,000. | | `rate_per_second` | none | Deliveries started per second. | | `retry_on_4xx` | false | Retry 4xx responses other than 408 and 429, which are always retried. | | `auto_pause` | none | `{after_failures: n}`: pause the queue by itself when its endpoint looks down (see [Auto-pause](#auto-pause)). | The full list is in the [API reference](https://workly.run/docs/api/). Queue names are lowercase letters, digits, `-` and `_`; names starting with `_` are Workly’s own. A **paused** queue keeps accepting tasks but doesn’t start deliveries; those already in flight finish. Workly shares deliveries fairly between queues, so a busy queue doesn’t hold up a quiet one. ### Auto-pause When an endpoint goes down, retries keep using up each task’s attempts, and an outage longer than the retry window leaves every task `failed`, to retry by hand afterwards. A queue with `auto_pause` stops instead:
```yaml
queues:
billing:
auto_pause:
after_failures: 20
```
* **What pauses it:** `after_failures` attempts in a row that end with a 5xx, a timeout or a network error. These say the endpoint itself is unwell. Other 4xx responses (usually about one task), 429 (the endpoint is pacing you, and `Retry-After` handles it) and lost attempts don’t count, and only a success resets the count. * **While it’s paused:** no deliveries start, and tasks wait as `pending` without using attempts. One task at a time goes as a probe: 30 seconds after the pause, then after twice as long each time a probe fails, up to every 10 minutes. A probe is a normal attempt of the queue’s oldest due task. * **Resuming:** the first success resumes the queue on its own, and the waiting tasks are delivered within its rate and concurrency limits. Resuming it by hand (`POST /v1/queues/{queue}/resume`, `workly queues resume`, or **Resume now** in the dashboard) does it at once. Pausing it by hand stops the probes too. * **Seeing it:** the queue’s `auto_paused` says since when, the error, how many probes failed and when the next goes; it’s absent when the queue isn’t auto-paused. [Alerts](https://workly.run/docs/alerts/) go out when a queue pauses itself and when it resumes, and the `workly_queue_auto_paused` metric is 1 while it’s paused. Use it for a queue whose tasks go to one service. In a queue that delivers to many endpoints, such as webhooks to your customers’ URLs, one of them being down would hold up the rest. Workly’s own queues (`_workflows`, `_callbacks`) can’t have it. A workflow step in an auto-paused queue waits like any task, and its run still fails after its `max_duration_seconds`. ## Tasks A **task** is one HTTP request Workly owes your endpoint: a URL, a method (`POST` by default), headers, and a body, either `payload` sent as JSON or `body` with its own `content_type`. `delay_seconds` or `run_at` sets when it’s first due. It can override its queue’s `max_attempts` and `timeout_seconds`, and with `on_complete` call you back each time it finishes (see [Callbacks](https://workly.run/docs/callbacks/)). A task is in one of these states: | State | | | ----------- | ---------------------------------------------------------------------------------------------------------------------------- | | `pending` | Waiting to be delivered: due now, waiting for its `run_at`, or waiting to be retried. | | `running` | An attempt is in flight. | | `succeeded` | An attempt got a 2xx response. | | `failed` | It used up its attempts, or got a response that isn’t retried. | | `cancelled` | Cancelled before it succeeded or failed. A running task’s current attempt still finishes and is recorded, but isn’t retried. | `failed` isn’t the end: once the endpoint is fixed, retry one task (`POST /v1/tasks/{id}/retry`) or every failure in a queue (`workly queues retry-failed`), and it gets a fresh set of attempts. Finished tasks are kept, with their attempts, for `WORKLY_RETENTION` (7 days by default), then deleted. ### Idempotency keys Enqueue a task with an `idempotency_key` that’s already in the queue and you get the existing task back (`200` instead of `201`). A key holds as long as its task is kept, so retrying an enqueue after a timeout or a crash is safe. This deduplicates enqueues; deduplicating deliveries is your handler’s job (see [Delivery semantics](#delivery-semantics)). ## Attempts Each delivery of a task is an **attempt**, recorded with its start and end, duration, status code, response headers, the first 4 KB of the response body, and how it ended: | Outcome | | | --------------- | ------------------------------------------------------------------------------------------- | | `success` | A 2xx response. | | `http_error` | Any other response. | | `timeout` | No response within `timeout_seconds`. | | `network_error` | The connection failed: DNS, refused, reset, TLS. | | `lost` | Workly stopped mid-delivery, so the result is unknown. The endpoint may have done the work. | This history is what the dashboard, the CLI and [MCP](https://workly.run/docs/mcp/) show when you ask why a task failed. ### Retries * **Retried:** 5xx, 408 and 429 responses, timeouts, network errors and lost attempts. * **Not retried:** other 4xx responses fail the task at once, since sending the same request again won’t change the answer. Set `retry_on_4xx` on the queue if your endpoint uses them for temporary problems. Connections the server refuses to make, such as to a private address on a server that only delivers to the public internet, aren’t retried either. * **Redirects:** Workly doesn’t follow them. A 3xx response (or a 1xx) fails the task at once, since it usually means the URL is wrong: point the task at where the redirect goes. * **Backoff:** retry *n* waits a random time between `min_backoff_seconds` and `min_backoff_seconds × backoff_multiplier^(n−1)`, capped at `max_backoff_seconds`. The randomness (jitter) spreads out a burst of failures. A `Retry-After` header on the response is honoured. ## Delivery semantics Delivery is **at least once**. Workly never drops a task, but your endpoint can see one more than once: * Workly can stop between sending a request and recording the response (a crash, a deploy, a lost connection to the database). The attempt is recorded as `lost` and the task is delivered again. * Your endpoint can do the work and then time out, or fail to answer. Workly can’t tell that apart from not doing it. So make handlers safe to run twice. Every delivery carries headers to help: | Header | | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `Workly-Task-Id` | The task, the same on every attempt. Use it as a deduplication key. | | `Workly-Queue` | The task’s queue. | | `Workly-Attempt` | The attempt number, from 1. | | `Workly-Timestamp`, `Workly-Signature` | The signature (see [Signing](#signing)). | | `traceparent` | The trace context of the request that enqueued the task, if it had one, so your handler’s spans join that trace. | Your own `headers` are sent too; `Workly-*` names are reserved. Schedule runs carry `Workly-Run-Id`, `Workly-Schedule` and `Workly-Scheduled-For` instead of the task headers ([schedules](https://workly.run/docs/schedules/#handling-a-run)), and a [workflow](https://workly.run/docs/workflows/#what-a-step-receives)’s steps carry `Workly-Workflow`, `Workly-Workflow-Run-Id`, `Workly-Workflow-Step`, `Workly-Workflow-Step-Id` and `Workly-Workflow-Run-Attempt` as well. Order isn’t guaranteed. Tasks start roughly in the order they’re due, but with several deliveries at once, and with retries, they can finish in any order. If order matters, set `max_concurrency: 1`; a retry still lets later tasks go first. ## Signing Every delivery is signed with a secret that Workly and your endpoint share (`WORKLY_SIGNING_SECRET` when you self-host; the API keys page on workly.run). Check the signature before trusting a request: anyone who can reach your endpoint can call it, but only Workly knows the secret. The [SDKs](https://workly.run/docs/quickstart/#handle-it) check it with `verify_signature` in Python and `verifySignature` in TypeScript. In another language it’s a few lines. ### Verifying a signature without an SDK A delivery carries two headers:
```text
Workly-Timestamp: 1790000000
Workly-Signature: v1=00f7255c88a43b3e2d47b211ff8e71cf935601cabff5e7adad0de2cc72a4e3c6
```
The signature is the hex HMAC-SHA256, keyed with the secret, of the timestamp, a `.`, and the request body exactly as it arrived:
```text
v1=hex(HMAC-SHA256(secret, "."))
```
To verify a request: 1. Read the **raw body**, before any JSON parsing: re-serializing it changes the bytes and breaks the signature. 2. Compute the HMAC of `timestamp + "." + body` with your secret. 3. Split `Workly-Signature` on commas. It holds more than one signature while the secret is being [rotated](https://workly.run/docs/configuration/#rotating-the-signing-secret); accept the request if any `v1=` value matches. Compare in constant time. 4. Reject the request if the timestamp is more than 5 minutes from your clock, so a captured request can’t be replayed later. The SDKs use the same tolerance. The timestamp is the time of the attempt, so every retry has a fresh one and a new signature. Test vector: the secret `secret`, the timestamp `1790000000` and the body `{"user_id":"123"}` give the signature above. Check it from a shell:
```sh
printf '%s' '1790000000.{"user_id":"123"}' | openssl dgst -sha256 -hmac secret
```
In Go:
```go
func verify(secret []byte, r *http.Request, body []byte) bool {
ts, err := strconv.ParseInt(r.Header.Get("Workly-Timestamp"), 10, 64)
if err != nil || time.Since(time.Unix(ts, 0)).Abs() > 5*time.Minute {
return false
}
mac := hmac.New(sha256.New, secret)
fmt.Fprintf(mac, "%d.", ts)
mac.Write(body)
want := mac.Sum(nil)
for _, part := range strings.Split(r.Header.Get("Workly-Signature"), ",") {
sig, ok := strings.CutPrefix(strings.TrimSpace(part), "v1=")
got, err := hex.DecodeString(sig)
if ok && err == nil && hmac.Equal(got, want) {
return true
}
}
return false
}
```
In Ruby (3.0 or later):
```ruby
require "openssl"
def verify(secret, headers, body)
ts = headers["Workly-Timestamp"].to_s
return false unless ts.match?(/\A\d+\z/) && (Time.now.to_i - ts.to_i).abs <= 300
want = "v1=" + OpenSSL::HMAC.hexdigest("SHA256", secret, "#{ts}.#{body}")
headers["Workly-Signature"].to_s.split(",").any? { |sig| OpenSSL.secure_compare(sig.strip, want) }
end
```
Answer a request that doesn’t verify with a 401. Workly doesn’t retry 4xx responses, so a wrong secret shows up at once as failed tasks with a 401, instead of an hour of retries.
# Schedules
> Call an HTTP endpoint on a cron timetable, in your timezone, with retries, signed requests, run history and a run-now button: cron for HTTP endpoints.
A schedule calls an HTTP endpoint on a cron timetable: a nightly report, a sync every 15 minutes, a weekly cleanup. Each call is a **run**, delivered, signed and retried like a task, with every attempt recorded. It’s cron for HTTP endpoints, with a history and a “run now” button. Schedules are separate from queues: a schedule has its own settings (retries, timeout, what to do when runs overlap) and history, and no queue’s limits apply to it. Its only tasks are [callbacks](https://workly.run/docs/callbacks/), if you ask for them.
```text
Queue → Task → Attempt
Schedule → Run → Attempt
```
## Create a schedule
```sh
curl -X PUT localhost:7337/v1/schedules/nightly-report -d '{
"cron": "0 3 * * *",
"timezone": "Europe/Stockholm",
"url": "https://api.example.com/reports/nightly",
"payload": {"kind": "daily"}
}'
```
`PUT` is declarative, like queues: the schedule ends up with exactly the settings given, and omitted ones go back to their defaults. The response includes `next_runs`, the next five times it will run, so you can check the expression. From code:
```python
client.schedules.put(
"nightly-report",
cron="0 3 * * *",
timezone="Europe/Stockholm",
url="https://api.example.com/reports/nightly",
payload={"kind": "daily"},
)
```
```ts
await workly.schedules.put('nightly-report', {
cron: '0 3 * * *',
timezone: 'Europe/Stockholm',
url: 'https://api.example.com/reports/nightly',
payload: { kind: 'daily' },
});
```
Or from the CLI:
```sh
workly schedules put nightly-report https://api.example.com/reports/nightly \
--cron '0 3 * * *' --timezone Europe/Stockholm -d '{"kind": "daily"}'
```
Or keep it in a `workly.yaml` and deploy it with `workly deploy` ([configuration as code](https://workly.run/docs/deploy/)):
```yaml
version: 1
schedules:
nightly-report:
cron: "0 3 * * *"
timezone: Europe/Stockholm
url: https://api.example.com/reports/nightly
payload: {kind: daily}
```
## Cron expressions Five fields: minute, hour, day of month, month and day of week. | Expression | Runs | | ----------------- | ------------------------------------ | | `*/15 * * * *` | every 15 minutes | | `0 * * * *` | every hour, on the hour | | `0 3 * * *` | every day at 03:00 | | `0 9 * * mon-fri` | weekdays at 09:00 | | `30 4 * * sun` | Sundays at 04:30 | | `0 0 1 * *` | the first of every month at midnight | Fields take `*`, numbers, lists (`1,15`), ranges (`1-5`), steps (`*/10`, `0-30/5`) and three-letter names for months and days (`jan`, `mon`). Day of week runs from 0 (Sunday) to 6, and 7 is Sunday too. The macros `@hourly`, `@daily`, `@weekly`, `@monthly` and `@yearly` work as usual. When both day fields are restricted, as in `0 0 13 * fri`, a day matches if either does (the 13th, or any Friday), as in classic cron. Seconds aren’t supported; the shortest interval is a minute. The expression is read in the schedule’s `timezone`, an IANA name such as `Europe/Stockholm` (default `UTC`). Across daylight saving changes, a time in the skipped hour runs right after it (02:30 runs at 03:00), and a time in the repeated hour runs once. ## Settings | Setting | Default | | | ------------------------------------------------------------------ | --------------- | ------------------------------------------------------------------------------------------------- | | `cron` | required | When to run. | | `timezone` | `UTC` | | | `url` | required | The endpoint to call. | | `workflow` | none | Instead of `url`: start a run of this [workflow](https://workly.run/docs/workflows/), with `payload` as its input. | | `method`, `headers`, `payload` or `body`, `content_type` | `POST` | The request, as for [tasks](https://workly.run/docs/quickstart/). | | `timeout_seconds` | `30` | How long to wait for a response. | | `max_attempts` | `3` | Attempts per run before it fails. | | `min_backoff_seconds`, `max_backoff_seconds`, `backoff_multiplier` | `5`, `300`, `2` | Backoff between retries, as for queues. | | `retry_on_4xx` | `false` | Retry 4xx responses other than 408 and 429. | | `overlap` | `skip` | What to do when a run is due while the previous one is still going. | | `missed` | `skip` | What to do about times missed while the server was down. | | `on_complete` | none | A [callback](https://workly.run/docs/callbacks/), `{url, headers}`, called with a summary each time a run finishes. | With `workflow`, a schedule sends no request of its own. Each run starts a workflow run, with the schedule run’s ID as its idempotency key, and succeeds once it has started; a workflow that doesn’t exist yet is retried like a failed request. The schedule run links to the workflow run (`workflow_run_id`), and `workflow_run_status` says whether it’s still going or how it ended. For `overlap`, the schedule counts as running until the workflow run finishes, so with `skip` a slow workflow never has two scheduled runs at once. `method`, `headers`, `body` and `content_type` don’t apply. Retries are fewer and shorter than a queue’s by default, so a failing run gives up before a frequent schedule’s next one. **Overlap.** With `skip`, a run that comes up while the previous run is still pending or running (or, for a schedule with `workflow`, while the workflow run it started is still going) is recorded as `skipped`, with `skip_reason: overlap`, and not delivered. With `allow`, it runs alongside. Runs you start by hand always run. **Missed runs.** A run is missed if no server could start it within a minute of its time. `skip` records the most recent missed time as a `skipped` run with `skip_reason: missed` and carries on; `catch_up` runs it once. Either way, a server that was down for a day doesn’t make up every run it missed. ## Runs Each run has a state: `pending`, `running`, `succeeded`, `failed`, `cancelled` or `skipped`. A run is created once per scheduled time, even if the server fails over at that moment; like tasks, its delivery is at least once.
```sh
workly runs ls --schedule nightly-report # newest first
workly runs ls --state failed # failed runs of every schedule
workly runs get # the run and its attempts
```
In the API, `GET /v1/runs` takes the same filters: `?schedule=nightly-report`, `?state=failed`. A run’s `started_at` is when its first attempt started, and `last_duration_ms` how long its latest attempt took. A run keeps the request as it was when the run was created, so it shows what was sent even after the schedule changes. Retries use the schedule’s current retry settings. * **Run now:** `POST /v1/schedules/{name}/trigger` (or `workly schedules trigger`, or the dashboard’s Run now) creates a run with `trigger: manual` and delivers it at once, even while the schedule is paused. It doesn’t change when the schedule next runs. * **Retry a run:** `POST /v1/runs/{id}/retry` delivers a failed, cancelled, succeeded or skipped run again, with a fresh set of attempts. Retrying a skipped run delivers the time it skipped. * **Pause:** `POST /v1/schedules/{name}/pause` stops new runs; runs already pending finish. A paused schedule has `paused_at`, no `next_run_at` and empty `next_runs`; `next_runs_if_resumed` are the times it would run if resumed now. Times that pass while it’s paused aren’t run. * **Delete:** deleting a schedule deletes its runs. Finished runs are kept for `WORKLY_RUN_RETENTION` (by default `WORKLY_RETENTION`, 7 days), except each schedule’s latest run, which is always kept, so a monthly job still shows how it last went. ## Handling a run Your endpoint gets the request with these headers: | Header | | | -------------------------------------- | --------------------------------------------------------------------------------------------------------------- | | `Workly-Run-Id` | The run, the same on every attempt. Use it to deduplicate. | | `Workly-Schedule` | The schedule’s name. | | `Workly-Scheduled-For` | The scheduled time (RFC 3339). Use it, not the clock, to decide which day or hour a late or retried run is for. | | `Workly-Attempt` | The attempt number. | | `Workly-Signature`, `Workly-Timestamp` | The signature, as for tasks: verify it with the SDKs’ `verify_signature` / `verifySignature`. | Runs don’t carry `Workly-Task-Id` or `Workly-Queue`, so one handler can tell a scheduled call from a task if it receives both. ## Monitoring The dashboard’s Schedules page shows each schedule’s next run, last run and recent runs; a schedule’s page has its upcoming times and every run with its attempts. Over [MCP](https://workly.run/docs/mcp/), `get_schedule` answers “why didn’t the 03:00 report run?” from the skipped and failed runs. For alerts, `workly_schedule_last_success_timestamp_seconds` tells you when each schedule last succeeded; alert when it’s older than the schedule’s period. See [metrics](https://workly.run/docs/metrics/).
# Task groups
> Fan out tasks and get one callback when they've all finished, with counts of what succeeded and failed: fan-out and fan-in for an HTTP task queue.
A task group tracks a set of tasks and calls you back once all of them have finished. Enqueue a task per item and get one callback with how they went: fan-out and fan-in, without writing the bookkeeping yourself.
```text
create group → enqueue tasks into it → close it → … tasks finish … → callback task
```
A group is one execution, not a reusable definition: a nightly job creates a new group each night, and can give them all the same `name`. A group has no dependencies between its tasks, no chains and no nested groups. To run steps in order with a group in the middle, use a [workflow](https://workly.run/docs/workflows/): a step can hand its completion to a group with `Workly-Wait-For`, and the next step starts once the group has succeeded. ## Example: refresh stock prices, then rebuild the views Every night, a [schedule](https://workly.run/docs/schedules/) calls `/jobs/start-price-update`. That endpoint enqueues one task per company into a group, and the group’s callback rebuilds the views that depend on the prices once every update has finished. In Python, with Flask:
```python
from flask import Flask, abort, request
from workly import Callback, SignatureVerificationError, Workly, verify_signature
app = Flask(__name__)
workly = Workly()
API = "https://api.example.com"
SYMBOLS = ["NET", "DDOG", "SNOW", ...] # the companies to update
def verified() -> dict:
try:
verify_signature(request.get_data(), request.headers)
except SignatureVerificationError:
abort(401)
return request.get_json()
@app.post("/jobs/start-price-update")
def start_price_update():
verified()
day = request.headers["Workly-Scheduled-For"][:10]
# Creates the group, enqueues the tasks into it in batches and closes it. The
# idempotency key makes a retried run finish the same group rather than start another.
workly.groups.enqueue(
"stock-prices",
({"url": f"{API}/jobs/update-stock-price", "payload": {"symbol": s}} for s in SYMBOLS),
name="daily-stock-prices",
idempotency_key=f"stock-prices-{day}",
on_complete=Callback(url=f"{API}/jobs/refresh-views"),
)
return "", 204
@app.post("/jobs/update-stock-price")
def update_stock_price():
symbol = verified()["symbol"]
... # fetch and store the price
return "", 204
@app.post("/jobs/refresh-views")
def refresh_views():
summary = verified()
# {"group_id": "grp_…", "name": "daily-stock-prices", "status": "succeeded",
# "total": 500, "succeeded": 500, "failed": 0, "cancelled": 0}
if summary["status"] == "succeeded":
... # refresh the materialized views
else:
... # tell someone: summary["failed"] updates failed
return "", 204
```
The same in TypeScript:
```ts
import { Workly } from '@worklydev/sdk';
const workly = new Workly();
const API = 'https://api.example.com';
export async function startPriceUpdate(day: string, symbols: string[]) {
await workly.groups.enqueue(
'stock-prices',
symbols.map((symbol) => ({ url: `${API}/jobs/update-stock-price`, payload: { symbol } })),
{
name: 'daily-stock-prices',
idempotencyKey: `stock-prices-${day}`,
onComplete: { url: `${API}/jobs/refresh-views` },
},
);
}
```
`groups.enqueue` does three steps in one call: create the group, enqueue the tasks into it (in batches of 100, the most one request takes) and close it. With the group’s `idempotency_key` it’s safe to retry. The group is found again rather than created; a task without its own `idempotency_key` gets one from the group and its position, so none is counted twice; and a group that’s already closed is returned as it is. It needs SDK 0.4.3 or later. With an older SDK, or over HTTP, do the three steps yourself:
```sh
curl -X POST localhost:7337/v1/groups -d '{
"name": "daily-stock-prices",
"on_complete": {"url": "https://api.example.com/jobs/refresh-views"}
}'
# {"id": "grp_01J9…", "status": "open", "total": 0, …}
curl -X POST localhost:7337/v1/queues/stock-prices/tasks -d '{
"url": "https://api.example.com/jobs/update-stock-price",
"payload": {"symbol": "NET"},
"group_id": "grp_01J9…"
}'
curl -X POST localhost:7337/v1/groups/grp_01J9…/close
curl localhost:7337/v1/groups/grp_01J9…
```
Or from the CLI:
```sh
group=$(workly groups create --name daily-stock-prices \
--callback-url https://api.example.com/jobs/refresh-views)
workly tasks enqueue stock-prices https://api.example.com/jobs/update-stock-price -d '{"symbol": "NET"}' --group $group
workly groups close $group
workly groups get $group
```
To try it locally, point the URLs at `http://localhost:7337/_dev/echo`. ## Lifecycle | Status | | | ----------- | -------------------------------------------------------------- | | `open` | Accepting tasks. Its tasks run as soon as they’re enqueued. | | `running` | Closed, with tasks still pending or running. | | `succeeded` | Every task succeeded. | | `failed` | Every task finished, and at least one failed or was cancelled. | * **Close the group once you’ve enqueued everything.** It can’t complete while it’s open, since more tasks may come, so you don’t need to know the number of tasks up front. Closing a closed group does nothing, so it’s safe to retry. * A group whose tasks have all finished when it’s closed completes at once, and so does a closed group with no tasks. * Tasks in a group are ordinary tasks, with their queue’s retries, limits and history. A task waiting to retry counts as remaining. A task that fails for good doesn’t stop the others. * `succeeded` and `failed` are final. Once a group has completed, nothing changes it. ## The callback `on_complete` is a [callback](https://workly.run/docs/callbacks/): a URL and optional headers, as for tasks and schedules. When the group completes, Workly enqueues a `POST` to the URL with the group’s summary as its JSON body:
```json
{
"group_id": "grp_01J9ZK3M5Q8R2T4V6W8X0Y2Z4A",
"name": "daily-stock-prices",
"status": "failed",
"total": 500,
"succeeded": 498,
"failed": 1,
"cancelled": 1
}
```
It’s enqueued exactly once: the group completes in the same database transaction that finishes its last task (or closes it), and that transaction enqueues the callback. A crash can’t lose it, and two completions racing can’t enqueue two. From there it’s an ordinary task in the [`_callbacks` queue](https://workly.run/docs/callbacks/#the-_callbacks-queue): signed, retried and recorded, with the group’s `callback_task_id` pointing to it. Delivery is at least once, so make the handler safe to run twice; `group_id` identifies the group. Headers in `on_complete` are sent with it. The callback fires whether the group succeeded or failed: check `status`. A group without `on_complete` still completes, and you can poll it instead. The body carries counts, not the tasks’ results. To see which tasks failed, list them: `GET /v1/tasks?group_id=grp_…&state=failed`. ## Retries, cancellations and deleted queues * **Retrying a task while its group is open or running** counts it as remaining again, so the group waits for the retry. That’s how to fix a failure before the group completes. * **Retrying a task after its group completed** delivers the task again but doesn’t change the group. To redo the work as a group, create a new group. * **Cancelling** a task counts it as cancelled, which makes the group `failed` once it completes. * **Deleting a queue** takes its tasks out of their groups (`total` goes down), and a group may complete without them. ## Adding tasks safely An enqueue checks that the group is open in the same transaction, so a task can’t slip in after the group closes and be missed. Enqueueing into a closed or completed group fails with `group_closed` (409), and into a missing one with `group_not_found` (404). In a batch, all the tasks are enqueued or none. Make it safe to retry (`groups.enqueue` in the SDKs does all of this): * Give tasks an `idempotency_key`. A task enqueued again with the same key isn’t counted twice, so you can resend a batch after a timeout. * Give the group an `idempotency_key`. Creating it again returns the same group, so a retried request (or schedule run, as in the example) continues it rather than starting a second. ## Watching groups The dashboard’s **Groups** page lists groups with their progress. A group’s page shows its counts, its callback and how it was delivered, and its tasks by state. `GET /v1/groups` lists groups newest first, filtered by `status` and `name`; `GET /v1/groups/{id}` returns one with `total`, `succeeded`, `failed`, `cancelled` and `remaining`. Agents get `list_groups` and `get_group` over [MCP](https://workly.run/docs/mcp/), and `list_tasks` takes a `group_id`. Completed groups are deleted after `WORKLY_RETENTION`, like finished tasks. Their tasks may go first; the group’s counts stay accurate without them.
# Workflows
> Run HTTP steps in dependency order, each delivered and retried as a task, with steps that wait for a task group: DAG workflows for an HTTP task queue.
A workflow is a set of HTTP steps that run in dependency order: a step is sent once the steps it depends on have succeeded. Each step is an ordinary [task](https://workly.run/docs/concepts/), delivered, retried and signed like any other: a workflow adds ordering to tasks, not a new kind of worker.
```text
prices ─┐
├─→ analytics ─┐
fx ─────┘ ├─→ notify
news ──────────────────┘
```
A step can also hand its completion to a [task group](https://workly.run/docs/groups/): its endpoint starts the work, answers `202` with `Workly-Wait-For: group/`, and the step finishes when the group does. That is how one step fans out over hundreds of tasks and the next step waits for all of them. A workflow is a static DAG declared up front: no conditions, loops, outputs passed between steps or nested workflows, and Workly never runs your code. If a step needs data from an earlier one, read it from where that step stored it. ## Define a workflow Declare workflows in [`workly.yaml`](https://workly.run/docs/deploy/) and apply them with `workly deploy`:
```yaml
version: 1
workflows:
daily-refresh:
max_duration_seconds: 7200
steps:
- name: prices
url: https://api.example.com/jobs/start-price-update
- name: fx
url: https://api.example.com/jobs/refresh-fx
- name: news
url: https://api.example.com/jobs/import-news
queue: imports
- name: analytics
url: https://api.example.com/jobs/rebuild-analytics
depends_on: [prices, fx]
- name: notify
url: https://api.example.com/jobs/notify
depends_on: [analytics, news]
payload: {channel: "#data"}
schedules:
nightly-refresh:
cron: "0 3 * * *"
timezone: Europe/Stockholm
workflow: daily-refresh
```
Or with `PUT /v1/workflows/{name}` or `workflows.put` in the SDKs. Steps without `depends_on` start as soon as a run does: here `prices`, `fx` and `news` run in parallel, `analytics` once `prices` and `fx` have both succeeded, and `notify` once `analytics` and `news` have. | Step setting | Default | | | -------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------- | | `name` | required | Unique within the workflow: up to 64 lowercase letters, digits, `-` and `_`. | | `url` | required | The endpoint to call. | | `depends_on` | none | The steps that must succeed first. | | `method`, `headers`, `payload` or `body`, `content_type` | `POST` | The request, as for [tasks](https://workly.run/docs/quickstart/). Without a `payload` or `body`, the step is sent the run’s context (below). | | `queue` | `_workflows` | The queue its task goes to, for that queue’s retries, timeout, rate and concurrency limits. | | `max_attempts`, `timeout_seconds` | the queue’s | Overrides for this step. | A workflow has at most 100 steps, which must form a DAG: Workly rejects unknown or self dependencies and names the cycle in one (`steps form a cycle: a → b → a`). `max_duration_seconds` (default a day) is how long a run may take before it fails. Steps go to the `_workflows` queue unless they name another. Like [`_callbacks`](https://workly.run/docs/callbacks/), it’s created when first needed and configured like any queue, but you can’t enqueue into it. A step that names one of your queues shares that queue’s limits with its other tasks: use this to keep a workflow from overloading an endpoint that other work also calls. Changing a workflow doesn’t touch runs already going: a run keeps the definition (and `version`) it started with. ## Run a workflow
```sh
workly workflows trigger daily-refresh --input '{"date": "2026-11-13"}'
```
```python
run = workly.workflows.trigger("daily-refresh", input={"date": "2026-11-13"}, idempotency_key="2026-11-13")
```
```typescript
const run = await workly.workflows.trigger('daily-refresh', { input: { date: '2026-11-13' }, idempotencyKey: '2026-11-13' });
```
Or with the dashboard’s Run now, or a [schedule](https://workly.run/docs/schedules/) with `workflow:` instead of `url:`, whose `payload` becomes each run’s input. An `idempotency_key` already used for a run of the workflow returns that run and starts nothing. A schedule uses its run’s ID as the key, so a retried schedule run never starts a second workflow run, and with `overlap: skip` (the default) it skips a slot while its last workflow run is still going. ## What a step receives Each step’s task is delivered with the usual [headers](https://workly.run/docs/concepts/) plus: | Header | | | ----------------------------- | ------------------------------------------------------------------------------------------- | | `Workly-Workflow` | The workflow’s name. | | `Workly-Workflow-Run-Id` | The run, `wfr_…`. | | `Workly-Workflow-Step` | The step’s name. | | `Workly-Workflow-Step-Id` | The step, `wfs_…`: the same on every delivery of the step, so use it as an idempotency key. | | `Workly-Workflow-Run-Attempt` | `1`, then one more each time the run is [retried](#retry-a-run). | A step without a body of its own is sent the run’s context as JSON:
```json
{
"workflow_run_id": "wfr_01M7AH8EP90ZQ4E9228337GGVR",
"workflow": "daily-refresh",
"step": "analytics",
"step_id": "wfs_01M7AH8EPATE4EPK37FWTA5TBQ",
"run_attempt": 1,
"input": {"date": "2026-11-13"}
}
```
A 2xx response succeeds the step. Anything else is retried with its queue’s policy, and a step whose task fails (out of attempts, or a 4xx that isn’t retried) fails the run. ## Waiting for a group To hand a step’s completion to a group, answer 2xx (typically `202`) with `Workly-Wait-For: group/`. The step stays `waiting` until the group completes, then succeeds if every task in the group succeeded, or fails, failing the run, if any failed or was cancelled. Close the group once its tasks are enqueued, or it never completes.
```python
@app.post("/jobs/start-price-update")
def start_price_update():
try:
verify_signature(request.get_data(), request.headers)
except SignatureVerificationError:
abort(401) # a 4xx isn't retried
# The same on every delivery of this step in this run attempt: a redelivered
# request reuses its group, and a retried run starts a fresh one.
key = f'{request.headers["Workly-Workflow-Step-Id"]}-{request.headers["Workly-Workflow-Run-Attempt"]}'
# Creates the group, enqueues the tasks into it and closes it; safe to retry.
group = workly.groups.enqueue(
"stock-prices",
({"url": f"{API}/jobs/update-stock-price", "payload": {"symbol": s}} for s in SYMBOLS),
name="daily-stock-prices",
idempotency_key=key,
)
return "", 202, {"Workly-Wait-For": f"group/{group.id}"}
```
The header names exactly one group in the workspace. A header Workly can’t use (another resource type, two groups, a group that doesn’t exist) fails the step at once, with the reason, rather than retrying. If the group has already completed, the step finishes at once. This is safe across restarts and replicas: the step’s completion is recorded in the same database transaction as the group’s, so a step can’t miss its group or finish twice. ## When a step fails A run stops at its first failed step: * steps still blocked are `skipped`, * step tasks not yet delivered are cancelled and their steps `skipped`, * steps waiting for a group are `skipped`; the group itself goes on, and its tasks still run, * requests already in flight finish, and their steps record how they went, but start nothing. The run is `failed`, with the step’s error. A run still going after `max_duration_seconds` fails the same way, except that its waiting steps are `failed`, since they didn’t finish in time. Cancelling a run (`workly workflows cancel`, or the dashboard) skips what hasn’t started. Step tasks can’t be retried one at a time: [retry the run](#retry-a-run) instead. A step’s task names its step and run (`workflow_step` in the API, and a link on its dashboard page), so a failure found in a queue leads back to its run. ## Retry a run Once you’ve fixed what failed, retry the run rather than starting over: the dashboard’s Retry run, `workly workflows retry ID`, `workflow_runs.retry(id)` in the SDKs or `retry_workflow_run` over [MCP](https://workly.run/docs/mcp/). A failed or cancelled run goes back to `running`, with the definition and input it started with, even if the workflow has changed since: * steps that succeeded stay succeeded, and aren’t sent again, * failed and skipped steps run again once their dependencies have succeeded, each on the task it had, with a fresh set of attempts, * a step that was waiting for a group follows the group: it succeeds if the group has succeeded since, waits again if it is still going, and runs again if it failed. The run’s `attempt` goes up by one, and its `max_duration_seconds` starts over. Steps get the attempt as `Workly-Workflow-Run-Attempt`, so work keyed on the step ID and the attempt, like the group above, is redone on a retry rather than reusing what failed. To run a changed definition, start a new run. ## Statuses | Step | | | --------------------- | ------------------------------------------------------------- | | `blocked` | Waiting for the steps it depends on. | | `pending` | Its task is queued, or waiting to retry. | | `running` | Its request is being delivered. | | `waiting` | Its request succeeded, and it waits for the group it named. | | `succeeded`, `failed` | Finished. | | `skipped` | Never ran, or was waiting for a group: the run stopped first. | A run is `running`, then `succeeded`, `failed` or `cancelled`. `GET /v1/workflow-runs/{id}` (`workly workflows run ID`, `get_workflow_run` over [MCP](https://workly.run/docs/mcp/)) shows each step with its task, its last status and error, what it is blocked by or waiting for, and how long each phase took. ## Why hasn’t my run finished? Open the run in the dashboard, or `workly workflows run ID`. Each step says why it is where it is: * **Blocked:** the steps named in “waiting for” haven’t succeeded yet; look at those. * **Pending with attempts:** its endpoint is failing and the task is retrying; the task page has every attempt’s response. * **Pending without attempts:** its queue is paused, at its concurrency limit or behind; look at the queue. * **Waiting for a group:** the group’s tasks are still going, or the group was never closed. The group page shows which. ## Delivery is at least once Steps are tasks, so a step’s request can arrive more than once, for example when a server stops mid-delivery. Workly starts each step’s task once per run, but your endpoints should still treat `Workly-Workflow-Step-Id` as an idempotency key, as the example above does with its group. ## Retention Finished runs are deleted after `WORKLY_RUN_RETENTION`, like schedule runs ([configuration](https://workly.run/docs/configuration/)); a workflow’s latest run is always kept. Step tasks are deleted with other tasks, so an old run’s steps may no longer link to theirs.
# Callbacks
> Have Workly POST a summary to your endpoint each time a task or scheduled run succeeds, fails or is cancelled: callbacks for an HTTP task queue.
A callback tells your app how something went once it has finished: a task, a schedule’s run, or a [task group](https://workly.run/docs/groups/). Give it an `on_complete` URL and, each time it finishes, Workly sends a `POST` there with a JSON summary: whether it succeeded, how many attempts it took, and the last status code and error.
```text
task or run finishes → callback task in _callbacks → POST to your on_complete URL
```
Use callbacks to react to a result without polling: mark an export as done, email the user that their report failed, or start the next step once this one has finished. To be told when something needs a human, use [alerts](https://workly.run/docs/alerts/) instead. ## On a task
```python
from workly import Callback, Workly
client = Workly()
client.tasks.enqueue(
"exports",
url="https://api.example.com/jobs/export",
payload={"export_id": 42},
idempotency_key="export-42",
on_complete=Callback(url="https://api.example.com/jobs/export-done"),
)
```
```ts
await workly.tasks.enqueue('exports', {
url: 'https://api.example.com/jobs/export',
payload: { exportId: 42 },
idempotencyKey: 'export-42',
onComplete: { url: 'https://api.example.com/jobs/export-done' },
});
```
```sh
curl -X POST localhost:7337/v1/queues/exports/tasks -d '{
"url": "https://api.example.com/jobs/export",
"payload": {"export_id": 42},
"idempotency_key": "export-42",
"on_complete": {"url": "https://api.example.com/jobs/export-done"}
}'
```
The CLI takes `--on-complete URL`, and `enqueue_task` over [MCP](https://workly.run/docs/mcp/) takes `on_complete`. When the task succeeds, fails for good or is cancelled, the callback gets:
```json
{
"task_id": "task_01J9ZK3M5Q8R2T4V6W8X0Y2Z4A",
"queue": "exports",
"state": "failed",
"attempt_count": 5,
"last_status": 500,
"last_outcome": "http_error",
"last_error": "HTTP 500 Internal Server Error",
"idempotency_key": "export-42",
"finished_at": "2026-11-12T10:15:02.114Z"
}
```
`last_error` is present only when the last attempt failed, so it’s absent when the task succeeded. `last_status`, `last_outcome` and `last_error` are all absent when there was no attempt (a task cancelled before it ran), and `idempotency_key` and `group_id` when the task has none. The body carries the outcome, not the response: to see what the endpoint answered, `GET /v1/tasks/{task_id}/attempts`. ## On a schedule A schedule’s `on_complete` is called for each of its runs: workly.yaml
```yaml
version: 1
schedules:
nightly-report:
cron: "0 3 * * *"
timezone: Europe/Stockholm
url: https://api.example.com/reports/nightly
on_complete:
url: https://api.example.com/reports/nightly-done
```
It can also be set with `PUT /v1/schedules/{name}`, `client.schedules.put(…, on_complete=…)`, `workly schedules put --on-complete`, `update_schedule` over MCP, or on the schedule’s page in the dashboard. The body:
```json
{
"run_id": "run_01J9ZK3M5Q8R2T4V6W8X0Y2Z4A",
"schedule": "nightly-report",
"state": "succeeded",
"trigger": "schedule",
"scheduled_for": "2026-11-12T02:00:00Z",
"attempt_count": 1,
"last_status": 200,
"last_outcome": "success",
"finished_at": "2026-11-12T02:00:01.873Z"
}
```
The `last_*` fields follow the same rules as a task’s: a failed run adds `last_error` (such as `"last_error": "HTTP 500 Internal Server Error"`), a run that succeeded has none, and a run cancelled before its first attempt has no `last_*` fields at all. Skipped runs (an overlap, or a slot missed while the server was down) were never delivered and don’t call back. The schedule’s current `on_complete` applies when a run finishes, so a change takes effect for runs already in progress. ## When a callback fires * **Once each time it finishes.** A task or run calls back when it succeeds, when its last attempt fails or it gets a response Workly doesn’t retry, and when it is cancelled. Attempts that will be retried don’t. * **Again after a retry by hand.** A retried task or run gets a fresh set of attempts and calls back again when it finishes, with the new outcome, so a fix-and-retry tells your app it finally worked. * **Not for deleted tasks.** Deleting a queue deletes its tasks without calling back. * **Groups call back once.** A [group’s](https://workly.run/docs/groups/) callback fires when the group completes, with its counts, and never again. The callback is enqueued in the same database transaction that finishes the task or run, so a crash can’t lose one, and a task cancelled mid-attempt calls back once, when it’s cancelled. ## The `_callbacks` queue Every callback is a task in `_callbacks`, a queue Workly creates when it first needs one. Only Workly enqueues into it, and callbacks never have callbacks of their own. Otherwise a callback is an ordinary task: signed, retried with the queue’s backoff, and recorded with every attempt. The task, run or group links to it (`callback_task_id`, the latest one), and the dashboard shows it next to them. A schedule’s runs also carry that task’s state (`callback_state`), so a list of runs says which callbacks were delivered without fetching each one. `_callbacks` has the default settings. To give callbacks more retries or a longer timeout, configure it like any queue, e.g. in [`workly.yaml`](https://workly.run/docs/deploy/):
```yaml
queues:
_callbacks:
max_attempts: 10
timeout_seconds: 300
```
If it has been deleted, it’s created again with default settings, so no callback is dropped. ## Handling a callback A callback is delivered like any task, so the handler is an ordinary endpoint: [verify the signature](https://workly.run/docs/concepts/#signing), read the summary, and answer 2xx. Headers in `on_complete` are sent with it, e.g. to route it or authenticate it:
```json
"on_complete": {"url": "https://api.example.com/jobs/done", "headers": {"X-Job": "export"}}
```
Delivery is at least once, so make the handler safe to run twice. The callback’s `Workly-Task-Id` stays the same across its retries; the summary’s `task_id` (or `run_id`) with `finished_at` identifies which finish it is about, since a task retried by hand calls back once per finish.
# Alerts
> Get an email or a Slack message when Workly tasks or schedule runs fail, so you don't have to watch the dashboard.
Workly can tell you when tasks or schedule runs fail, so you don’t have to watch the dashboard. Alerts go to **channels**: email addresses and Slack incoming webhooks. Add them on the dashboard’s Alerts page or through the API. ## What you get A task or run counts as failed once it has used up its attempts, or got a response Workly doesn’t retry. Retries that later succeed, and cancelled tasks, never alert. * Within about a minute of a failure, every channel gets an alert. It names the queue or schedule, how many failed and the most common errors, and links to the failures in the dashboard. * After that, a queue or schedule gets **at most one alert every 30 minutes**, covering everything that failed since its previous alert. A queue that keeps failing reminds you every half hour instead of once per task. * Failures in several queues and schedules at once arrive as one alert. Alerts cover failures from the moment the first channel is added; earlier ones aren’t news. A queue with [auto-pause](https://workly.run/docs/concepts/#auto-pause) also alerts at once when it pauses itself, with the error that paused it, and when it resumes, with how long it was paused. These two don’t wait for the 30 minutes. ## Slack In Slack, create an [incoming webhook](https://api.slack.com/messaging/webhooks) for the channel alerts should go to, then add its URL:
```sh
curl -X POST localhost:7337/v1/alert-channels -d '{
"type": "slack",
"target": "https://hooks.slack.com/services/T000/B000/XXXX"
}'
```
The webhook URL is a secret: the API and the dashboard only show its end. ## Email
```sh
curl -X POST localhost:7337/v1/alert-channels -d '{"type": "email", "target": "oncall@example.com"}'
```
A self-hosted server sends email through your SMTP server, set with `WORKLY_SMTP_URL` and `WORKLY_SMTP_FROM` (see [Configuration](https://workly.run/docs/configuration/)). Without them, `email_enabled` is `false` in `GET /v1/alert-channels` and adding an email channel fails with `email_unavailable`. Slack channels need no setup. On workly.run, email alerts go to members of the workspace. ## Check a channel Send a test alert to make sure a channel works:
```sh
curl -X POST localhost:7337/v1/alert-channels/ach_01J9ZK3M5Q8R2T4V6W8X0Y2Z4A/test
```
If it can’t be sent, the error says why (`alert_failed`), e.g. Slack’s answer for a removed webhook. From code:
```python
channel = client.alerts.create("slack", "https://hooks.slack.com/services/T000/B000/XXXX")
client.alerts.test(channel.id)
```
```ts
const channel = await workly.alerts.create('slack', 'https://hooks.slack.com/services/T000/B000/XXXX');
await workly.alerts.test(channel.id);
```
## Links Alerts link to the dashboard when the server knows its address. Set `WORKLY_PUBLIC_URL` to where people open Workly, e.g. `https://workly.example.com`. `workly dev` links to `http://localhost:7337`. ## Delivery Alerts are sent by the replica that delivers tasks, so a cluster sends each alert once. If every channel fails to take an alert (Slack is down, say), the next check, a minute later, tries again. A channel can get an alert twice if the server stops just after sending it.
# Configuration as code
> Keep Workly queues, workflows and schedules in a workly.yaml next to your code: validate it in CI, see the plan, and apply it with workly deploy.
Keep your queues, workflows and schedules in a `workly.yaml` next to your code, and deploy them with one command:
```sh
workly deploy
```
`workly deploy` compares the file with the server, shows what it will change, and creates and updates queues, workflows and schedules until the server matches. Running it again changes nothing, so it can run on every deploy of your app. ## Example A nightly job that refreshes stock prices (see [task groups](https://workly.run/docs/groups/) for the job itself) and a morning refresh of financial data:
```yaml
version: 1
queues:
stock-prices:
max_concurrency: 10
rate_per_second: 2 # the price API allows 120 calls a minute
max_attempts: 3
financial-data:
max_concurrency: 5
timeout_seconds: 120
schedules:
daily-stock-prices:
cron: "0 22 * * 1-5"
timezone: Europe/Stockholm
url: ${API_URL}/jobs/start-price-update
headers:
Authorization: Bearer ${JOBS_TOKEN}
refresh-financial-data:
cron: "0 6 * * *"
timezone: Europe/Stockholm
url: ${API_URL}/jobs/refresh-financial-data
payload:
full: true
```
Check it, see the plan, deploy it:
```console
$ workly validate
workly.yaml is valid: 2 queues, 2 schedules.
$ workly plan
Plan for workly.yaml on https://api.workly.run:
+ create queue stock-prices
~ update queue financial-data
max_concurrency: 3 → 5
timeout_seconds: 30 → 120
+ create schedule daily-stock-prices
~ update schedule refresh-financial-data
cron: 0 5 * * * → 0 6 * * *
headers.Authorization: changed
2 to create, 2 to update, 0 unchanged.
$ workly deploy
…
Apply these changes to https://api.workly.run? [y/N] y
Created queue stock-prices
Updated queue financial-data
Created schedule daily-stock-prices
Updated schedule refresh-financial-data
Deployed workly.yaml: 2 created, 2 updated.
```
## Commands | Command | | | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `workly validate` | Checks the file without contacting the server: field names and types, cron expressions, timezones, URLs, headers, and that every `${NAME}` is set. It runs the server’s own validation, so a file that passes is one the server accepts. | | `workly plan` | Shows what `deploy` would create, update and (with [`prune`](#pruning)) delete, setting by setting, and which `${NAME:-default}` values took their default. Changes nothing. | | `workly deploy` | Shows the plan, asks, and applies it. `--yes` (`-y`) skips the question, as CI needs; without a terminal to ask on, it’s required. | All three read `workly.yaml` in the current directory, or the file given with `--file` (`-f`). `plan` and `deploy` connect like the other commands: `--url` or `WORKLY_URL`, and `--api-key` or `WORKLY_API_KEY`. They exit with a non-zero status on an invalid file or a failed change; a deploy that fails partway reports what it had applied, and running it again finishes the job. ## The file | Key | | | ----------- | ------------------------------------------------------------------------------------------------------ | | `version` | Required. `1`. | | `prune` | `true` to delete what the file leaves out. See [Pruning](#pruning). Default `false`. | | `queues` | Queue names to their settings. | | `workflows` | Workflow names to their `steps` (a list) and `max_duration_seconds`. | | `schedules` | Schedule names to their settings. | | `x-…` | Ignored, so a key such as `x-defaults` can hold [anchors](#sharing-settings) for the rest of the file. | Settings are the fields of the `PUT` requests in the [API](https://workly.run/docs/api/), with the same names, defaults and limits: * **Queues:** `max_attempts`, `min_backoff_seconds`, `max_backoff_seconds`, `backoff_multiplier`, `timeout_seconds`, `retry_on_4xx`, `rate_per_second`, `max_concurrency` and `auto_pause` (`{after_failures: n}`, see [Auto-pause](https://workly.run/docs/concepts/#auto-pause)). * **Workflows:** `steps`, each with the settings in [Workflows](https://workly.run/docs/workflows/#define-a-workflow), and `max_duration_seconds`. * **Schedules:** `cron` and `url` or `workflow` (required), `timezone`, `method`, `headers`, `payload` or `body` and `content_type`, the retry settings, `overlap` and `missed` (see [Schedules](https://workly.run/docs/schedules/)). A `payload` can be any YAML, and is sent as JSON. Settings work as `PUT` does: **a setting left out of the file gets its default**, not whatever it was before. The file describes each queue and schedule completely, so a change made in the dashboard to something the file declares is undone by the next deploy. To change it for good, change the file. ### Environment variables `${NAME}` in a value is replaced with the environment variable `NAME`, so one file can serve several environments and keep secrets out of the repository. A variable that isn’t set is an error, not an empty string. An unquoted value is read again after the replacement, so `max_concurrency: ${CONCURRENCY}` is a number. `${NAME:-default}` uses `default` when `NAME` is unset or empty, as in the shell, so one file can work locally without exporting anything:
```yaml
url: ${API_URL:-http://localhost:8000}/jobs/nightly
```
`validate`, `plan` and `deploy` list every default they used, so a variable CI forgot to set shows up in the plan instead of quietly sending production schedules to `localhost`:
```console
Not set, so their defaults were used:
API_URL: "http://localhost:8000" (schedules.daily-stock-prices.url, schedules.refresh-financial-data.url)
```
Keep secrets as plain `${NAME}`, without a default, so a missing one stops the deploy. The plan never shows header values or bodies, only that they changed, since they often carry secrets. ### Sharing settings YAML anchors (`&name`), aliases (`*name`) and merge keys (`<<`) work, so queues and schedules can share settings. Top-level keys starting with `x-` are ignored, which makes them a place to keep them:
```yaml
version: 1
x-price-api: &price-api
max_concurrency: 2
rate_per_second: 1 # one API key, shared by these queues
queues:
stock-prices: *price-api
earnings-dates:
<<: *price-api
max_attempts: 10 # keys given here win over merged ones
```
## Locally `workly dev --config workly.yaml` applies the file each time the dev server starts, so a fresh database starts with your queues’ limits and there’s no separate deploy step. Restart it to apply changes. With `${NAME:-default}` in the URLs, the same file serves local and production. ## What deploying changes, and what it doesn’t * It **creates** queues, workflows and schedules the file declares that the server doesn’t have, and **updates** those whose settings differ; one that already matches isn’t touched, so its `updated_at` stays put. Workflows go before schedules, so a schedule can start one declared in the same file. A workflow run already going keeps the definition it started with. * It **doesn’t delete** anything unless the file sets [`prune: true`](#pruning). The plan lists the workflows and schedules on the server that aren’t in the file, so you notice ones you meant to remove (`workly schedules rm NAME`, `workly workflows rm NAME`). Queues aren’t listed: enqueueing creates them, so most have never been in a file. * It **never pauses or resumes**. You pause during an incident, and a deploy shouldn’t undo that: a paused queue or schedule stays paused, and the plan says so. * A section left out of the file isn’t managed by it: a file without `schedules` doesn’t look at schedules at all. An empty one (`schedules: {}`) declares none, which only changes what the plan lists, or with `prune`, deletes every schedule. ### Pruning With `prune: true` at the top of the file, each section it has is the whole truth: deploying also deletes the queues, workflows and schedules on the server that the section leaves out, whoever created them. The plan lists them:
```plaintext
Plan for workly.yaml on https://api.workly.run:
- delete schedule old-report
- delete workflow old-refresh
- delete queue old-emails (and its 3 failed tasks)
0 to create, 0 to update, 3 to delete, 4 unchanged.
```
* A section left out of the file is still not managed: a file with `prune: true` but no `schedules` leaves every schedule alone. * Deleting a schedule deletes its runs. Deleting a workflow keeps its runs, and those still going finish. * Deleting a queue deletes its tasks, so a queue the file leaves out must have nothing left to deliver. If one has tasks pending, scheduled or running, the plan fails and names it, and nothing is deployed: add it under `queues`, or let its tasks finish (or `workly queues cancel-pending NAME`). Its failed tasks are deleted with it, and the plan says how many. Workly’s own queues (`_workflows`, `_callbacks`) are never deleted. * Since enqueueing creates a queue, every queue your app enqueues to must be in the file, or the next deploy fails while it has work and deletes it when it hasn’t. * Deletes happen after the creates and updates: schedules first, then workflows, then queues, so nothing left points at something gone. In CI, `deploy --yes` deletes without asking, so a queue or schedule removed from the file in a pull request is gone once it merges. [Run `workly plan` on pull requests](#review-the-plan-in-pull-requests) so the deletes are part of the review. ## In CI Deploy the file with your app. With GitHub Actions:
```yaml
- name: Deploy Workly configuration
run: |
curl -fsSL "https://raw.githubusercontent.com/worklydev/workly/$WORKLY_VERSION/install.sh" | sh
workly deploy --yes
env:
WORKLY_VERSION: v0.5.0 # pin the CLI, and bump it like any other dependency
WORKLY_URL: https://api.workly.run
WORKLY_API_KEY: ${{ secrets.WORKLY_API_KEY }}
API_URL: https://api.example.com
JOBS_TOKEN: ${{ secrets.JOBS_TOKEN }}
```
Run it once the app it calls is live: a new schedule or workflow step can be delivered as soon as it’s deployed, and if its endpoint isn’t out yet, those deliveries fail and retry. A step after your app’s deploy step in the same job does this. If your host deploys on its own and reports the deployment to GitHub, as Vercel does, run it on the `deployment_status` event once the production deployment has succeeded:
```yaml
on: deployment_status
jobs:
workly:
if: github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'Production'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4 # the deployed commit
- name: Deploy Workly configuration
run: |
curl -fsSL "https://raw.githubusercontent.com/worklydev/workly/$WORKLY_VERSION/install.sh" | sh
workly deploy --yes
env:
WORKLY_VERSION: v0.5.0
WORKLY_URL: https://api.workly.run
WORKLY_API_KEY: ${{ secrets.WORKLY_API_KEY }}
```
The environment’s name is whatever your host calls it (Vercel’s is `Production`). GitHub starts this workflow for every status the host reports. Railway, for example, reports `in_progress` and then `success`, so each deploy starts two runs: the first shows as skipped, and the one for `success` deploys. In a Python project, `workly-cli==0.5.0` in your dev or CI requirements installs the same pinned binary with `pip` (it’s what `workly[cli]` pulls in). Run `workly validate` in pull requests to catch mistakes before they merge. It needs no server or API key, only the environment variables the file uses. ### Review the plan in pull requests `workly plan` shows what merging would change. `--format markdown` writes it for a pull request comment or the job’s summary page, with deletes called out at the top, and `--detailed-exitcode` exits 0 when nothing would change, 2 when something would, and 3 when that includes deletes (errors exit 1, as always). With GitHub Actions:
```yaml
on: pull_request
jobs:
workly-plan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Plan Workly configuration
run: |
curl -fsSL "https://raw.githubusercontent.com/worklydev/workly/$WORKLY_VERSION/install.sh" | sh
code=0
workly plan --format markdown --detailed-exitcode >> "$GITHUB_STEP_SUMMARY" || code=$?
# 0: no changes, 2: changes, 3: changes that delete something.
if [ "$code" = 3 ]; then
echo "::warning::This pull request deletes queues, workflows or schedules on merge. See the plan in the job summary."
elif [ "$code" != 0 ] && [ "$code" != 2 ]; then
exit "$code"
fi
env:
WORKLY_VERSION: v0.5.0
WORKLY_URL: https://api.workly.run
WORKLY_API_KEY: ${{ secrets.WORKLY_API_KEY }}
```
To make a pull request that deletes something wait for a closer look, fail the step on 3 instead (`exit 1`) and make the check required. For environments that differ by more than a few values, keep a file per environment and pick one with `--file`:
```sh
workly deploy --file workly.staging.yaml --yes
```
# MCP: Workly for coding agents
> Connect Claude Code, Cursor, VS Code or Codex to Workly over MCP, so coding agents can enqueue test tasks, read failures and retry them.
Every Workly server serves [MCP](https://modelcontextprotocol.io) over HTTP at `/mcp`, so there’s nothing to install. Agents such as Claude Code, Codex, Cursor and VS Code use it to read queues, tasks and schedules, replay a delivery against a local handler, and retry what failed. ## Connect Paste this into your agent and it connects itself:
```prompt
Connect this app to Workly over MCP. Workly (https://workly.run) is an HTTP task queue and scheduler: it delivers HTTP requests to my endpoints with retries, runs cron schedules, and records every attempt.
1. Add Workly's MCP server (streamable HTTP) with this app's own way of adding a remote MCP server. Use the endpoint that fits this project, or ask me which:
- Hosted Workly: https://api.workly.run/mcp. It signs in with OAuth in the browser; no API key.
- `workly dev` on this machine: http://localhost:7337/mcp. No sign-in.
- A self-hosted server: https:///mcp, with the server's WORKLY_API_KEY as a bearer token. Read the key from an environment variable; don't write it into the config.
2. If it needs a sign-in, tell me how to start it in this app.
3. Once connected, call get_docs to learn how Workly works, then help me with what I'm building.
If this app can't connect to remote MCP servers, read https://workly.run/llms.txt and use Workly's HTTP API instead.
```
Or set it up yourself. How an agent signs in depends on where Workly runs: | Server | Endpoint | Sign-in | | -------------------------------------------------------------- | ---------------------------- | ----------------------------------- | | [Hosted: workly.run](#hosted-worklyrun) | `https://api.workly.run/mcp` | OAuth in the browser, or an API key | | [Local dev server: `workly dev`](#local-dev-server-workly-dev) | `http://localhost:7337/mcp` | None | | [Self-hosted](#self-hosted) | `https:///mcp` | `WORKLY_API_KEY` as a bearer token | ### Hosted: workly.run Add the server, then sign in. The console opens in your browser: sign in or create an account, pick a workspace, and choose read-only or read and write. Claude Code
```sh
claude mcp add --transport http --scope user workly https://api.workly.run/mcp
claude mcp login workly
```
Codex
```sh
codex mcp add workly --url https://api.workly.run/mcp # opens the sign-in
```
Cursor
```jsonc
// .cursor/mcp.json, or use the install link below
{
"mcpServers": {
"workly": { "url": "https://api.workly.run/mcp" }
}
}
```
VS Code
```jsonc
// .vscode/mcp.json
{
"servers": {
"workly": { "type": "http", "url": "https://api.workly.run/mcp" }
}
}
```
Cursor can also install it from a link: [Add to Cursor](cursor://anysphere.cursor-deeplink/mcp/install?name=workly\&config=eyJ1cmwiOiJodHRwczovL2FwaS53b3JrbHkucnVuL21jcCJ9). For Claude or ChatGPT, see [chat apps](#claude-chatgpt-and-other-chat-apps). #### Plugin for Claude Code and Codex The Workly plugin ([`plugin/`](https://github.com/worklydev/workly/tree/v0.5.0/plugin)) adds the same server along with two skills: one for finding out why deliveries fail and retrying them once they’re fixed, and one for adding Workly to an app and testing the handler end to end. It signs in the same way. Claude Code
```sh
claude plugin marketplace add worklydev/workly
claude plugin install workly@workly
```
Codex
```sh
codex plugin marketplace add worklydev/workly
codex plugin add workly@workly
```
For an agent that can’t open a browser, such as one in CI, create an API key in the console (**API keys**) and send it as a bearer token, as for a [self-hosted](#self-hosted) server. The agent gets that key’s workspace. ### Local dev server: `workly dev` The dev server takes no key, and prints the Claude Code command when it starts. Claude Code
```sh
claude mcp add --transport http workly-dev http://localhost:7337/mcp
```
Codex
```sh
codex mcp add workly-dev --url http://localhost:7337/mcp
```
Cursor
```jsonc
// .cursor/mcp.json
{
"mcpServers": {
"workly-dev": { "url": "http://localhost:7337/mcp" }
}
}
```
VS Code
```jsonc
// .vscode/mcp.json
{
"servers": {
"workly-dev": { "type": "http", "url": "http://localhost:7337/mcp" }
}
}
```
### Self-hosted Send the server’s `WORKLY_API_KEY` as a bearer token. Each config reads it from an environment variable (VS Code prompts for it once and stores it), so it can be committed and shared with your team. Claude Code
```sh
claude mcp add --transport http --scope project workly-prod https://workly.example.com/mcp \
--header 'Authorization: Bearer ${WORKLY_API_KEY}'
```
Codex
```sh
codex mcp add workly-prod --url https://workly.example.com/mcp --bearer-token-env-var WORKLY_API_KEY
```
Cursor
```jsonc
// .cursor/mcp.json
{
"mcpServers": {
"workly-prod": {
"url": "https://workly.example.com/mcp",
"headers": { "Authorization": "Bearer ${env:WORKLY_API_KEY}" }
}
}
}
```
VS Code
```jsonc
// .vscode/mcp.json
{
"inputs": [{ "type": "promptString", "id": "workly-key", "description": "Workly API key", "password": true }],
"servers": {
"workly-prod": {
"type": "http",
"url": "https://workly.example.com/mcp",
"headers": { "Authorization": "Bearer ${input:workly-key}" }
}
}
}
```
### Claude, ChatGPT and other chat apps Chat apps that support remote MCP servers add workly.run as a custom connector: add one with the URL `https://api.workly.run/mcp`. Workly’s console opens; sign in or create an account, pick a workspace, and choose read-only or read and write. A chat app connects from its vendor’s servers, not from your machine, so it can’t reach `workly dev`. It reaches a self-hosted server only if the server has a public address and the app can send its API key as a bearer token. ## Read-only by default `WORKLY_MCP` sets which tools the server offers: | Value | Tools | Default for | | ----------- | ------------------------ | -------------- | | `read-only` | Read tools only | `workly serve` | | `full` | Read and write tools | `workly dev` | | `off` | None: `/mcp` answers 404 | | An agent pointed at production can’t enqueue, retry or cancel anything unless whoever runs the server sets `WORKLY_MCP=full`. workly.run serves the full tools to API keys, and to OAuth connections given read and write access; a connection given read-only access gets the read tools. The endpoint takes the same credentials as `/v1`: with `WORKLY_API_KEY` set, an agent needs the key. Anyone with the key can already do everything through the API, so `read-only` is a guard against an agent’s mistakes, not an access control. ## Tools Tools return a record’s fields with no value as `null` rather than leaving them out, so every task, run or schedule has the same keys. Read tools (marked read-only, so clients can run them without asking): | Tool | What it does | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `list_queues` | All queues in summary: paused or not (by hand, or auto-paused after failures in a row), rate and concurrency limits, current work (due, scheduled, running, failed, how far behind) and the last 24 hours (succeeded, failed, and p50 and p95 attempt duration with its sample size) | | `get_queue` | One queue with its full policy, the last 24 hours, how many tasks succeeded, failed and were cancelled recently, and its failed tasks grouped by error | | `list_tasks` | Task summaries, newest first, by queue, state, group, idempotency key, last error and creation time. Each says when its first attempt started and how long the latest took, so queue wait and handler time can be told apart | | `get_task` | A task with its attempts (outcomes, status codes, errors, response bodies, durations), how many attempts it gets and how many are left | | `get_task_delivery` | The exact signed request for a task, with a curl command to replay it against a local handler | | `wait_for_task` | Waits for an attempt since the task was enqueued or retried, and returns how the latest went, with whether the task has finished or will be retried. Returns at once with the reason if its queue is paused or it isn’t due before the wait would end | | `list_groups` | [Task groups](https://workly.run/docs/groups/), newest first, with their status and counts, by status and name | | `get_group` | One group: its counts, its most recently failed tasks and its callback task | | `list_schedules` | All [schedules](https://workly.run/docs/schedules/) in summary: cron, timezone, URL, paused (and since when), next run (with the schedule’s UTC offset) and how the last one went | | `get_schedule` | One schedule, its next five times (with the schedule’s UTC offset) and its recent runs, including skipped ones and why. A paused schedule’s times are `next_runs_if_resumed` | | `list_runs` | Runs of one schedule or of all of them, newest first, optionally by state | | `get_run` | A run with its attempts | | `get_run_delivery` | The exact signed request for a run, with a curl command | | `wait_for_run` | Waits for an attempt of a run, and returns how the latest went, with whether the run has finished | | `list_workflows` | All [workflows](https://workly.run/docs/workflows/): their steps and how the last run went | | `get_workflow` | One workflow’s definition: steps, dependencies, queues | | `list_workflow_runs` | Runs of one workflow or of all of them, newest first, optionally by status | | `get_workflow_run` | A run with each step’s status, task, last result and what it is blocked by or waiting for | | `get_docs` | These docs, for the server’s version, as Markdown: the list of pages, or one page, SDK and CLI references included | Write tools (`WORKLY_MCP=full`): | Tool | What it does | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `enqueue_task` | Enqueues a task (creating the queue if needed), optionally into a group | | `retry_task` | Delivers a task again now; replays a finished task | | `cancel_task` | Cancels a pending or running task | | `retry_failed_tasks` | Retries every failed task in a queue, or only those with one error | | `cancel_pending_tasks` | Cancels a queue’s pending tasks; they stay in the history and can be retried | | `pause_queue`, `resume_queue` | Stops and restarts a queue’s deliveries; resuming also ends an auto-pause | | `update_queue_policy` | Changes some of a queue’s settings, keeping the rest, including `auto_pause` | | `delete_queue` | Deletes a queue and its finished tasks. Refuses while it has tasks due, scheduled or running, and for Workly’s own queues | | `update_schedule` | Creates a schedule, or changes some of its settings, keeping the rest | | `trigger_schedule` | Runs a schedule now | | `pause_schedule`, `resume_schedule` | Stops and restarts a schedule | | `retry_run`, `cancel_run` | Delivers a run again, or cancels it | | `delete_schedule` | Deletes a schedule and its runs | | `trigger_workflow` | Starts a workflow run, with optional input and idempotency key | | `cancel_workflow_run` | Cancels a workflow run: steps not yet started are skipped | | `retry_workflow_run` | Reopens a failed or cancelled workflow run: its failed and skipped steps run again | | `delete_workflow` | Deletes a workflow’s definition; its runs are kept | Write tools that change one task, run, queue or schedule return it as changed, so an agent can confirm the change without reading it again. The delete tools can’t be undone, so they’re marked destructive, and clients that confirm destructive tools ask first. `delete_queue` won’t delete a queue with work left in it, so an agent can’t lose tasks it didn’t know about; run `cancel_pending_tasks` first if they should go. Arguments are checked against each tool’s schema before anything runs, and errors name the argument at fault (`limit must be an integer, not string`, `state must be one of …`), so an agent can correct its call. ## What agents see Task payloads, headers, response bodies and error messages come from your endpoints and whatever enqueues tasks, so they could contain anything. The server tells agents to treat them as untrusted, and shortens them to keep an agent’s context small: response bodies to 2 KB, payloads to 8 KB, and a task’s attempts to the latest 20. The full data is always in the dashboard and the API. Each tool call is logged (`mcp tool call`, with the tool name, duration and any error).
# Workly for Python
> The Workly Python SDK: enqueue tasks, verify signatures, and manage queues, schedules, task groups, workflows and alerts, sync or async.
The Python SDK for [Workly](https://github.com/worklydev/workly), the open-source HTTP task queue. Enqueue a task, and Workly delivers it to your endpoint over HTTP, retrying with backoff until it succeeds.
```sh
pip install workly # the SDK
pip install "workly[cli]" # the SDK plus the workly binary, for `workly dev`
```
Requires Python 3.11+. The only dependency is [httpx](https://www.python-httpx.org). The `cli` extra adds [`workly-cli`](https://pypi.org/project/workly-cli/), the `workly` binary (about 7 MB): the dev server (`workly dev`) and the CLI, including `workly validate`, `plan` and `deploy` for [configuration as code](https://workly.run/docs/deploy/). Install it on dev machines and in CI; production images that only enqueue tasks don’t need it. ## Quickstart Start a local server with `workly dev` (SQLite, no auth, on :7337), then:
```python
from workly import Workly
client = Workly() # WORKLY_URL, else http://localhost:7337; WORKLY_API_KEY
task = client.tasks.enqueue(
"emails",
url="https://api.example.com/send-email",
payload={"user_id": 123},
)
print(task.id, task.state)
```
Your endpoint receives a `POST` with the JSON payload. Any 2xx marks the task succeeded. A 5xx, 408, 429, timeout or connection error is retried with backoff. Delivery is at-least-once, so make handlers safe to run twice. `Workly-Task-Id` is stable across attempts and works as a deduplication key. ## Verifying deliveries Every delivery is signed: with the server’s `WORKLY_SIGNING_SECRET` when you run Workly yourself, or with your workspace’s secret on [workly.run](#hosted-workly). Check the signature against the **raw** body:
```python
from workly import SignatureVerificationError, verify_signature
# Flask
@app.post("/send-email")
def send_email():
try:
verify_signature(request.get_data(), request.headers) # secret from WORKLY_SIGNING_SECRET
except SignatureVerificationError:
abort(401)
...
# FastAPI / Starlette
@app.post("/send-email")
async def send_email(request: Request):
verify_signature(await request.body(), request.headers, secret)
...
```
By default, signatures older than 5 minutes are rejected (`tolerance=`). `workly dev` signs with `workly-dev-secret`. To test your handler without a server, use `workly.sign(body, secret)`, which returns the headers Workly would send. ## Hosted Workly On [workly.run](https://workly.run), point the client at the hosted API and use the keys from the [console](https://console.workly.run):
```sh
export WORKLY_URL=https://api.workly.run
export WORKLY_API_KEY=... # API keys page
export WORKLY_SIGNING_SECRET=whsec_... # same page; each workspace has its own
```
Nothing else changes. Hosted workspaces have a request rate limit and a monthly task quota: requests over the rate limit are retried for you, and an enqueue over the quota raises `PermissionDeniedError` with the code `quota_exceeded`. ## Tasks
```python
from datetime import datetime, timedelta, timezone
client.tasks.enqueue(
"emails",
url="https://api.example.com/send-email",
payload={"user_id": 123}, # JSON; or body="raw text", content_type="text/csv"
method="POST", # default
headers={"X-Tenant": "acme"}, # sent with every delivery
delay_seconds=60, # or run_at=datetime(..., tzinfo=timezone.utc)
idempotency_key="welcome-123", # an existing task with this key is returned instead
max_attempts=5, # overrides the queue's setting
timeout_seconds=10, # likewise
)
# Up to 100 at once, all or nothing.
client.tasks.enqueue_many(
"emails",
[{"url": "https://api.example.com/send-email", "payload": {"user_id": i}} for i in range(100)],
)
task = client.tasks.get("task_01J9...")
for attempt in client.tasks.attempts(task.id):
print(attempt.number, attempt.outcome, attempt.status_code, attempt.duration_ms)
client.tasks.retry(task.id) # run now; replays a finished task
delivery = client.tasks.delivery(task.id) # the signed request, to replay by hand
client.tasks.cancel(task.id)
# Newest first, across pages. All filters are optional.
for task in client.tasks.list(
queue="emails", state="failed", created_after=datetime.now(timezone.utc) - timedelta(days=1)
):
...
```
## Queues Queues are created automatically on first enqueue. `put` sets a queue’s configuration declaratively, which makes it easy to keep in code: settings you leave out are reset to their defaults.
```python
client.queues.put(
"emails",
max_attempts=10,
min_backoff_seconds=1,
max_backoff_seconds=3600,
backoff_multiplier=2,
timeout_seconds=30,
rate_per_second=50,
max_concurrency=20,
auto_pause_after=20, # pause after 20 failures in a row, until a probe succeeds
)
queue = client.queues.get("emails")
print(queue.stats.due, queue.stats.oldest_due_at)
client.queues.pause("emails") # stop delivering; enqueueing still works
client.queues.resume("emails")
client.queues.cancel_pending("emails") # cancel pending tasks
client.queues.retry_failed("emails") # replay failed tasks, e.g. after an outage
for group in client.queues.failures("emails"): # failed tasks grouped by error
print(group.count, group.error)
client.queues.retry_failed("emails", last_error="HTTP 503 Service Unavailable") # replay one group
client.queues.activity("emails") # succeeded/failed/cancelled in the last 7 days
client.queues.delete("emails") # delete the queue and all its tasks
```
## Schedules A schedule calls a URL on a cron timetable; each call is a run, retried and recorded like a task. Schedules are separate from queues. `put` is declarative, like `queues.put`. See [Schedules](https://workly.run/docs/schedules/).
```python
schedule = client.schedules.put(
"nightly-report",
cron="0 3 * * *",
timezone="Europe/Stockholm",
url="https://api.example.com/reports/nightly",
payload={"kind": "daily"},
max_attempts=3,
overlap="skip", # skip a slot while the previous run is still going
)
print(schedule.next_runs) # the next five times
run = client.schedules.trigger("nightly-report") # run it now
for run in client.runs.list(schedule="nightly-report", state="failed"):
print(run.id, run.scheduled_for, run.last_error)
client.runs.attempts(run.id)
client.runs.retry(run.id) # deliver it again, e.g. a failed or skipped run
client.schedules.pause("nightly-report")
client.schedules.resume("nightly-report") # from the next scheduled time
client.schedules.delete("nightly-report") # with all its runs
```
Runs are delivered with `Workly-Run-Id`, `Workly-Schedule` and `Workly-Scheduled-For` headers and the same signature, so `verify_signature` works unchanged. ## Task groups A group tracks a set of tasks and enqueues a callback task once it’s closed and they’ve all finished: a `POST` of the group’s status and counts. See [Task groups](https://workly.run/docs/groups/). Like every callback, it goes to Workly’s `_callbacks` queue. `groups.enqueue` does the whole job in one call: it creates the group, enqueues the tasks into it in batches of 100 and closes it, and it’s safe to retry with the same `idempotency_key`.
```python
from workly import Callback
group = client.groups.enqueue(
"stock-prices",
(
{"url": "https://api.example.com/jobs/update-stock-price", "payload": {"symbol": c.symbol}}
for c in companies
),
name="daily-stock-prices",
idempotency_key="stock-prices-2026-10-19", # a retry finishes this group rather than start another
# or a dict: {"url": "https://api.example.com/jobs/refresh-views"}
on_complete=Callback(url="https://api.example.com/jobs/refresh-views"),
)
```
Or step by step: `client.groups.create(...)`, then `client.tasks.enqueue(..., group_id=group.id)` or `enqueue_many` for each task, then `client.groups.close(group.id)`, after which it completes once its tasks have finished.
```python
group = client.groups.get(group.id)
print(group.status, group.total, group.succeeded, group.failed, group.remaining)
for task in client.tasks.list(group_id=group.id, state="failed"):
print(task.id, task.last_error)
for group in client.groups.list(name="daily-stock-prices", status="failed"):
print(group.id, group.finished_at)
```
## Callbacks Tasks and schedules take the same `on_complete`: each time the task, or a run of the schedule, succeeds, fails or is cancelled, Workly POSTs a summary of it to the URL. See [Callbacks](https://workly.run/docs/callbacks/).
```python
client.tasks.enqueue(
"exports",
url="https://api.example.com/jobs/export",
payload={"export_id": 42},
on_complete=Callback(url="https://api.example.com/jobs/export-done"),
)
client.schedules.put(
"nightly-report",
cron="0 3 * * *",
url="https://api.example.com/reports/nightly",
on_complete={"url": "https://api.example.com/reports/nightly-done"},
)
```
The summary arrives as the callback task’s JSON body: `task_id`, `queue`, `state`, `attempt_count`, `last_status`, `last_error` and `finished_at` for a task; `run_id`, `schedule` and the same outcome fields for a run. ## Workflows A workflow is a DAG of HTTP steps: each step is delivered as a task once the steps in its `depends_on` have succeeded, and a failed step stops the run. Most workflows live in a `workly.yaml` and are applied with `workly deploy`; `put` does the same from code, declaratively. See [Workflows](https://workly.run/docs/workflows/).
```python
client.workflows.put(
"daily-refresh",
steps=[
{"name": "prices", "url": "https://api.example.com/jobs/prices"},
{"name": "fx", "url": "https://api.example.com/jobs/fx"},
{
"name": "analytics",
"url": "https://api.example.com/jobs/analytics",
"depends_on": ["prices", "fx"],
},
],
)
run = client.workflows.trigger(
"daily-refresh", input={"date": "2026-11-13"}, idempotency_key="refresh-2026-11-13"
)
run = client.workflow_runs.get(run.id)
for step in run.steps:
print(step.name, step.status, step.blocked_by, step.error)
for run in client.workflow_runs.list(workflow="daily-refresh", status="failed"):
client.workflow_runs.retry(run.id) # its failed and skipped steps run again
client.workflow_runs.cancel(run.id) # steps not yet started are skipped
```
A step without `payload` or `body` is sent the run’s context as JSON, with `input`. A step can also wait for a task group: its endpoint answers 2xx with `Workly-Wait-For: group/`, and the step finishes when the group does. ## Alerts Alert channels get an email or a Slack message when tasks or schedule runs fail: within about a minute, then at most one every 30 minutes per queue or schedule. See [Alerts](https://workly.run/docs/alerts/).
```python
channel = client.alerts.create(
"slack", "https://hooks.slack.com/services/..."
) # or "email", "ops@example.com"
client.alerts.test(channel.id) # send a test alert now
print(client.alerts.list().channels)
client.alerts.delete(channel.id)
```
## Async `AsyncWorkly` has the same methods, as coroutines:
```python
from workly import AsyncWorkly
async with AsyncWorkly() as client:
task = await client.tasks.enqueue("emails", url="...", payload={"user_id": 123})
async for task in client.tasks.list(queue="emails"):
...
```
## Errors | Exception | When | | ----------------------- | ----------------------------------------------------------------------- | | `BadRequestError` | 400, invalid request | | `AuthenticationError` | 401, missing or wrong API key | | `PermissionDeniedError` | 403, not allowed, or a hosted limit was reached (e.g. `quota_exceeded`) | | `NotFoundError` | 404, no such queue or task | | `ConflictError` | 409, e.g. retrying a running task | | `PayloadTooLargeError` | 413 | | `RateLimitError` | 429, still rate limited after the retries | | `APIError` | base class of the above; has `.status`, `.code` (stable) and `.message` | | `APIConnectionError` | the server couldn’t be reached | Requests are retried up to `max_retries` times (2 by default), honouring `Retry-After`. A 429 is always retried, since the server refused the request without doing anything. Connection errors and 502–504 are retried for requests that are safe to repeat: an enqueue only when it has an `idempotency_key`, or when the connection failed before the request was sent. ## License [Apache-2.0](https://github.com/worklydev/workly/blob/main/LICENSE)
# Workly for TypeScript
> The Workly TypeScript SDK for Node, Bun, Deno and edge runtimes: enqueue tasks, verify signatures, and manage queues, schedules, groups and workflows.
The TypeScript SDK for [Workly](https://github.com/worklydev/workly), the open-source HTTP task queue. Enqueue a task, and Workly delivers it to your endpoint over HTTP, retrying with backoff until it succeeds.
```sh
npm install @worklydev/sdk
```
No dependencies. It uses `fetch` and Web Crypto, so it runs on Node 22+, Bun, Deno and edge runtimes (Cloudflare Workers, Vercel Edge). ESM, with types included. ## Quickstart Start a local server with `workly dev` (SQLite, no auth, on :7337; `pip install "workly[cli]"` or `curl -fsSL https://raw.githubusercontent.com/worklydev/workly/main/install.sh | sh`), then:
```ts
import { Workly } from '@worklydev/sdk';
const workly = new Workly(); // WORKLY_URL, else http://localhost:7337; WORKLY_API_KEY
const task = await workly.tasks.enqueue('emails', {
url: 'https://api.example.com/send-email',
payload: { userId: 123 },
});
console.log(task.id, task.state);
```
Your endpoint receives a `POST` with the JSON payload. Any 2xx marks the task succeeded. A 5xx, 408, 429, timeout or connection error is retried with backoff. Delivery is at-least-once, so make handlers safe to run twice. `Workly-Task-Id` is stable across attempts and works as a deduplication key. ## Verifying deliveries Every delivery is signed: with the server’s `WORKLY_SIGNING_SECRET` when you run Workly yourself, or with your workspace’s secret on [workly.run](#hosted-workly). Check the signature against the **raw** body, before parsing it:
```ts
import { SignatureVerificationError, verifySignature } from '@worklydev/sdk';
// Next.js route handler, Hono, Cloudflare Workers: anything with a Fetch API Request
export async function POST(request: Request) {
const body = await request.text();
try {
await verifySignature(body, request.headers); // secret from WORKLY_SIGNING_SECRET
} catch (err) {
if (err instanceof SignatureVerificationError) return new Response(null, { status: 401 });
throw err;
}
const payload = JSON.parse(body);
// ...
}
// Express: keep the raw body
app.post('/send-email', express.raw({ type: '*/*' }), async (req, res) => {
await verifySignature(req.body, req.headers, secret);
// ...
});
```
By default, signatures older than 5 minutes are rejected (`{ toleranceSeconds }`). `workly dev` signs with `workly-dev-secret`. To test your handler without a server, `await sign(body, secret)` returns the headers Workly would send. ## Hosted Workly On [workly.run](https://workly.run), point the client at the hosted API and use the keys from the [console](https://console.workly.run):
```sh
export WORKLY_URL=https://api.workly.run
export WORKLY_API_KEY=... # API keys page
export WORKLY_SIGNING_SECRET=whsec_... # same page; each workspace has its own
```
Nothing else changes. Hosted workspaces have a request rate limit and a monthly task quota: requests over the rate limit are retried for you, and an enqueue over the quota throws a `PermissionDeniedError` with the code `quota_exceeded`. ## Tasks
```ts
await workly.tasks.enqueue('emails', {
url: 'https://api.example.com/send-email',
payload: { userId: 123 }, // JSON, sent as is; or body: 'raw text', contentType: 'text/csv'
method: 'POST', // default
headers: { 'X-Tenant': 'acme' }, // sent with every delivery
delaySeconds: 60, // or runAt: new Date(...)
idempotencyKey: 'welcome-123', // an existing task with this key is returned instead
maxAttempts: 5, // overrides the queue's setting
timeoutSeconds: 10, // likewise
});
// Up to 100 at once, all or nothing.
await workly.tasks.enqueueMany(
'emails',
users.map((u) => ({ url: 'https://api.example.com/send-email', payload: { userId: u.id } })),
);
const task = await workly.tasks.get('task_01J9...');
for (const attempt of await workly.tasks.attempts(task.id)) {
console.log(attempt.number, attempt.outcome, attempt.statusCode, attempt.durationMs);
}
await workly.tasks.retry(task.id); // run now; replays a finished task
const delivery = await workly.tasks.delivery(task.id); // the signed request, to replay by hand
await workly.tasks.cancel(task.id);
// Newest first, across pages. All filters are optional.
for await (const task of workly.tasks.list({
queue: 'emails',
state: 'failed',
createdAfter: new Date(Date.now() - 24 * 3600 * 1000),
})) {
// ...
}
```
Field names are camelCase: the SDK converts them to and from the [API](https://github.com/worklydev/workly/blob/main/api/openapi.yaml)’s snake_case (`retry_on_4xx` is `retryOn4xx`). Payloads and headers are your own data, so they are sent and returned exactly as they are. Timestamps are RFC 3339 strings, and fields added by newer servers come through too. ## Queues Queues are created automatically on first enqueue. `put` sets a queue’s configuration declaratively, which makes it easy to keep in code: settings you leave out are reset to their defaults.
```ts
await workly.queues.put('emails', {
maxAttempts: 10,
minBackoffSeconds: 1,
maxBackoffSeconds: 3600,
backoffMultiplier: 2,
timeoutSeconds: 30,
ratePerSecond: 50,
maxConcurrency: 20,
autoPause: { afterFailures: 20 }, // pause after 20 failures in a row, until a probe succeeds
});
const queue = await workly.queues.get('emails');
console.log(queue.stats?.due, queue.stats?.oldestDueAt);
await workly.queues.pause('emails'); // stop delivering; enqueueing still works
await workly.queues.resume('emails');
await workly.queues.cancelPending('emails'); // cancel pending tasks
await workly.queues.retryFailed('emails'); // replay failed tasks, e.g. after an outage
for (const group of await workly.queues.failures('emails')) {
console.log(group.count, group.error); // failed tasks grouped by error
}
await workly.queues.retryFailed('emails', { lastError: 'HTTP 503 Service Unavailable' }); // replay one group
await workly.queues.activity('emails'); // succeeded/failed/cancelled in the last 7 days
await workly.queues.delete('emails'); // delete the queue and all its tasks
```
## Schedules A schedule calls a URL on a cron timetable; each call is a run, retried and recorded like a task. Schedules are separate from queues. `put` is declarative, like `queues.put`. See [Schedules](https://workly.run/docs/schedules/).
```ts
const schedule = await workly.schedules.put('nightly-report', {
cron: '0 3 * * *',
timezone: 'Europe/Stockholm',
url: 'https://api.example.com/reports/nightly',
payload: { kind: 'daily' },
maxAttempts: 3,
overlap: 'skip', // skip a slot while the previous run is still going
});
console.log(schedule.nextRuns); // the next five times
const run = await workly.schedules.trigger('nightly-report'); // run it now
for await (const r of workly.runs.list({ schedule: 'nightly-report', state: 'failed' })) {
console.log(r.id, r.scheduledFor, r.lastError);
}
await workly.runs.attempts(run.id);
await workly.runs.retry(run.id); // deliver it again, e.g. a failed or skipped run
await workly.schedules.pause('nightly-report');
await workly.schedules.resume('nightly-report'); // from the next scheduled time
await workly.schedules.delete('nightly-report'); // with all its runs
```
Runs are delivered with `Workly-Run-Id`, `Workly-Schedule` and `Workly-Scheduled-For` headers and the same signature, so `verifySignature` works unchanged. ## Task groups A group tracks a set of tasks and enqueues a callback task once it’s closed and they’ve all finished: a `POST` of the group’s status and counts. See [Task groups](https://workly.run/docs/groups/). Like every callback, it goes to Workly’s `_callbacks` queue. `groups.enqueue` does the whole job in one call: it creates the group, enqueues the tasks into it in batches of 100 and closes it, and it’s safe to retry with the same `idempotencyKey`.
```ts
const group = await workly.groups.enqueue(
'stock-prices',
companies.map((c) => ({ url: 'https://api.example.com/jobs/update-stock-price', payload: { symbol: c.symbol } })),
{
name: 'daily-stock-prices',
idempotencyKey: 'stock-prices-2026-10-19', // a retry finishes this group rather than start another
onComplete: { url: 'https://api.example.com/jobs/refresh-views' },
},
);
```
Or step by step: `workly.groups.create(...)`, then `workly.tasks.enqueue(..., { groupId: group.id })` or `enqueueMany` for each task, then `workly.groups.close(group.id)`, after which it completes once its tasks have finished.
```ts
const g = await workly.groups.get(group.id);
console.log(g.status, g.total, g.succeeded, g.failed, g.remaining);
for await (const task of workly.tasks.list({ groupId: group.id, state: 'failed' })) {
console.log(task.id, task.lastError);
}
```
The callback’s body is a `GroupSummary`. It arrives as the task’s payload, so its fields are snake_case: `group_id`, `status`, `total`, `succeeded`, `failed`, `cancelled`. ## Callbacks Tasks and schedules take the same `onComplete`: each time the task, or a run of the schedule, succeeds, fails or is cancelled, Workly POSTs a summary of it to the URL. See [Callbacks](https://workly.run/docs/callbacks/).
```ts
await workly.tasks.enqueue('exports', {
url: 'https://api.example.com/jobs/export',
payload: { exportId: 42 },
onComplete: { url: 'https://api.example.com/jobs/export-done' },
});
await workly.schedules.put('nightly-report', {
cron: '0 3 * * *',
url: 'https://api.example.com/reports/nightly',
onComplete: { url: 'https://api.example.com/reports/nightly-done' },
});
```
The bodies are a `TaskSummary` (`task_id`, `queue`, `state`, `attempt_count`, `last_status`, `last_error`, `finished_at`, …) and a `RunSummary` (`run_id`, `schedule`, `state`, …), in snake_case as sent. ## Workflows A workflow is a DAG of HTTP steps: each step is delivered as a task once the steps in its `dependsOn` have succeeded, and a failed step stops the run. Most workflows live in a `workly.yaml` and are applied with `workly deploy`; `put` does the same from code, declaratively. See [Workflows](https://workly.run/docs/workflows/).
```ts
await workly.workflows.put('daily-refresh', {
steps: [
{ name: 'prices', url: 'https://api.example.com/jobs/prices' },
{ name: 'fx', url: 'https://api.example.com/jobs/fx' },
{ name: 'analytics', url: 'https://api.example.com/jobs/analytics', dependsOn: ['prices', 'fx'] },
],
});
const started = await workly.workflows.trigger('daily-refresh', {
input: { date: '2026-11-13' },
idempotencyKey: 'refresh-2026-11-13',
});
const run = await workly.workflowRuns.get(started.id);
for (const step of run.steps) console.log(step.name, step.status, step.blockedBy, step.error);
for await (const failed of workly.workflowRuns.list({ workflow: 'daily-refresh', status: 'failed' })) {
await workly.workflowRuns.retry(failed.id); // its failed and skipped steps run again
}
await workly.workflowRuns.cancel(run.id); // steps not yet started are skipped
```
A step without `payload` or `body` is sent the run’s context as JSON, with `input`. A step can also wait for a task group: its endpoint answers 2xx with `Workly-Wait-For: group/`, and the step finishes when the group does. ## Alerts Alert channels get an email or a Slack message when tasks or schedule runs fail: within about a minute, then at most one every 30 minutes per queue or schedule. See [Alerts](https://workly.run/docs/alerts/).
```ts
const channel = await workly.alerts.create('slack', 'https://hooks.slack.com/services/...'); // or 'email', 'ops@example.com'
await workly.alerts.test(channel.id); // send a test alert now
console.log((await workly.alerts.list()).channels);
await workly.alerts.delete(channel.id);
```
## Errors | Error | When | | ----------------------- | ----------------------------------------------------------------------- | | `BadRequestError` | 400, invalid request | | `AuthenticationError` | 401, missing or wrong API key | | `PermissionDeniedError` | 403, not allowed, or a hosted limit was reached (e.g. `quota_exceeded`) | | `NotFoundError` | 404, no such queue or task | | `ConflictError` | 409, e.g. retrying a running task | | `PayloadTooLargeError` | 413 | | `RateLimitError` | 429, still rate limited after the retries | | `APIError` | base class of the above; has `.status`, `.code` (stable) and `.message` | | `APIConnectionError` | the server couldn’t be reached, or the request timed out | All of them extend `WorklyError`. Requests are retried up to `maxRetries` times (2 by default), honouring `Retry-After`. A 429 is always retried, since the server refused the request without doing anything. Connection errors and 502–504 are retried for requests that are safe to repeat: an enqueue only when it has an `idempotencyKey`, or when the connection failed before the request was sent. ## Options
```ts
const workly = new Workly({
url: 'https://workly.internal', // default WORKLY_URL, then http://localhost:7337
apiKey: '...', // default WORKLY_API_KEY
timeoutMs: 30_000, // per call, including retries
maxRetries: 2,
fetch: customFetch,
});
// Every method takes an AbortSignal last.
await workly.tasks.get(id, { signal: AbortSignal.timeout(1000) });
```
## License [Apache-2.0](https://github.com/worklydev/workly/blob/main/LICENSE)