Workflows
Run jobs that depend on each other without a workflow engine.
A workflow is a set of your endpoints that run in order: each step starts once the steps it depends on have succeeded. Every step is an ordinary task, retried, signed and recorded like one, so a workflow adds ordering to what you already have, not a new kind of worker.
Declare the steps, and what each depends on.
Here prices, fx and news start together. analytics runs once prices and fx have both succeeded, and notify once analytics and news have. A schedule starts it every night at 03:00.
Workflows live in workly.yaml next to your code, or are set through the API and SDKs. Start a run from the dashboard, the CLI, the SDKs, a schedule or your coding agent.
# workly.yaml, applied with `workly deploy`
version: 1
workflows:
daily-refresh:
max_duration_seconds: 7200
steps:
- name: prices # fans out over a task group
url: https://api.example.com/jobs/start-price-update
- name: fx
url: https://api.example.com/jobs/refresh-fx
- name: news
url: https://api.example.com/jobs/import-news
queue: imports # that queue's rate and concurrency
- name: analytics
url: https://api.example.com/jobs/rebuild-analytics
depends_on: [prices, fx]
- name: notify
url: https://api.example.com/jobs/notify
depends_on: [analytics, news]
schedules:
nightly-refresh:
cron: "0 3 * * *"
timezone: Europe/Stockholm
workflow: daily-refresh# Start a run by hand, with input for its steps
workly workflows trigger daily-refresh --input '{"date": "2026-11-13"}'
# Fixed what failed? Pick up where it stopped
workly workflows retry wfr_01M7AH8EP90ZQ4E9228337GGVRFan out inside a step, then wait.
The prices step starts 500 price updates. Its endpoint puts them in a task group and answers 202 with Workly-Wait-For. The step waits until the group completes, and analytics starts only if every update succeeded.
from workly import SignatureVerificationError, Workly, verify_signature
workly = Workly() # WORKLY_URL, WORKLY_API_KEY
@app.post("/jobs/start-price-update")
def start_price_update():
try:
verify_signature(request.get_data(), request.headers)
except SignatureVerificationError:
abort(401) # a 4xx isn't retried
# The same on a redelivery, so it reuses the group;
# new when the run is retried, so that starts a fresh one
key = (request.headers["Workly-Workflow-Step-Id"] + "-"
+ request.headers["Workly-Workflow-Run-Attempt"])
# One task per company, in a group that's closed once they're in
group = workly.groups.enqueue("prices", price_updates(), idempotency_key=key)
# The step finishes when the group does
return "", 202, {"Workly-Wait-For": f"group/{group.id}"}
Steps are tasks, so they retry like tasks.
Steps are tasks
Each step is delivered, signed and retried like any task, in the _workflows queue or one you name, so it shares that queue's rate and concurrency limits with everything else that calls the endpoint.
Parallel where it can be
Steps without dependencies start together; a step starts once every step in its depends_on has succeeded. Up to 100 steps, and a cycle is rejected with its path.
Wait for a group
A step that starts many tasks answers 202 with Workly-Wait-For: group/<id>, and stays waiting until that task group completes. That's fan-out and fan-in inside a run.
Stops at the first failure
A step that fails for good fails the run: blocked steps are skipped and queued ones cancelled, so nothing downstream runs on bad data.
Retry from where it failed
Fix the handler and retry the run. Steps that succeeded aren't sent again; failed and skipped ones run under the definition and input the run started with.
Start a workflow from a schedule
A schedule can start a workflow instead of calling a URL. With overlap: skip, a slow run means the next slot is skipped, not doubled.
Started once
Trigger with an idempotency_key and a repeat returns the same run. Scheduled runs use the schedule run's ID, so a retried slot never starts a second run.
A deadline per run
max_duration_seconds, a day by default, fails a run that hasn't finished, and says which steps didn't.
Changes don't touch running runs
Deploy a new definition at any time: a run keeps the version it started with, through retries too.
Each step says why it hasn't finished.
Open the run and every step says why it is where it is, with its task, its last status and error, and how long each phase took. Your coding agent reads the same thing over MCP.
Blocked
The steps it waits for haven't succeeded yet; look at those.
Pending, with attempts
Its endpoint is failing and the task is retrying; the task page has every response.
Pending, no attempts
Its queue is paused, at its concurrency limit or behind.
Waiting for a group
The group's tasks are still going, or it was never closed. The group page shows which.
When you need a workflow engine instead.
Static by design
A workflow is a DAG declared up front: no conditions, loops, outputs passed between steps or nested workflows. A step that needs an earlier step's data reads it from where that step stored it.
Workly never runs your code
Every step is a request to your app. For durable workflows written as code, with branches, sleeps and state between steps, use Temporal, Inngest or Hatchet.
Steps are at least once
A step's request can arrive twice, for example after a server restart mid-delivery. Workly-Workflow-Step-Id is the same on every delivery: use it as an idempotency key.
Questions
- How do I run background jobs in a specific order?
- Declare them as steps of a workflow, each with the steps it depends_on. Workly sends a step once all of its dependencies have succeeded, runs independent steps in parallel, and stops the run if a step fails for good. Each step is an HTTP request to your app, retried like any task.
- Do I need Airflow or Temporal for this?
- Not for a pipeline of endpoints that run in a fixed order, such as a nightly import, refresh and notify. A Workly workflow does that with nothing new to run. If your workflow needs branches, loops, sleeps or results passed between steps in code, a workflow engine such as Temporal, Inngest or Hatchet is the better fit.
- What happens when a step fails?
- Its task is retried with its queue's policy first. If it still fails, the run fails: steps that haven't started are skipped, and requests already in flight finish but start nothing. Once you've fixed the endpoint, retry the run: steps that succeeded aren't sent again.
- Can one step fan out over many tasks?
- Yes. The step's endpoint creates a task group, enqueues the tasks, closes the group and answers 202 with Workly-Wait-For: group/<id>. The step finishes when the group does, and the next step starts only if every task in it succeeded.
- How do steps share data?
- Steps are sent the run's input and IDs, not each other's responses. A step that produces data stores it, in your database or object storage, and a later step reads it from there, keyed on the run ID.
Try it in five minutes.
workly dev gives you a server and dashboard on your laptop with nothing else to start, and the Python and TypeScript SDKs let your app enqueue to it. Or sign in to hosted Workly and get a workspace and an API key in a minute, free on the Hobby plan.