Skip to content

Coming from Celery

Most Celery concepts have a direct counterpart. The differences that remain are deliberate, and they are listed at the bottom rather than discovered in production.

The structural difference

Celery enqueues into a broker through an ambient app object; the send is invisible to your database transaction. gylo enqueues into your database, on a connection you pass, so the job commits with your data or not at all:

import gylo

app = gylo.Gylo()


@app.task(retry_on=(TimeoutError,))
async def send_receipt(order_id: int) -> None: ...


async def place_order(conn) -> None:
    async with conn.transaction():
        order = await create_order(conn)
        await send_receipt.options(delay=60, queue="emails").enqueue(
            conn, order.id
        )

If the order rolls back, so does the receipt — the dual-write window every broker-backed queue lives with does not exist. Where no connection is at hand, await send_receipt.submit(order_id) borrows one from the app's pool, with the weaker promise that entails.

The other split worth internalising: failure policy lives on the task (retry_on, no_retry_on, timeout, store_result), placement lives at enqueue (queue, priority, delay, max_attempts, unique, concurrency caps — all via .options()). There is no routing table in config; the call site says where the job goes.

Concept map

Celery gylo
Celery(broker=…, backend=…) gylo.Gylo() — Postgres is broker and backend
@app.task @app.task
task.delay(x) await task.enqueue(conn, x)
apply_async(countdown=60) task.options(delay=60).enqueue(conn, …)
apply_async(queue=…, priority=…) task.options(queue=…, priority=…) — lower number runs first
bind=True, self context=True, a gylo.JobContext first parameter
autoretry_for=(…) retry_on=(…) — already the default for Exception
retry_backoff, retry_jitter on by default; shaped worker-wide by --retry-base / --retry-cap
max_retries=3 options(max_attempts=4) — attempts, not retries
self.retry() just raise — the policy decides
Reject / Ignore raise gylo.NoRetryError(…)
acks_late=True always true — a job is acknowledged by finishing
time_limit / soft_time_limit timeout — on by default at 300s
chain, group, chord gylo.chain, gylo.group, gylo.chord
task.s(x) / task.si(x) task.signature(x) — every signature is immutable
beat + beat_schedule @app.cron("*/5 * * * *") — no beat process, no leader
AsyncResult.get() await gylo.outcome(conn, job_id) — poll, never block
revoke() gylo.cancel(conn, *ids) — not-yet-started jobs only
celery-once and friends options(unique=True) or unique="your-key", built in
Flower gylo queue, gylo jobs, Prometheus metrics

What gylo will not do the same way

Chains do not pass results. A Celery chain feeds each task's return value to the next; every gylo signature behaves like si() — a child receives its own arguments, never its parents' output. Share state through your own tables keyed by something the tasks agree on. This is scope, not a gap: implicit result piping is how a queue becomes a badly specified dataflow engine. Workflows explains the model.

Running jobs cannot be revoked. gylo.cancel cancels what has not started and reports how many that was. Celery's revoke(terminate=True) kills a worker process mid-task, taking every sibling job with it; gylo declines to offer the footgun and tells you exactly what it did instead. Cooperative cancellation and timeouts cover the rest — Results and cancellation.

There is no rate_limit. Keyed concurrency bounds how many jobs run at once per key — the multi-tenant fairness problem — but nothing bounds jobs per second yet.

No absolute ETA. delay is seconds from now. For "run at 9am", compute the difference, or give the task a cron schedule.

Attempts are counted, not retries. ctx.attempt starts at 1 and max_attempts is the total budget. Celery's max_retries=3 means four runs; say max_attempts=4.

Results are opt-in and expire. store_result=True, and the retrieval window is the completed-job retention window (24h by default). There is no blocking .get(); a caller that needs the answer synchronously usually wanted a function call.

One backend. Postgres. No RabbitMQ, no Redis, no SQS. If the job must survive, it goes where your data already lives.