DAG Workflows

How Do DAG Dependencies Work in Argo Workflows?

Master DAG templates in Argo Workflows. Learn tasks, dependencies, enhanced depends logic, conditional branches, loops, failFast, output passing and how DAGs compare with steps.

10 min read 2,011 wordsUpdated By Argo Workflows Editorial Team

A directed acyclic graph (DAG) is the most flexible way to describe execution order in Argo Workflows. Instead of arranging work into fixed sequential stages, you list tasks and declare what each one depends on. The controller then starts every task as soon as its dependencies are satisfied, which maximises parallelism and keeps complex pipelines readable. This guide explains DAG templates from first principles through advanced dependency logic.

What a DAG Template Looks Like

A DAG template contains a list of tasks. Each task has a name, a template to run and, optionally, the dependencies that must be satisfied first.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-basic-
spec:
  entrypoint: pipeline
  templates:
    - name: pipeline
      dag:
        tasks:
          - name: checkout
            template: run
            arguments: {parameters: [{name: cmd, value: checkout}]}
          - name: unit-tests
            depends: checkout
            template: run
            arguments: {parameters: [{name: cmd, value: unit-tests}]}
          - name: lint
            depends: checkout
            template: run
            arguments: {parameters: [{name: cmd, value: lint}]}
          - name: package
            depends: unit-tests && lint
            template: run
            arguments: {parameters: [{name: cmd, value: package}]}
    - name: run
      inputs: {parameters: [{name: cmd}]}
      container:
        image: busybox:1.36
        command: [sh, -c, "echo running {{inputs.parameters.cmd}}; sleep 3"]

checkout has no dependencies, so it starts immediately. unit-tests and lint both depend on it and run in parallel. package waits for both. Load this manifest into the DAG visualizer to see the graph rendered.

DAG vs Steps: Which Should You Use?

Both orchestration styles create the same pods. The difference is how ordering is expressed.

Aspect Steps DAG
Ordering model Sequential stages of parallel steps Explicit dependencies per task
Best for Linear pipelines Irregular or wide graphs
Start time of independent work Waits for the whole previous stage Starts as soon as its own dependencies finish
Readability for large graphs Gets awkward Stays clear
Reference syntax {{steps.name...}} {{tasks.name...}}

Consider a pipeline where task X depends only on A, while B and C are slow. With steps, X must wait for the entire stage containing A, B and C. With a DAG, X starts as soon as A finishes. On large graphs that difference can cut total runtime significantly.

Two Ways to Declare Dependencies

The dependencies Field

The original syntax is a list of task names. All listed tasks must succeed before the task runs.

- name: package
  dependencies: [unit-tests, lint]
  template: run

It is simple, but it cannot express "run if A failed" or "run if either B or C succeeded".

Enhanced depends Logic

The depends field accepts a boolean expression over task results. A bare task name means "succeeded or skipped, or daemoned", and you can add explicit result selectors:

Selector Meaning
A A succeeded, was skipped or is daemoned
A.Succeeded A completed successfully
A.Failed A failed
A.Errored A encountered an Argo error
A.Skipped A was skipped by a when condition
A.Omitted A was omitted because its own dependencies could not be met
A.Daemoned A is a daemon task that is running
A.AnySucceeded At least one iteration of a looped task succeeded
A.AllFailed Every iteration of a looped task failed

Combine selectors with &&, || and !, using parentheses for grouping:

- name: notify-failure
  depends: "build.Failed || test.Failed"
  template: alert
- name: deploy
  depends: "build.Succeeded && (test.Succeeded || test.Skipped)"
  template: deploy

Do not mix dependencies and depends in the same task. New manifests should prefer depends because it is more expressive and is what the official documentation recommends.

Passing Data Between DAG Tasks

Outputs are referenced through tasks instead of steps:

- name: version
  template: get-version
- name: build
  depends: version
  template: build-image
  arguments:
    parameters:
      - name: tag
        value: "{{tasks.version.outputs.parameters.tag}}"
- name: scan
  depends: build
  template: scan-image
  arguments:
    artifacts:
      - name: sbom
        from: "{{tasks.build.outputs.artifacts.sbom}}"

A task can only reference outputs of tasks it depends on, directly or transitively. Referencing an unrelated task fails validation because there is no guarantee the value exists yet.

Conditional Execution in DAGs

when works on DAG tasks just as on steps. A task whose condition is false becomes Skipped, and tasks that depend on it treat a skip as satisfied when using bare names:

- name: deploy-prod
  depends: package
  when: "'{{workflow.parameters.branch}}' == 'main'"
  template: deploy
- name: smoke-test
  depends: deploy-prod
  template: smoke

If you want smoke-test to run only after a real deployment, write depends: deploy-prod.Succeeded instead, so a skipped deploy omits the smoke test.

Loops Inside DAGs

withItems, withParam and withSequence expand a single task into many parallel nodes:

- name: train
  depends: prepare
  template: train-model
  arguments:
    parameters: [{name: lr, value: "{{item}}"}]
  withItems: ["0.001", "0.01", "0.1"]
- name: select-best
  depends: train.AnySucceeded
  template: compare

Here select-best runs as long as at least one learning rate succeeded. Collect looped outputs with {{tasks.train.outputs.parameters.score}}, which resolves to a JSON list of all iteration values.

Failure Behaviour: failFast and continueOn

By default a DAG uses fail-fast behaviour: when one task fails, the controller stops scheduling new tasks, lets running ones finish, and marks the DAG as failed. Set failFast: false on the DAG template to keep scheduling every task whose dependencies are still satisfiable.

- name: pipeline
  dag:
    failFast: false
    tasks: []

For individual non-critical tasks, continueOn: {failed: true} treats a failure as acceptable for dependency purposes, so downstream tasks proceed. Use this for optional work such as uploading metrics or warming caches.

Dynamic DAGs With withParam

Static loops are useful, but the most powerful DAGs adapt to data discovered at run time. A first task lists the work — files in a bucket, tables in a schema, services in a monorepo that changed — and prints it as JSON. A second task loops over that list with withParam, creating one node per item. Because each item is an object, you can pass several fields at once:

- name: discover
  template: list-services
- name: build
  depends: discover
  template: build-service
  arguments:
    parameters:
      - {name: path, value: "{{item.path}}"}
      - {name: image, value: "{{item.image}}"}
  withParam: "{{tasks.discover.outputs.result}}"

If list-services prints [{"path": "svc/api", "image": "api"}, {"path": "svc/web", "image": "web"}], the DAG builds exactly those two services. When nothing changed, the list is empty and the build task is skipped entirely, which is how monorepo pipelines avoid rebuilding everything on every commit. Combine this pattern with parallelism on the DAG template so a large change set does not overwhelm your build nodes.

Nesting DAGs

A task's template can itself be a DAG or steps template. This is the best way to keep very large pipelines understandable: a top-level DAG of stages, each stage a DAG of tasks. Combined with templateRef, nested DAGs let platform teams publish reusable sub-pipelines — for example, a "build, scan and sign" DAG that every application pipeline calls.

Daemon Tasks: Services That Live During the DAG

Some pipelines need a temporary service — a database for integration tests, a mock API or a cache — that other tasks talk to while it runs. Mark the template with daemon: true and Argo treats the task as complete once its container is running and ready, rather than when it exits. Downstream tasks can reach it through its pod IP, available as {{tasks.db.ip}}, and the controller stops the daemon automatically when the template that started it finishes.

- name: postgres
  daemon: true
  container:
    image: postgres:16-alpine
    env: [{name: POSTGRES_PASSWORD, value: test-only}]
    readinessProbe:
      exec: {command: [pg_isready, -U, postgres]}

A readiness probe is essential: without it, dependent tasks may start before the service accepts connections. Daemons are best reserved for test fixtures; production services belong in normal Deployments.

A Real-World Example: Machine Learning Pipeline

The DAG below models a typical training pipeline. Data validation and feature engineering run first, three model variants train in parallel, evaluation picks the best result, and registration happens only if evaluation passes a quality gate.

- name: ml-pipeline
  dag:
    tasks:
      - {name: validate-data, template: step, arguments: {parameters: [{name: cmd, value: validate}]}}
      - {name: features, depends: validate-data, template: step, arguments: {parameters: [{name: cmd, value: features}]}}
      - name: train
        depends: features
        template: train
        arguments: {parameters: [{name: model, value: "{{item}}"}]}
        withItems: [xgboost, lightgbm, logistic]
      - {name: evaluate, depends: train.AnySucceeded, template: evaluate}
      - name: register
        depends: evaluate.Succeeded
        when: "{{tasks.evaluate.outputs.parameters.auc}} >= 0.85"
        template: register

Notice how the graph documents the process. Anyone reading it understands the order of operations, which models are trained, what tolerance for failure exists and what gate protects the model registry. That readability is one of the strongest arguments for DAG templates in data and ML teams, where pipelines are reviewed by people with very different backgrounds.

Lifecycle Hooks on DAG Tasks

Each task can declare hooks that run another template when an expression becomes true, without affecting the task's own result. A common use is notifying a channel as soon as a critical task fails, instead of waiting for the workflow-level exit handler at the very end:

- name: deploy
  depends: package
  template: deploy
  hooks:
    failed:
      expression: tasks.deploy.status == "Failed"
      template: page-on-call

Hooks keep alerting logic next to the work it relates to, which is easier to maintain than a single large exit handler full of conditions.

Designing Good DAGs

  1. Draw it first. Sketch the graph or use the visualizer; unnecessary edges are easy to spot visually.
  2. Depend on the minimum. Every extra dependency delays the task and reduces parallelism.
  3. Keep tasks coarse enough. Pod start-up takes seconds; thousands of tiny tasks waste time in scheduling.
  4. Name tasks for humans. Task names appear in the UI and in expressions; build-api-image beats t7.
  5. Limit concurrency deliberately. Use parallelism on the DAG template to protect downstream systems such as databases.
  6. Plan for partial failure. Decide which tasks are critical and use depends selectors or continueOn for the rest.

Troubleshooting DAG Problems

  • "dependency cycle detected" — two tasks depend on each other directly or indirectly. The validator reports the cycle path.
  • "task X depends on undefined task Y" — a typo in depends. Names are case-sensitive.
  • Task never runs and shows Omitted — its depends expression became impossible, often because an upstream task was skipped and you used .Succeeded.
  • "Unable to resolve {{tasks.a.outputs...}}" — the referencing task does not depend on a, or a is a container template without the declared output.
  • Everything runs sequentially — dependencies were declared too broadly; check whether each edge is truly needed.
  • Looped outputs are a string, not a list — parse them with fromJSON in an expression tag or pass them to withParam.

Frequently Asked Questions

What happens when a task in a loop fails?

With fail-fast enabled, one failed iteration stops new tasks from being scheduled and fails the DAG. To tolerate partial failures, use AnySucceeded in downstream depends expressions and set failFast: false, then inspect individual iterations in the UI.

Can I limit how many DAG tasks run at once?

Yes. Set parallelism on the DAG template to cap concurrent children, or on the workflow spec to cap all pods. Semaphores provide limits shared across several workflows.

Can I add a task to a running DAG?

No. The graph is fixed when the workflow starts. Dynamic behaviour comes from loops over runtime data with withParam, from conditions, and from recursion.

Can a DAG have multiple starting tasks?

Yes. Every task without dependencies starts immediately, in parallel.

Is there a limit to DAG size?

There is no fixed task limit, but very large workflows can exceed the size limit of a single Kubernetes object. Enable node status offloading for workflows with thousands of nodes.

Can a task depend on a task in another DAG?

Not directly. Wrap both in a parent DAG and express the dependency there, or pass data through artifacts.

How do I visualise a DAG without a cluster?

Paste the YAML into our DAG visualizer; it parses tasks and both dependency syntaxes in your browser.

Key Takeaways

  • A DAG starts every task as soon as its own dependencies are satisfied.
  • Prefer depends over dependencies for new manifests.
  • Result selectors such as .Failed and .AnySucceeded express real-world failure logic.
  • Reference outputs only from tasks you depend on.
  • Use withParam for data-driven fan-out and parallelism to protect shared systems.
  • Nest DAGs and reuse sub-graphs with templateRef to keep large pipelines readable.
  • Draw the graph before writing it, and remove every dependency that is not strictly required.

Conclusion

DAG templates express real-world dependency graphs directly, start work as early as possible and offer precise control over failure handling through depends logic. Start with simple task-name dependencies, adopt result selectors as your pipelines grow, and keep graphs readable with nesting and clear names. Next, learn how to package DAGs for reuse in the WorkflowTemplates guide. Good DAGs are both fast and easy to read.

Argo Workflows Editorial Team

Written and reviewed according to our editorial policy. Spotted an error? Tell us and we will fix it. This site is independent and not affiliated with the Argo Project.