Machine learning pipelines
Training pipelines combine steps with very different needs. Data preparation runs on CPU nodes, training requests GPUs through resource limits and node selectors, and evaluation compares metrics before a model is registered. Each step is a separate pod, so it gets exactly the hardware it needs.
Read the related guide →Typical pipeline
- Prepare data
- Train (GPU)
- Evaluate
- Register model
Argo Workflows features used
- withItems for hyperparameter sweeps
- Artifacts for datasets and models
- when gate on accuracy
