Local Docker Orchestrator
How to orchestrate pipelines locally in Docker
The local Docker orchestrator is an orchestrator flavor which comes built-in with ZenML and runs your pipelines locally using Docker.
When to use it
You should use the local Docker orchestrator if:
-
you want the steps of your pipeline to run locally in isolated environments.
-
you want to debug issues that happen when running your pipeline in Docker containers without waiting and paying for remote infrastructure.
How to deploy it
To use the local Docker orchestrator, you only need to have Docker installed and running.
How to use it
To use the local Docker orchestrator, we can register it and use it in our active stack:
You can now run any ZenML pipeline using the local Docker orchestrator:
Additional configuration
For additional configuration of the Local Docker orchestrator, you can pass LocalDockerOrchestratorSettings
when defining or running your pipeline. Check out the API docs for a full list of available attributes and this docs page for more information on how to specify settings.
For more information and a full list of configurable attributes of the local Docker orchestrator, check out the API Docs.
Enabling CUDA for GPU-backed hardware
Note that if you wish to use this orchestrator to run steps on a GPU, you will need to follow the instructions on this page to ensure that it works. It requires adding some extra settings customization and is essential to enable CUDA for the GPU to give its full acceleration.