What is dstack?
dstack is a lightweight command-line tool for running reproducible ML workflows in the cloud.
- Define workflows, incl. dependencies, environment, and required compute resources, via declarative configuration files.
- Run workflows in the cloud via the
- Save output artifacts of workflows and reuse them in other workflows.
- Use workflows to process data, train models, host apps, and launch dev environments.
How does it work?
- Make sure the AWS account credentials are configured locally
- Configure the AWS region (where to provision infrastructure) and an S3 storage bucket name (where to save data)
- Define workflows in
.dstack/workflows.yamlwithin your project directory
- Use the
dstackCLI to run workflows, manage their state and artifacts
- When you run a workflow, the
dstackCLI provisions the required cloud resources, fetches your code, prepares environment, downloads dependencies, runs the workflow, saves artifacts, and tears down cloud resources.