Use spot instances in own cloud
A runner is a machine that can run
dstack workflows. With
dstack you can run workflows using spot instances
in your cloud.
dstack will set up and tear down spot instances automatically based on the need.
This tutorial describes how to use the
dstack autoscale feature with an AWS account.
If you use another cloud vendor, such as GCP, Azure (or some other), please write to
How autoscaling works
- You provide
dstackcredentials to create spot instances in your AWS account.
- You define what types of EC2 instances
dstackis allowed to use, and what maximum number per instance type.
- When you run workflows,
dstackcreates spot instances and automatically configures there
dstack-runnerdaemon that picks up assigned workflows to run.
- When workflows are finished,
dstacktears down unecessary spot instances.
Configure your AWS account
Before you'll be able to use spot instances as runners in your cloud, you have to provide
dstack.ai the credentials
to your AWS account. This can be done by the
dstack aws configure command:
dstack aws configure AWS Access Key ID: AWS Secret Access Key: Region name: Artifact S3 bucket[None]:
Artifact S3 bucket is optional and has to be specified only if you want to use your own S3 bucket to store
Required IAM permissions
dstack autoscale feature requires the following permissions:
ec2:Describe* ec2:RequestSpotInstances ec2:TerminateInstances ec2:CancelSpotInstanceRequests ec2:CreateSecurityGroups ec2:AuthorizeSecurityGroupIngress ec2:AuthorizeSecurityGroupEgress
Manage autoscale rules
Once you've configured the credentials to your AWS account, you'll be able to manage autoscale rules.
Add or replace rules
Type the following to see how to add or replace autoscale rules:
dstack autoscale rules set --help usage: dstack autoscale rules set --max MAX INSTANCE_TYPE positional arguments: INSTANCE_TYPE optional arguments: --max MAX The maximum number of instances
The required positional argument
INSTANCE_TYPE can be any of the instance types
supported by AWS for spot instances.
--max MAX argument can be any integer number. It specifies the maximum number of spot instances
allowed to create per the given instance type.
Let's look at an example:
dstack autoscale rules set m5.xlarge --max 1
This command instructs
dstack that it can create at max one instance of the type
If you try to add a rule that is not supported by your AWS account, you'll get an error.
To remove a rule for a given instance type is the same as setting the corresponding maximum number of instances to 0:
dstack autoscale rules set m5.xlarge 0
As soon as you decrease the number of allowed instances,
dstack immediately shuts down extra spot instances to match
the maximum number of allowed instances.
You can always see the list of all rules by the following command:
dstack autoscale rule list
Clear autoscale rules
To quickly delete all defined rules, you the following command:
dstack autoscale rule clear
This command will immediately shut down all running spot instances if there are any.
Pause and unpause autoscaling
You can pause and unpause autoscaling with a single command:
dstack autoscale pause
dstack autoscale unpause
These commands allow you to pause temporarily autoscaling without deleting the rules.
pause command will immediately shut down all running spot instances if there are any.