Start/Stop AWS ECS Services on a schedule
Sometimes infrastructure we build on AWS is only for private use or development purpose. Sometimes we are not looking for a robust and high availability environment. Sometimes we just need a bunch of AWS resources, that are accessible while we work and don’t create costs during off-hours.
The AWS ECS service might be an example. Consider having a few containers for front-/ backend running while you test and develop new features. Containers you might don’t need during the night or at weekends and want them to shut down and start on a schedule.
The approach to such use cases is a combination of AWS Lambda and AWS Cloudwatch.
We can use Cloudwatch rules, which can be configured similar to a cronjob, and let them trigger a lambda function for us. This lambda will be used to stop or start any AWS resource we want. This works with EC2 machines, ECS Services or you can scale up and down RDS capacity.
It is not limited to cost optimization. You can use a configuration like this to automatically copy files from one S3 Bucket to another on a specific schedule. You could join and remove users to and from a group during special time periods. Maybe someone supports you on the weekends, and you can easily remove him from a specific group between MO-FR.
Basically, there are a million things you can let the lambda do for you and just need to implement a few commands in the lambda code and configure the right schedule for the Cloudwatch Rules.
Here is my approach to scale all ECS Services to the desired task count of 0 after work and back to 1 before work.
Prerequisites
For our Lambda code, we use a zip file within an S3 bucket. It is important that the file and the S3 bucket is in place before we run our Cloudformation code.
Download the zip file directly from my GitHub repository and save it in an S3 bucket.
AWS Cloudformation
You can do anything manually but I decided to write a small Cloudformation file to automatically create the Lamba, the right permissions, and the Cloudwatch rules.
It is important to know, that our Lambda gets full permission on ECS. For simplicity this is fine, but if you run this on a production environment you should consider a least privileges approach.
AWS Lambda
In my case, I decided to write the script in Python and use Boto3 for accessing the AWS APIs. Since Boto3 is supported naturally we didn’t need to include the package in our python .zip file.
The entire code looks like this.
When the AWS Cloudwatch alarm triggers the Lambda function, it passes an action parameter and a cluster name to the Lambda.
Depending on action == run or action == stop, it sets the desired task count to either 0 or 1.
With the cluster name, the Lambda queries the cluster and looks for all services which have the launch type “Fargate” and schedulingStrategy of “Replica”. From the reply it creates a list and iterates over all entries to stop or start all services.
Conclusion
This is an example of a basic setup to do things in AWS on a schedule. As explained at the start, this is not limited to ECS or just cost savings. You can do a lot of different tasks and just need to configure the Cloudwatch Rules and Lambda functions accordingly.
