Your first days working with cloud environments can be confusing because you’re not used to the different paradigms, and some of your first questions may have simple answers. Looking for advice, many companies ask us how to be profitable when they start in the cloud.
While this post may be a bit basic, we think it may help those who have some early questions, as well as help some companies save some money when starting out in the cloud. In “the basics of cloud cost efficiency” you will learn how to economize your cloud costs in testing and pre-production environments on Amazon Web Services using simple AWS CLI commands.
Normally, most of the infrastructure costs are related to the computational capacity of the servers. In AWS, when we talk about compute capacity, we talk about EC2 instances, where the cost depends only on the actual hours you use on each instance. Also, a larger instance in terms of computational capacity is logically more expensive. It is quite common to use testing and pre-production servers only during business hours, as it makes no sense to have these kinds of environments running 24 hours a day. For this reason, it is interesting, in terms of cost efficiency in the cloud, to take advantage of how easy it is to start and stop instances in the AWS cloud. For example, you can stop all instances overnight and restart them in the morning. There are two different approaches:
- Long distance instances (not autoscaling)
- Instances within groups Autoscaling (ASG)
Cost efficiency in the cloud: long-haul instances
For long-running instances, we can use a simple bash script that can be scheduled to run in a cronjob. This cron would use AWS CLI commands to stop specific instances. Imagine that all your instances within an AWS account are tagged with the “Environment” tag, with different values depending on the environment they depend on: production, pre-production, and testing. The script will simply check which instance IDs have a specific tag (“Environment = Preproduction” for example), and once it has their IDs, we’ll use the AWS API to send a stop signal to these EC2 instances. The script might look something like the following:
In the previous command, we are filtering by environment label, and once we have all the information of the different instances, we will use JMESPath to get only the instance ID. Once we have it, we will send a stop-instance command to all these instances.
To start them, we have another script that executes the start-instances command:
Cost efficiency in the cloud: Autoscaling clusters
For instances within an ASG (AutoScaling Group), the approach will be different due to the fact that, if you turn off an instance within an ASG, the ASG will replace it with a new one.
The ASG configuration has three different values corresponding to the number of instances within an ASG:
- Minimum: minimum number of instances within an ASG
- Maximum: maximum number of instances within an ASG
- Desired: actual number of instances, which will be between the minimum and maximum, depending on the autoscaling group policies and the usage of EC2 instances.
Imagine you have an autoscaling group with 4 instances (minimum=4, desired=4, maximum=8). In this case, instead of stopping instances, what you could do is to reduce the number of instances to zero (minimum and desired = 0) at night, and restore the normal values in the morning (increasing the minimum and desired = 4).
In order to execute these actions, we will again use some of the available AWS CLI commands. In this case we will use what are known as “ASG scheduled actions“. These scheduled actions are based on universal cron expressions, where we have to use the UTC time zone. We will create two scheduled actions, the first one executed in the evening:
aws autoscaling put-scheduled-update-group-action --scheduled-action-name 'scale-down-night' --auto-scaling-group-name ASG-NAME --region eu-west-1 --recurrence '0 18 * * *' --min-size 0 --desired-capacity 0
The above command, executed only once, will create a scheduled action called “scale-down-night” that AWS will execute recurrently every day at 18:00 UTC (note that the rule is triggered at 19:00 in Spain because it was in UTC +1 at that time), and will decrease the number of ASG instances to 0. And this next one would be executed in the morning:
aws autoscaling put-scheduled-update-group-action --scheduled-action-name 'scale-up-morning' --auto-scaling-group-name ASG-NAME --region eu-west-1 --recurrence '0 7 * * *' --min-size 4 --desired-capacity 4
Be aware that once you have increased the number of instances inside the ASG, the autoscaling policies will again decide the actual number of instances needed (desired value) depending on the ASG load (e.g. CPU utilization). CloudWatch alarm-based scaling policies define how ASGs scale out or scale in.
Conclusion
These are very simple AWS CLI commands to start/stop instances easily, to save costs in AWS test/dev environments, and of course, there are usually more elements to worry about in a full pre-production environment.
However, you can create powerful scripts with more complex logic and intelligence using your favorite programming language. AWS offers different SDKs for practically all programming languages: Python, Ruby, Java, PHP, .Net, Go, C++, Node.js…
I hope you enjoyed this post and I encourage you to check our blog to read other posts that may be of interest to you. Don’t hesitate to contact us if you would like us to help you with your projects.
