Auto-Scaling Amazon ElastiCache

Contributed by Khurshidali Shaikh on 30 Aug 2013

• 
• Tweet
• Get in Touch

Amazon ElastiCache is an in-memory caching service provided by Amazon. It is protocol compliant with memcached. So one can use ElastiCache as a memcached server without having to run and manage a memcached server.

Launching a cluster of ElastiCache nodes is a matter of a few clicks with the use of AWS console or a few AWS API calls. As the demand varies, the cluster size can be scaled up and down using the console or API. Amazon Auto Scaling is another convenient mechanism that can be used to scale EC2 clusters automatically based on various load conditions. A similar mechanism does not exist for ElastiCache. The only way to scale ElastiCache is by either using the console (manually) or by writing a set of scripts which monitor the usage and add/remove nodes by using the AWS API.

Automation is the way to go and at Neev, we built the auto-scaling Elasticache as a component for the use of Neevites and other AWS users. It was developed in Python and this is how it works:

1. Accepts AWS region name, ElastiCache cluster name, max nodes, min nodes, AWS key/secret and the memory usage threshold as parameters.
2. Runs as a periodic cron job and collected average memory usage across the ElastiCache cluster
3. Adds node to the cluster if the average memory usage exceeds the specified threshold.
4. Removes a node from the cluster if the average memory usage falls below the threshold.

The above has been released as an open source utility and is available here https://github.com/neevtechnologies/aws_elasticache_autoscale. Feel free to use it and provide feedback.

Neev RazorFish, is one of the first AWS consulting partners in India. Neev specializes in architecting and developing web and mobile applications that run on the cloud, utilizing commonly available services that are tried and tested thereby providing high quality and reliability with reduced ROI.

Credits:
Thanks to Jimmy Sebastian (jimmy at neevtech dot com) for handling the development and testing of this utility.

Visit us at Neevtech.com to know more about our offerings.

Tags: autoscale, aws, elasticache, memcached, pyhton