Geek Answers Handbook

How does an EC2 application automatically detect ElastiCache instances?

Say I have a webapp running on a number of load-balanced EC2 servers, storing and retrieving metadata from SimpleDB with large chunks of data stored on S3 (due to only 1 KB of SimpleDB limitation). Since S3 is quite high latency, and I do not want there to be many requests, I want the cache level for the information ... introduced ElastiCache.

Okay, that's why I provide the ElastiCache server to the X endpoint, so I hard-code X to my EC2 application and it works happily until I get a few hundred thousand new users, and suddenly my cache server greatly underestimated the demand. Fortunately, I can just start several new cache servers ... but then I realized that I have endpoints X, Y and Z, and my application knows that I need to try X, so I still have a problem.

So right now, I'm just trying to spin my head wrapped around different parts of this puzzle, and I haven't gotten the coding part yet, but isn't that a problem? I read the documentation for ElastiCache and it mentions that it is a cache cluster, but each server in the cluster seems to have its own endpoint. Is there a way for an application running on EC2 to know about all running cache servers, as well as which one contains data for a particular key? Is it possible to ask the cluster as a whole to store or retrieve some of the information?

+7
source share
3 answers

Aws announced cache discovery today. Your problem has been resolved. http://aws.typepad.com/aws/2012/11/amazon-elasticache-now-with-auto-discovery.html .

+3
source

If your application is deployed from source control (I hope so), you simply edit the configuration file and redeploy the application. I do not see a huge problem with this approach, but maybe I am missing the obvious.

Tell me.

0
source

Amazon Elasticache Autodiscovery is absolutely terrible. It is almost impossible to establish what is crazy, because it should be very simple.

I wrote a simple function in PHP to create the URL of an elastic node page, given the number of nodes you started. Yes, you need to update your code if you change the number of nodes (or perhaps put this value in env var).

It maps the same keys to the same nodes:

function get_elasticache_node_url( $key, $config_url, $num_nodes ) { $node = hexdec( substr( md5($key), 0, 15) ) % $num_nodes + 1; $nodestr = str_pad($node, 4, "0", STR_PAD_LEFT); return str_replace('.cfg.','.'.$nodestr.'.',$config_url); } $num_nodes = 10; $config_url = 'cluster-name.xyzxyz.cfg.use1.cache.amazonaws.com'; echo get_elasticache_node_url("key1", $config_url, $num_nodes ); echo get_elasticache_node_url("key2", $config_url, $num_nodes );

Output:

 cluster-name.xyzxyz.0001.use1.cache.amazonaws.com cluster-name.xyzxyz.0004.use1.cache.amazonaws.com
0
source

More articles:


All Articles