Memcached - how to work with adding / deploying servers

Question

How do you handle replacing / adding / removing memcached nodes in your production applications? I will have many applications that will be cloned and configured due to the fact that each client must work on the same web server, so I guess there will be a day when some of the nodes will be changed.

Here, how memcached is populated in the usual way:

$m = new Memcached(); $servers = array( array('mem1.domain.com', 11211, 33), array('mem2.domain.com', 11211, 67) ); $m->addServers($servers); 

My initial idea is to make the $ servers array populated from the database, also cached, but based on files executed once a day or something like that, with the possibility of forced updating for the next run of a function that contains a call addersers. However, I suggest that this may add some extra overhead, since the drives are rather slow storage ...

What do you think?

Answer 1

It is always better to define such things in the source. Of course, you can use parse_ini_file() or something like that to read the configuration file. Indeed, this belongs to the source. But if defined in an INI file, there aren't as many security issues as injecting code and the like, rather than defining your entries in PHP.

Since there are several applications that can read the same INI file, you can place it in a common place that all applications have access to (for example, using group permissions). This is actually not recommended.

Answer 2

Use a configuration file - be it from parse_ini_file or some other parsing scheme. If you are worried about the parsing time, you can put these configs in the php ini download path http://brian.moonspot.net/using-ini-files-for-php-application-settings - this precedes everything you need, but you you need to restart apache to load any changes.

Once the files have been read enough (until you have too many cracks on your computer), the configuration file will be in the cache with memory anyway, so it will be very fast. You can optimize using a serialized php form that loads quickly and even optimizes using the APC user cache.

Finally, use the latest memcache client libraries - they now use consistent hash algorithms, and this will help add / remove individual servers.

Answer 3

Depending on your requirements, I would use a list that supports automatic eviction after some TTL (passive mode) or some type of listener if you want it to be updated immediately (active mode). Wherever you save your main source (properties file, database, etc.), it is not very important if necessary, and your TTL is large enough (as you mentioned in your comment).

For a passive approach, you can use a repetitive local caching library that supports TTL (for example, EHCache in Java), putting your list of servers in the cache under some known key. For example, for an active approach, Java has java.util.EventListener . This (un), fortunately, has been many years since I did something in PHP, so I can’t recommend it.

I highly recommend not introducing configuration data into the code. In the world I live in, hard coding is a serious sin.

You can also watch how last.fm uses a consistent hash algorithm to add / remove servers to / from the memcached pool without causing a complete reassignment of all keys.

Answer 4

As far as I know, the main problem here is how to cache the configuration, keeping it mostly updated when it changes.

Well, the best option here is obviously to store it in a database and update from the database, say, every 15 seconds. Having a large load on the application, a query to the database every 15 seconds will not change anything. Loading data from the database is pretty fast, since you only need a couple of fields.

Another option that may work here is to use a separate memcached :) Seriously, just cache the configuration downloaded from the database by clearing this cache key when updating the configuration.

To summarize: any temporary expiration scheme will work. The simplest solution is to maintain it yourself (keep the latest update time and check it with every function call); a bit more advanced - use something like memcached.

update : DB is the best solution, because it scales quite well: you do not need to copy configs to 20 servers with every configuration update.

Answer 5

• Use a service locator, for example - https://www.consul.io/ . Add or remove services, and users will receive information about the service from the consul in real time.
• If the consul does more overhead for managing a new piece of software, move the information in a centralized configuration. When updating, reload the configuration or trigger update events. Consumers listen to events and update information.