Recommendations for caching the web interface in front of the site?

Question

Summary

As I look at and around the stackoverflow network, I find that overall there is no good documentation on best practices for caching a high-performance site that uses sessions. It would be helpful if we could share some ideas around some of the basic building blocks, especially around caching. For the purposes of this discussion, I avoid memcache and focus on caching static and fully generated pages.

So, to set up the script, imagine a web server (say nginx), a reverse proxy (say, varnish), an application server (whatever), a db server (say mysql).

Anonymous

• Static elements (gif / jpg, etc.)
• Semi-Dynamics (js / css)
• Dynamic

Recorded by

• Static
• Semi-Dynamics (js / css)
• Dynamic

Generally speaking, all Anon must be cacheable and most are registered (ignore dynamic, without ESI).

Anon # 1

• Set long time is running out
• Install ETag if possible
• Cache-control: max-age = 315360000

Anon # 2 (has a reverse proxy cache result if other Anon # 1 rules are dynamically generated)

• Cache-Control: public, s-maxage = 3000

Anon # 3

• Cache-Control: public, s-maxage = 300

Recorded # 1

• Set long time is running out
• Install ETag if possible
• Cache-control: max-age = 315360000

Recorded # 2 (have a reverse proxy cache result if another is dynamically generated. Rule No. 1 entered into force)

• Cache-Control: public, s-maxage = 3000

Recorded # 3

• Cache-control: s-maxage = 0, must-revalidate

What are your suggestions? I update the message when replies are answered.

Answer 1

I don't know all about caching, but here are some tips:

Anon # 1,2: (static, semi-dynamic elements) You can set them to never end. If you need to change them, change their URL. If-modified, because checks are cheap, but not free.

Anon # 3: (dynamic elements) Here, where ETags and / or Last-Modified are very convenient. Depending on what you service, you can create a good Last-Modified header. If your database stores the modified date of all the items you planned to show, you can do something with the result of SELECT MAX(last_updated) FROM items_to_show . Caveat: This takes into account the age of the data, not the age of your template, so if you changed your django, you don’t understand how to report that in the header.

Or you can do something similar with ETag. This may be a checksum of the content that is generated. This will take into account the template change.

Something to consider with both of these approaches to caching dynamic content is that they really save more bandwidth than downloading server / database on the Internet. You can always use Expires wisely, however, to help in cases where the changes on the page are periodic and predictable.

My login suggestions would be similar, but I would look at the Vary header. This may signal caching of proxy servers that different registered users will not serve the same content.

In general, I would use either ETag or Last-modified, but not both.

Answer 2

My best answer to this question is that you have many options for all static files that can bring many advantages in their own way, each of which is useful in a particular scenario, so weigh the pros and cons according to your specific need.

However, what most people neglect is their dynamic content, strong db caching results, etc., but are still related to running the PHP / ASP parsing engine or whatever.

If you look at the super-cache plugin for wordpress, you will notice that it has the ability to actually prepare your html as static files. Not only that, but it also makes a copy of gzip and uses rewriting rules to check for the presence of these files as a suitable alternative to starting the parser. This will obviously give you a better result, as it not only saves your processing time, but also bandwidth.

If you want to see a performance mismatch, compare the results of apachebench <?php die('hello world'); showing a static .html page.

Obviously, you need to be careful with this kind of caching, but it can be very useful to replace full page caching inside an interpreter such as PHP.

Answer 3

There are related offers on the ySlow page.

Ego may not be a good idea .

Answer 4

I would recommend reading Scalable Internet Architectures. There are several chapters on scaling through caching, CDN, etc. He should point you in the right direction. Helped me expand the site, which I really support.

-