Geek Answers Handbook

Lucky / Apache Random 503 Errors

I run cPanel on a Rackspace cloud server that hosts about 30 websites. I am using Varnish 3.0.3 + Apache 2.2.23. I get random 503 errors. If I restart the httpd process, they will disappear. I checked Apache error_log and I don't see anything related to error 503.

Here is my Varnish configuration:

backend default { .host = "198.61.161.65"; .port = "8080"; .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; }

Here are my varnish launch options:

 -a :80 -f /etc/varnish/default.vcl -T 127.0.0.1:6082 -t 120 -w 1,1000,120 -u varnish -g varnish -S /etc/varnish/secret -p http_max_hdr=256 -p http_resp_hdr_len=8192 -p http_range_support=on -s malloc,768M

I ran varnishlog and recorded some of the errors. All of them have TxResponse = Service Unavailable in them. For me, this means that Apache does not respond to a request from varnish in a timely manner. Here is an example:

 17 SessionOpen c 74.133.75.136 54227 :80 17 ReqStart c 74.133.75.136 54227 1219297893 17 RxRequest c GET 17 RxURL c / 17 RxProtocol c HTTP/1.1 17 RxHeader c Host: www.archerytrade.org 17 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 17 RxHeader c Accept-Language: en-US,en;q=0.5 17 RxHeader c Accept-Encoding: gzip, deflate 17 RxHeader c DNT: 1 17 RxHeader c Connection: keep-alive 17 VCL_call c recv lookup 17 VCL_call c hash 17 Hash c / 17 Hash c www.archerytrade.org 17 VCL_return c hash 17 VCL_call c miss fetch 17 Backend c 18 default default 17 FetchError c http first read error: -1 0 (Success) 17 VCL_call c error deliver 17 VCL_call c deliver deliver 17 TxProtocol c HTTP/1.1 17 TxStatus c 503 17 TxResponse c Service Unavailable 17 TxHeader c Server: Varnish 17 TxHeader c Content-Type: text/html; charset=utf-8 17 TxHeader c Retry-After: 5 17 TxHeader c Content-Length: 441 17 TxHeader c Accept-Ranges: bytes 17 TxHeader c Date: Wed, 17 Apr 2013 01:39:52 GMT 17 TxHeader c X-Varnish: 1219297893 17 TxHeader c Age: 0 17 TxHeader c Via: 1.1 varnish 17 TxHeader c Connection: close 17 Length c 441 17 ReqEnd c 1219297893 1366162792.398471832 1366162792.541639328 0.000244379 0.143036604 0.000130892 17 SessionClose c error 17 StatSess c 74.133.75.136 54227 0 1 1 0 0 0 257 441

I tried adding the following to the default.vcl file:

 sub vcl_recv { set req.grace = 15s; ... } sub vcl_fetch { if (beresp.status == 500 || beresp.status == 503) { set beresp.saintmode = 10s; return(restart); } set beresp.grace = 1h; ... }

During approximately 503 errors, I discovered a PHP Fatal error, an invalid end to the script headers, and Request exceeded the limit of 10 internal redirects due to a possible configuration error in Apache error_log.

I tried to increase / decrease memory for varnish and timeout value. I tried to add higher values ​​of http_max_hdr and http_resp_hdr_len.

+2
source share
2 answers

We use APC on this server, and the size of the allocated memory is only 96M. After increasing memory several times and tracking usage, we settled on 312M. The cache hit ratio is 99.7%, and fragmentation is 0.2%. Apache 503 errors almost completely stopped.

+1
source

Here is what helped me solve the problem:

After executing the following command, I knew about the error:

 $ varnishlog -b -q 'FetchError'

The error was "- FetchError http first read error: EOF", details can be found on the varnishing site

Then after setting "first_byte_timeout" the problem was fixed:

 backend default { .host = "127.0.0.1"; .port = "8888"; .connect_timeout = 1s; # Wait a maximum of 1s for backend connection (Apache, Nginx, etc...) .first_byte_timeout = 5s; # Wait a maximum of 5s for the first byte to come from your backend .between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent }

Details at Varnish 503 Solutions

0
source

More articles:


All Articles