We have a weird timeout performance problem nginx. The timeout says below:
"upstream timed out(110: Connection timed out)"
Here the upstream server jettyruns on the same host where nginx is also running. Jetty runs on the port 8080where nginx runs on 443.
Checking the above error, I checked the berths and nginx logs. Although the jetty returns a response in less than a second, the response sent by nginx is delayedclose to 60 secs. And nginx is the time at which a timeout warning is triggered after 60 seconds with an HTTP response code 504. This is happening intermittently.
The following are the jetty and nginx logs for the requested time.
Jetty Magazine:
{"EVT": 1494519426927, "intelId": "50", "intelSeq": 112506, "intelVer": "1", "time": "2017-05-11T16: 17: 06.927Z", "uiCorrelationIdV1": "SUI-1494519425839-42047", "threadName": "qtp754853679-357", "wResource": " http://m.xxx.com/search/facet/women/womens-handbags-9780510203/shoulder/_/N -53f3Z7hk9Z1z0roil "," wMethod ":" GET "," wStatus ": 200," wDurationMicros ": 1087801," wJlpLocation ":", "wFwdFor": "Nada", "wHostHdr": "m.xxxx.com", "wReferer": " https://m.xxxx.com/search/facet/women/womens-handbags-9780510203/_/N-53f3Z7hk9?search-term=Bags&sortBy = priceLow & facet = Handbag% 20Style "," wHttpVer ":" HTTP / 1.0 "," wWsgClientIp ":" 80.192.191.2 "," wSrcIP ":" 127.0.0.1 "," wUserAgent ":" Mozilla / 5.0 (iPhone, iPhone, iPhone OS 10_3_1, like Mac OS X) AppleWebKit / 603.1.30 (KHTML, like Gecko) Version /10.0 Mobile / 14E304 Safari / 602.1 "," intelCropped "false" intelLength ": 834}
Nginx log:
2017-05-11T17: 18: 09 + 01: 00 intelId = "56" intelVer = "2" wMethod = "GET" wResource = "//// -9780510203-/-///N-53f3Z7hk9Z1z0roq4Z1z0roil -TERM = &? SortBy = priceLow & = %20Style" wStatus = "504" wCacheStatus = "MISS" wSrcIP = "172.17.233.135" wSize = "176" wDurationSeconds = "60.001" wHostHdr = "m.xxx.com" wReferer = "https://m.xxxx.com/search/facet/women/womens-handbags-9780510203/shoulder//N-53f3Z7hk9Z1z0roil?search-term=Bags&sortBy=priceLow&facet=Handbag%20Style" wSSL = "on" wSSLver = "771" wSSLciph = "TLS1.2-ECDHE-RSA-AES256-GCM-SHA384" wWsgClientIp = "80.192.191.2" wFwdFor = "-" wJlpLocation = "-" wProtocol = "HTTP/1.1" wUpstreamAddr = "127.0.0.1:8080" wPort = 443 s_vi = "[CS] v1 | 2B4D4CA80501261B-600001064000B11D [CE]" s_ppv = "jl% 253Asearch% 2C14% 2C100% 2C5789% 2C414% 2C628% 2C414% 2C736% 2C3% 2CP" User = "true" wUiCorrelationIdV1 = "-" wUserAgent = "Mozilla/5.0 (iPhone, iPhone, iPhone OS 10_3_1, Mac OS X) AppleWebKit/603.1.30 (KHTML, , Gecko) /10.0 Mobile/14E304 Safari/602.1" DeviceType = ""
, 200 2017-05-11T16:17:06.927Z, nginx 2017-05-11T17:18:09+01:00. 60 . , nginx .
- , .
.