Description of the problem -
We use a router between the Internet client and the downstream service. The router (service) is written in Node.js. His responsibility is to forward the Internet client request to the appropriate downstream service and return the response to the Internet client. We are facing some delay at the router level.
The library used for http-proxy is
https://github.com/nodejitsu/node-http-proxy
node-http-proxy Example -
Explanation with an example example. We run into a problem at 99 percentiles -
We performed load / performance testing with 100 concurrency.
As a result, up to 95 percent response time is great for an Internet client.
But at 99 percentiles, the Downstream service responds at the expected time (~ 250 ms). But the router takes 10 times longer than expected (~ 2500 ms).
Service Information -
Both routers and downstream services are in the same region and on the same subnet. Thus, this delay is not related to the network.
The possibilities of this delay are
- Some threads are blocked at the node service level. That's why, unable to listen to service response requests.
- More time finding dns.
- Node counts the number of threads less, which is why it is not able to listen to the entire incoming response from the downstream service.
To analyze this -
We were shaking below the configurations -
keepAlive, maxSockets, maxFreeSockets, keepAliveMsecs, log level. Verifying the PLease node configuration for the http / https agent - configuring the http agent
Node service code snippet -
var httpProxy = require('http-proxy'); var http = require('http'); var https = require('https'); var agent = new https.Agent({ maxSockets: nconf.get(25), keepAlive: true, maxFreeSockets: nconf.get(10), keepAliveMsecs : nconf.get(5000) }); var proxy = httpProxy.createServer({ agent: agent }); var domain = require('domain'); var requestTimeout = parseInt(nconf.get('REQUEST_TIMEOUT')); process.env.UV_THREADPOOL_SIZE = nconf.get(4);
Questions -
- I am new to node services. It would be great if you could help me configure the above configurations with the correct values. If I missed any configuration, please let me know?
- Is there a way to intercept network traffic on a router that will help me analyze the aforementioned 10x router latency?
- If you know any profiling tool (network level) that can help me dive more, please share with me?
[Update 1]
I found one interesting link - a military story .
If I missed any required information here, please ask me to add here.