The reason the browser does not show X-Forwarded-For headers

Question

Note Please read the full question.

I'm trying to understand why browsers don't show me the X-Forwarded-For header on every page request

By the way, my request headers look like

Request URL:http://localhost:3000/users/sign_in Request Method:GET Status Code:304 Not Modified 

Request Header:

 Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Encoding:gzip,deflate,sdch Accept-Language:en-GB,en-US;q=0.8,en;q=0.6 Cache-Control:max-age=0 Connection:keep-alive Cookie:undefined=0; poasterapp=s%3A4faaa6b1723e7c6fbd949083532c52598652547b.sNX%2BKOEed2TEQkQN7I7K5lgpoHMRpwerKFvUegMnTVI; _minerva_session=BAh7CUkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgphbGVydFQ6DEBjbG9zZWRGOg1AZmxhc2hlc3sGOwpJIgAGOwBUOglAbm93MEkiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--6bd89ce9d29e9bdcf56573f9a153dc663a8fe755 Host:localhost:3000 If-None-Match:"785d34e3998360353567fc710af123fb" User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36 6bd89ce9d29e9bdcf56573f9a153dc663a8fe755 Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Encoding:gzip,deflate,sdch Accept-Language:en-GB,en-US;q=0.8,en;q=0.6 Cache-Control:max-age=0 Connection:keep-alive Cookie:undefined=0; poasterapp=s%3A4faaa6b1723e7c6fbd949083532c52598652547b.sNX%2BKOEed2TEQkQN7I7K5lgpoHMRpwerKFvUegMnTVI; _minerva_session=BAh7CUkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgphbGVydFQ6DEBjbG9zZWRGOg1AZmxhc2hlc3sGOwpJIgAGOwBUOglAbm93MEkiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--6bd89ce9d29e9bdcf56573f9a153dc663a8fe755 Host:localhost:3000 If-None-Match:"785d34e3998360353567fc710af123fb" User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36 

Answer headers (not needed, but still)

 Cache-Control:max-age=0, private, must-revalidate Connection:close ETag:"785d34e3998360353567fc710af123fb" Server:thin 1.5.0 codename Knife Set-Cookie:_minerva_session=BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--dfb3ce9f5c97463cfcd0229a133654e6cc606d98; path=/; HttpOnly X-Request-Id:41a6f3062dc8bc36b7b3eae71dc5075d X-Runtime:89.238257 X-UA-Compatible:IE=Edge 3D% 3D - dfb3ce9f5c97463cfcd0229a133654e6cc606d98; Cache-Control:max-age=0, private, must-revalidate Connection:close ETag:"785d34e3998360353567fc710af123fb" Server:thin 1.5.0 codename Knife Set-Cookie:_minerva_session=BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--dfb3ce9f5c97463cfcd0229a133654e6cc606d98; path=/; HttpOnly X-Request-Id:41a6f3062dc8bc36b7b3eae71dc5075d X-Runtime:89.238257 X-UA-Compatible:IE=Edge 

Now, as said, I don’t see X-Forwarded-For in the request headers

Reading the X-Forwarded-For wiki pages makes me feel like it's something done by server caching (which in my case I consider my ISP) so am I safe to believe that the **X-Forwarded-For** headers is something that is added at the caching server side (ISP provider)

If yes , this is the one that deceives me, and this

why? - this is the same (i.e. X-Forwarded-For does not appear in the request headers) for my server running locally on my machine and I access them through a browser, for example http://localhost:3000

Answer 1

X-Forwarded-For is not a standard request header, as specified in RFC 2616 Section 5.3 , which addresses standard protocol request headers that (as specified in the RFC)

• To accept
• Accept-charset
• Accept-encoding
• Accept language
• Resolution
• Expect
• FROM
• Host
• If-match
• If-Modified-Since
• If-none-match
• If-range
• If-Unmodified-C
• Max Forwards
• Proxy-authorization
• Range
• Referer
• TE
• User agent

For your inbound request to have a custom [X-Forwarded-For] header, it must be explicitly added to this request by the calling client. The simplest explanation for why you do not see this header is that the client sending the request did not manually add it.

The difficulty is that the header that you expect to see is not the header that you should expect to receive if there is no contract between your service and the subscriber that differs from HTTProtocol, indicating that you should expect the X- value Forwarded-For will be indicated in your request header. As others have already pointed out, the XFF header is usually set by a proxy server or by a load balancer to indicate who the real requestor is, who acts through his proxy.

As a service provider, if you require that all requests be set to the [X-Forwarded-For] header, you must enforce it at the service policy level. If you do not want to service proxy server accounts that do not identify who they are protecting their IP address, scan their request with a forbidden 403. If you are in a situation where you should service these requests, but depend on this set header, then you have to come up with your own process in which you could report an error.

Here is what HTTProtocol has to say about anonymity:

Because the source of the link may be personal information or it is possible to identify otherwise a private source of information, it is highly recommended that the user can select the Sender field is sent. For example, a browser client may have a toggle switch for viewing openly / anonymously, which accordingly enable / disable sending Referer and From information.

Clients MUST NOT include the Referer header field in (insecure)

HTTP request if the link page has been migrated using the secure protocol.

Authors of services that use the HTTP protocol MUST NOT use GET
based on the form for the presentation of confidential data, as this will force this data to be encoded in the Request-URI. Many existing servers, proxies, and user agents will register request URIs in some
where it can be seen by third parties. Servers can use Instead, send a request based on POST ...

Design custom accept header fields sent in each request, in particular if they include quality values, can be used by servers as relatively reliable and durable user identifiers. Such user identifiers will allow content providers to track click tracking, and will allow collaborating content providers to map cross-click paths or presentation forms of individual users. Note that for many users not behind the proxy, the host network address of the launch user agent will also serve as a long-term user identifier. In environments where proxies are used to improve privacy, user agents should be conservative, offering to accept header configuration settings for end users. As an extreme measure of privacy, proxies can filter adoption headers in relayed requests. General-purpose user agents that provide a high degree of header Configurability SHOULD warn users about the loss of privacy that may be involved.

Personally, I would bounce the request using 401.2 and send the requestor to the request screen through the WWW-Authenticate response header, which notifies them that they will not be allowed anonymous access to your site. This is a kind of obscene way to use the WWW-Authenticate header, but it looks like you expect the X-Forwarded-For header to confirm and identify the real requester and allow access to public non-anonymous ones at your service. For me, this is an authentication problem.

Answer 2

• ISP providers do not add X-Forwarded-For.
• X-Forwarded-For is not for the end user to identify the application behind the proxy / balancer.
• X-Forwarded-For is intended for use behind a proxy / balancer to identify the user.

For example: You have a web application (php, java, etc.), you also have an http server (Apache, nginx, etc.), then:

• The user makes a request to the http server.
• Request to redirect the Http server to a web application using X-Forwarded-For for user ip.
• Your web application knows that it is behind the http server, so it reads X-Forwarded-For as a user ip.

Answer 3

Why do you expect X-Forwarded-For to appear in the first place? You are connecting to a web server running on localhost , so there is no ISP at all. Even if you connected to your web server through your ISP, you can still hardly add X-Forwarded-For to your requests. X-Forwarded-For typically added by an HTTP proxy or load balancer, none of which pass. X-Forwarded-For never included in a web browser.