Efficient data reload / transfer data from server to client

Question

I’m looking for a “way to go” (that is, the most efficient, most used, generally accepted way) when it comes to overloading data from a web server to the forefront. In the final application, I will have several output fields into which the data should be written, for example:

Data streams will differ from each other in the final application. Lines should be reloaded with fresh, updated data from the server.

I was thinking about using Ajax requests for updates, like every second, but there should be a different way to do this. Ajax requests will cause a lot of data traffic. In addition, when using Facebook chat, you don’t have to wait every second, chats are received almost instantly. However, I do not see any Ajax polls when I use the Mozilla Firefox developer tools. It made me wonder if there would be another way to do this.

I looked through Node.js but it seems like this is not possible with my host.

I heard people talking about Ajax Push, is that what I should use? If so, can you give me a basic use case?

If not, what will happen when you need to have several data streams that need to be reloaded in a second?

Requirements are speed and low data traffic. Therefore, in my opinion, this will not be an opportunity to constantly poll the server, because it will create huge overhead.

^{I don't think this makes any difference, but I use PHP5.3 at the end and JavaScript with jQuery 1.9.1 in the interface.}

Answer 1

This question was asked several times, but somewhat differently. Here are some links to read:

• What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet?
• Using comet with PHP?
• Apache with Comet support
• Server Scalability - HTML 5 vs. Comet Web Sites
• How to implement event listener in PHP

In conclusion : if you want to create your solution using PHP on Apache, then maintaining open persistent connections (long polling or streaming HTTP message) will use resources very quickly (very inefficient). Thus, you are better off using a hosted solution (* disclaimer - I work with a hosted solution).

HTTP long polling and streaming HTTP streams are solutions that have been replaced by Server-Sent and WebSockets events. So, where possible (where the web client provides support), you should use one of these solutions before returning to the HTTP solution. Good real-time web technology will automatically handle this for you.

Since your chart shows that you are subscribing to multiple data streams, you should also consider publishing / subscribing , which naturally matches that. Again, a good real-time web technical solution will provide you with this.

Also see the real-time web technology guide .

Answer 2

I think what you are looking for is usually called Comet . This method was often used to work:

• The client (web browser) makes a request to the server for new data. This is not a page reload, but rather JavaScript.
• The server responds when it has some data for the client. Again, this does not affect the user interface, since the page itself is not overloaded: the data loaindg is executed "in the background", so to speak, in JavaScript code.
• On the service side, the request waits for new data and returns new data when it is available, or returns nothing if the wait interval (defined on the server) is reached. This timeout is usually set below the HTTP browser timeout. The reason for this is because the server can find out if a particular client received a particular piece of data. If the request allows a timeout on the client side, the server may respond to the original request after the client has timed out, and the client will not receive data, even if the server believes that it did.

Data is usually transmitted as JSON, but you can choose whatever encoding you want. See here for an example of how to do this. Goosh is another example of this technique, and therefore Interactive Python Shell . Available code for everyone.

---

On the PHP side, you'll want to create a page that will respond to these "background" comet JavaScript requests. This may be the same page as the user downloading, but, say, this is different, for the convenience of explanation. Thus, the user loads index.php , and the JavaScript comet code calls getNewData.php to retrieve new data.

In your getNewData.php you will need to wait for your event and return the data. You do not want to use a poll for this, but there are PHP libraries that allow you to use various strategies for interaction between processes to wait for events, see this question, for example . The high-level pseudo-code for your getNewData.php will look like this:

• parse a json request
• Enter an effective wait state (with a timeout) while waiting for the "new data available" event
• Was there a previous timeout?
  Yes: send a response indicating lack of data
  No: send a response with new data