Cache a large amount of client-side json result

Question

I have an asp.net mvc application that returns a JSON result containing up to n number of years of data, which are then displayed in a Javascript diagram.

In order to have a good user interface (in terms of performance), I am looking for the best solution whether it is possible to cache JSON data on the client side, so when a user clicks on Chart with various parameters, such as day, week view, etc., those JSON data is requested without getting to the server.

Can someone help us make the best decision on caching best practices regarding whether to cache data on the client side or on the server side, or should we directly access the database to switch each schedule?

Thanks in advance.

Answer 1

First of all, where is the database located? If you are on a local network with a gigabit LAN, then getting into it will not be a problem. However, this is not the case on the Internet. People have limited bandwidth, especially on mobile devices, and so you need to limit your HTTP calls. In addition, fewer HTTP calls mean less server load.

Here are some suggestions:

• Consider pagination

  When downloading "2 years worth" I imagine a lot, for example, the thesis of 100+ pages. Unzip the data, not download it all at once. This saves you bandwidth as well as cache space (if limited).

  How to do it: server script cut data according to what the client wants. It is very easy to create pagination in SQL using LIMIT in a query. The logic is similar to starting_item = (page_needed - 1) * items_per_page

• JSONify Data

  Use JSON to transport data to and from the network. Besides being lightweight, it is also structured. It will be easier to analyze and save later.

  A practical guide. PHP has a json_encode function to convert arrays to JSON strings. I assume your structure has a similar function. Highlight a string on a page, then use JSON.parse to convert from a JSON string to a JS object. JSON methods are included in modern browsers , but if you need to serve older browsers, Crockford has a library to analyze it

• Use the well-known storage infrastructure

  If the cache on one page requires persistent storage, I recently stumbled upon PersistJS , which abstracts localStorage to those available in the browser. Also, here is the JS implementation of LZW . Keep it handy as localstorage uses strings to store data and has a 5-10 MB limit.

  How to convert data to a string using JSON.stringify and save it using PersistJS. Then to retrieve, get the string and JSON.parse() it with JSON.parse()

• Call only when necessary

  If the caching system calls only the server, if something is changed, added or something is not there. If there is data, why should you call it a server?

• Cache sync

  If you are afraid for outdated data, try some AJAX to sync your caching system using some method of extracting live data, as described in this Comet wiki.

The last two points depend on your cache structure. But BackboneJS allows you to synchronize models and collections with the server, which have the same functionality that I talked about.

Answer 2

I did what you are trying to do, and here is my experience. I am using Oracle Site Studio middleware at work. I was looking for a structure that would work with it, but could not find it. So I tried both options below.

1) The database query returns a certain number of rows. I tried 2000 as a test. A simple foreach loop converts the returned data into JSON data. Thus, it literally creates a long array of JSON variables as it moves along the lines. Thus, you imitate the snapshot of the local dababase. JS can access array elements very quickly, and this may surprise you with how quickly you can sort, modify, and delete information.

 <script> var appData = [{'id':'0','first':'Sam','last':'Smith'},{'id':'1','first':'Dan','last':'Smith'}]; </script> 

JSON data is contained in the script tag. Then jQuery on doc.ready reads the data and, if necessary, adds it to the html text. When the user changes the value of the JSON data, ajax fires and saves the changes to the database. It is not difficult to add such a system to your application. I later used Google Angular.js to bind data to the user interface in order to have a clean MV template, as well as easily perform quick sorting and filtering on the client side. As already mentioned, Backbone.js and other JS structures can synchronize client data with the server.

2) The second way I saved the data on the html page is to scroll the returned lines again using foreach. Then I saved the data in HTML using the old fashioned

 <input type="hidden" name="someName" value="someValue" /> 

Then I used jQuery to process the data and add it to the user interface. If you really want to deal with JSON, you can insert it into HTML variables this way

 <input type="hidden name="row1" value="{'color':'red','style':'fancy','age':'44'}" /> 

Then you can use jQuery or Angular.js to process the data and bind it to the user interface.

Interestingly, many application frameworks do not have a built-in client-side caching system. It is really inefficient to sort the server side selection menu and then rebuild the html. Better sort it in JS and dynamically rebuild the selection menu. There is a security issue here, and you do not want to print personal information in JSON or HTML variables, as it is visible in the view source. You can also embed data on pages using more complex methods. Consider below:

 <span class="data" value="1234"></span> $(function () { $('.data').each( function() { var val = $(this).attr('value'); console.log(val); //process data }); }); 

Then you can use jQuery on doc.ready to process classes called data. You can also enter JSON data into a value and parse it later. Keep in mind that jQuery people are against developers using classes in this way. In my experience, if you don’t go overboard, it works great.

Answer 3

• Extract data from the database and save it as static files on the server. Give the extension .css or .png. (The browser automatically caches style sheets and image files.).
• Save the data file name with a timestamp in a hidden field. (to ensure that the last file is downloaded from the server, if there is a change in the file)
• Download the file from the server using AJAX, first it will be downloaded from the server, but the next time it will be downloaded from the browser cache.
• You can use JSON.Parse () to get the result of a Parse AJAX request.