Knockout.js is incredibly slow under semi-large datasets

Question

I'm just starting out with Knockout.js (I always wanted to try it, but now I have an excuse!). However, I ran into some very bad performance issues when linking a table to a relatively small data set (about 400 rows or so).

In my model, I have the following code:

this.projects = ko.observableArray( [] ); //Bind to empty array at startup this.loadData = function (data) //Called when AJAX method returns { for(var i = 0; i < data.length; i++) { this.projects.push(new ResultRow(data[i])); //<-- Bottleneck! } }; 

The problem is that the for loop above takes about 30 seconds or about 400 lines. However, if I changed the code to:

 this.loadData = function (data) { var testArray = []; //<-- Plain ol' Javascript array for(var i = 0; i < data.length; i++) { testArray.push(new ResultRow(data[i])); } }; 

Then the for loop ends instantly. In other words, the push method of the Knockout observableArray incredibly slow.

Here is my template:

 <tbody data-bind="foreach: projects"> <tr> <td data-bind="text: code"></td> <td><a data-bind="projlink: key, text: projname"></td> <td data-bind="text: request"></td> <td data-bind="text: stage"></td> <td data-bind="text: type"></td> <td data-bind="text: launch"></td> <td><a data-bind="mailto: ownerEmail, text: owner"></a></td> </tr> </tbody> 

My questions:

• Is this the right way to bind my data (that comes from the AJAX method) to the observable collection?
• I expect push to do heavy recounts every time I call it, for example, it is possible to rebuild related DOM objects. Is there a way to either delay this recalc, or maybe click on all my items at once?

I can add more code if necessary, but I'm sure it matters. For the most part, I just followed Nockout tutorials from the site.

UPDATE:

In the tip below, I updated my code:

 this.loadData = function (data) { var mappedData = $.map(data, function (item) { return new ResultRow(item) }); this.projects(mappedData); }; 

However, this.projects() still takes about 10 seconds for 400 lines. I admit I'm not sure how fast it will be without Knockout (just adding lines via the DOM), but I have a feeling that it will be much faster than 10 seconds.

UPDATE 2:

In another tip below, I gave jQuery.tmpl a snapshot (which is initially supported by KnockOut), and this template engine will draw about 400 lines in just 3 seconds. This seems like a better approach, except for a solution that will dynamically load more data as it scrolls.

Answer 1

As stated in the comments.

The knockout has its own template engine associated with bindings (foreach, with). It also supports other template engines, namely jquery.tmpl. Read more here . I did not benchmark with different engines, so I don’t know if this will help. Reading your previous comment, in IE7 you can try to get what you are after.

Aside, KO supports any js template engine if someone wrote an adapter for it. You might want to try others, as jquery tmpl should be replaced with JsRender .

Answer 2

See: Knockout.js Gotcha Performance # 2 - Manipulating Observed Arrays

The best template is to get a link to our base array, click on it, and then call .valueHasMutated (). Now, our subscribers will receive only one notification indicating that the array has changed.

Answer 3

Use pagination with KO in addition to using $ .map.

I had the same problem with large data sets of 1400 records, until I used paging with knockout. Using $.map to load records really mattered, but the DOM rendering time was still disgusting. Then I tried using pagination, and it made my dataset fast, as good as it was more user friendly. A page size of 50 made the dataset much less overwhelming and drastically reduced the number of DOM elements.

It is very easy to do with KO:

http://jsfiddle.net/rniemeyer/5Xr2X/

Answer 4

KnockoutJS has some great tutorials, particularly about loading and saving data.

In their case, they retrieve data using getJSON() , which is very fast. From their example:

 function TaskListViewModel() { // ... leave the existing code unchanged ... // Load initial state from server, convert it to Task instances, then populate self.tasks $.getJSON("/tasks", function(allData) { var mappedTasks = $.map(allData, function(item) { return new Task(item) }); self.tasks(mappedTasks); }); } 

Answer 5

Give KoGrid an appearance. It intelligently controls the rendering of strings to make it more efficient.

If you are trying to link 400 rows to a table using the foreach binding, you will have problems getting a lot to do through KO in the DOM.

KO does some very interesting things using foreach , most of which are very nice operations, but they begin to break down into perf as your array grows in size.

I was on a long dark path trying to bind large datasets to tables / grids, and you eventually need to split / print the data locally.

KoGrid does it all. It was created to display only those lines that the viewer can see on the page, and then virtualize the remaining lines until they are needed. I think you will find his performance for 400 items much better than you worry.

Answer 6

The solution to avoid blocking the browser when rendering a very large array is to "throttle" the array so that only a few elements are added at a time, with a sleep between them. Here's a function that will do just that:

 function throttledArray(getData) { var showingDataO = ko.observableArray(), showingData = [], sourceData = []; ko.computed(function () { var data = getData(); if ( Math.abs(sourceData.length - data.length) / sourceData.length > 0.5 ) { showingData = []; sourceData = data; (function load() { if ( data == sourceData && showingData.length != data.length ) { showingData = showingData.concat( data.slice(showingData.length, showingData.length + 20) ); showingDataO(showingData); setTimeout(load, 500); } })(); } else { showingDataO(showingData = sourceData = data); } }); return showingDataO; } 

Depending on your use case, this can lead to a significant improvement in UX, as the user can see only the first batch of lines before scrolling.

Answer 7

Using the accept () accepting () arguments that the variables give, in my case there was better performance. 1300 rows were loaded for 5973 ms (~ 6 sec.). With this optimization, the boot time decreased to 914 ms (<1 s)
This is an 84.7% improvement!

Additional Information on Clicking Elements on the Observed Array

 this.projects = ko.observableArray( [] ); //Bind to empty array at startup this.loadData = function (data) //Called when AJAX method returns { var arrMappedData = ko.utils.arrayMap(data, function (item) { return new ResultRow(item); }); //take advantage of push accepting variable arguments this.projects.push.apply(this.projects, arrMappedData); }; 

Answer 8

I dealt with such huge volumes of data that valueHasMutated brought to me as a charm.

Show model:

 this.projects([]); //make observableArray empty --(1) var mutatedArray = this.projects(); -- (2) this.loadData = function (data) //Called when AJAX method returns { ko.utils.arrayForEach(data,function(item){ mutatedArray.push(new ResultRow(item)); -- (3) // push to the array(normal array) }); }; this.projects.valueHasMutated(); -- (4) 

After calling (4) , the array data will be loaded into the required observable array, which is this.projects automatically.

if you have time to look at it and just in case of any problems let me know

The trick here is:. Thus, if in the case of any dependencies (calculated, subscribers, etc.) you can avoid push, and we can force them to execute in one pass after the call (4) .

Answer 9

I experimented with performance and have two contributions that I hope can be useful.

My experiments focus on DOM manipulation time. Therefore, before going into this, it is definitely worth following the points above about entering JS into an array before creating an observable array, etc.

But if the DOM manipulation time is still bothering you, this might help:

---

1: a template to wrap the boot fairing around slow rendering and then hide it using afterRender

http://jsfiddle.net/HBYyL/1/

This is actually not a problem for the performance problem, but shows that a delay is probably inevitable if you loop over a thousand elements and use a template in which you can make sure that you have a load counter before the long KO operation, then hide him afterwards. Thus, it improves UX, at least.

Make sure you can load the counter:

 // Show the spinner immediately... $("#spinner").show(); // ... by using a timeout around the operation that causes the slow render. window.setTimeout(function() { ko.applyBindings(vm) }, 1) 

Hide counter:

 <div data-bind="template: {afterRender: hide}"> 

which causes:

 hide = function() { $("#spinner").hide() } 

---

2: Using html binding as a hack

I remembered the old technique from the moment when I worked on the console with Opera, creating a user interface using DOM manipulation. This was terribly slow, so the solution was to store large chunks of HTML as strings and load the strings by setting the innerHTML property.

Something similar can be achieved with an html binding and a computed one that outputs the HTML for the table as a large piece of text and then applies it at a time. This fixes the performance issue, but a huge drawback is that it severely limits what you can do with the binding inside each row of the table.

Here's a script that shows this approach, along with a function that can be called from within the rows of a table to remove an element with an undefined KO-like way. Obviously, this is not as good as the right KO, but if you really need powerful (ish) performance, this is a possible workaround.

http://jsfiddle.net/9ZF3g/5/

Answer 10

A possible collaboration in conjunction with jQuery.tmpl is to simultaneously move elements to the observable array asynchronously using setTimeout;

 var self = this, remaining = data.length; add(); // Start adding items function add() { self.projects.push(data[data.length - remaining]); remaining -= 1; if (remaining > 0) { setTimeout(add, 10); // Schedule adding any remaining items } } 

Thus, when you add only one item at a time, the /knockout.js browser can spend its time manipulating the DOM accordingly, without blocking the browser for several seconds, so that the user can scroll through the list at a time.

Answer 11

I also noticed that the IE mechanism for js knockout works slower in IE, I replaced it with underscore.js, it works faster.

Answer 12

If you are using IE, try closing the dev tools.

Having developer tools open in IE greatly slows down this operation. I add ~ 1000 elements to the array. When Dev tools open, it takes about 10 seconds, and IE freezes when this happens. When I close the dev tools, the operation is instantaneous and I do not see any slowdown in IE.