How to sort a JS array in batches (for performance)

I have a JS application that needs to execute the complex look of a large array and then display it. Using the built-in array.sort(cb) method can take up to 1 second with my data. It's long enough for my user interface to get noisy.

Since the user interface is tall enough to display a subset of the sorted array on the screen and the rest are below scrolling or paginated, I had an idea. What if I made an algorithm that went through a large array and quickly did a sort so that the top N elements were perfectly sorted, but the rest of the elements in the array were incorrectly sorted. Every time I run my algorithm, it would sort a little more array from top to bottom.

So I could break my processing into pieces and have a smooth interface. For the first few seconds, the array will not be perfectly sorted, but the flaws will be below the scroll bar so that they are not noticed.

My naive solution would be to write my own "Selection Sort" with the possibility of breaking after N matches and resuming later, but "Selection Sort" is a pretty terrible algorithm. Faster algorithms (in my opinion) should be completed to ensure that the top N elements are stable.

Does anyone know of an existing solution for this? I'm crazy? Any suggestions?

UPDATE

Taking the idea suggested by @moreON, I wrote a custom QuickSort that resets as soon as it has the required accuracy. For this data, its own form took 1 sec. Regular QuickSort took about 250 ms, which is already surprisingly better. QuickSort, which helps out after the first 100 items are sorted, took more than 10 ms, which is much better. Then I can take an extra 250 ms to finish the sort, but that doesn't really matter, because the user is already looking at the data. This reduces the user delay from 1 sec to 10 ms, which is pretty good.

 //Init 1 million random integers into array var arr1 = []; var arr2 = []; for(var i=0;i<1800000;i++) { var num = Math.floor(Math.random() * 1000000); arr1.push(num); arr2.push(num); } console.log(arr1); //native sort console.time("native sort"); arr1.sort(function(a,b) { return ab; }); console.timeEnd("native sort"); //1sec console.log(arr1); //quicksort sort Ref: https://www.nczonline.net/blog/2012/11/27/computer-science-in-javascript-quicksort/ function swap(arr, a, b) { var temp = arr[a]; arr[a] = arr[b]; arr[b] = temp; } function cmp(a,b) { return (a<b); } function partition(items, left, right) { var pivot = items[Math.floor((right + left) / 2)]; var i = left; var j = right; while (i <= j) { while (cmp(items[i],pivot)) i++; while (cmp(pivot,items[j])) j--; if (i <= j) { swap(items, i, j); i++; j--; } } return i; } function quickSort(items, left, right, max) { if(max && left-1 > max) return items; //bail out early if we have enough if (items.length > 1) { var index = partition(items, left, right); if (left < index - 1) quickSort(items, left, index - 1, max); if (index < right) quickSort(items, index, right, max); } return items; } //sort first 100 console.time("partial Quicksort"); arr2 = quickSort(arr2,0,arr2.length-1,100); console.timeEnd("partial Quicksort"); //10ms console.log(arr2); //sort remainder console.time("finishing Quicksort"); arr2 = quickSort(arr2,100,arr2.length-1); //250ms console.timeEnd("finishing Quicksort"); console.log(arr2); 
+5
source share
4 answers

If you want a heapify array , which I believe can be done in O(n) time ( https://en.wikipedia.org/wiki/Binary_heap#Building_a_heap ), you could extract each element of N so that in O(N log n) time ( N became less during extraction).

+2
source

Here is a cleaned version of my solution that sorts a large array in batches, so the JS thread doesn't stutter. In my example here, it takes 1 second of array.sort(cb) and turns it into five separate 100 ms operations. You will want to choose pageSize wisely based on your data. More pages will make the final look busy longer, fewer pages will make parties longer.

 var BatchedQuickSort = { swap: function(arr, a, b) { var temp = arr[a]; arr[a] = arr[b]; arr[b] = temp; }, partition: function(items, left, right, cmp) { var pivot = items[Math.floor((right + left) / 2)]; var i = left; var j = right; while (i <= j) { while (cmp(items[i],pivot)<0) i++; while (cmp(pivot,items[j])<0) j--; if (i <= j) { this.swap(items, i, j); i++; j--; } } return i; }, sort: function(items, cmp, max, left, right) { //Ref: https://www.nczonline.net/blog/2012/11/27/computer-science-in-javascript-quicksort/ if (items.length > 1) { left = typeof left != "number" ? 0 : left; right = typeof right != "number" ? items.length - 1 : right; var index = this.partition(items, left, right, cmp); if (left < index - 1) this.sort(items, cmp, max, left, index - 1); if (index < right && (!max || index<=max)) this.sort(items, cmp, max, index, right); } return items; } } //Example Usage var arr = []; for(var i=0;i<2000000;i++) arr.push(Math.floor(Math.random() * 1000000)); function myCompare(a,b) { return ab; } var pageSize = Math.floor(arr.length/5); var page = 1; var timer = window.setInterval(function() { arr = BatchedQuickSort.sort(arr, myCompare, pageSize*page,pageSize*(page-1)); if(page*pageSize>=arr.length) { clearInterval(timer); console.log("Done",arr); } page++; },1); 
+1
source

I think your question comes down to the following:

How to find the top N elements in a large array

which can be answered here: Find the top N elements in the array

This can be solved by going once, and simply select the top N elements. Θ (n).

Have a look here: https://jsfiddle.net/jeeh4a8p/1/

 function get_top_10(list) { var top10 = new Array(10).fill(0) for(i in list) { var smallest_in_top10 = Math.min.apply( Math, top10 ) if(list[i] > smallest_in_top10) { top10.splice(top10.indexOf(smallest_in_top10),1) top10.push(list[i]) } } return top10 } console.log(get_top_10([1,2,3,4,5,6,7,8,9,10,11,12])) var random_list = []; for (var i = 0; i < 100; i++) { random_list.push(Math.round(Math.random() * 999999)) } console.log(get_top_10(random_list)) function sortNumber(a,b) { return a - b; } 
0
source

First of all, take a look at the prospects for improving performance. Effective sorting algorithms are O (N * log2 (N)). For N = 1,000,000 elements, N * log2 (N) ~ N * 20. I am sure that you have many elements that you are trying to display on a web page.

If you only need to display the first 25 rows, the Selection Sort will accept N * 25 to order them, so that would be really worse if we assume that there is comparable fixed overhead.

If you want to experiment further with this, one of the algorithms that I can think of is this: maintain a binary tree of the PAGE_SIZE smallest elements. Keep updating it in a single pass on the data, deleting the largest items when smaller ones are found. Ignoring the rebalancing, you will need N * log2 (PAGE_SIZE) to populate the tree and display the first page of results.

-1
source

All Articles