Solution for streaming JSON using oboe.js in AngularJS?

Question

I'm new to Angular, so maybe I'm asking for the impossible, but anyway, here is my task.

Since our server cannot paginate JSON data, I would like to pass JSON and add it page by page to the controller model. The user does not need to wait for the entire stream to load, so I am updating the view for each X record (classified).

I found oboe.js to parse the JSON stream and added it using the gazebo to my project. (bower install oboe --save).

I want to update the controller model during streaming. I did not use the qq implementation of pomises $ q, because there is only one .resolve (...), and I want several pages of data to be loaded through the stream, so $ digest needs to be called from each page. A recovery service called: / service / tasks / search

I created a factory with a search function that I call from the controller:

'use strict'; angular.module('myStreamingApp') .factory('Stream', function() { return { search: function(schema, scope) { var loaded = 0; var pagesize = 100; // JSON streaming parser oboe.js oboe({ url: '/service/' + schema + '/search' }) // process every node which has a schema .node('{schema}', function(rec) { // push the record to the model data scope.data.push(rec); loaded++; // if there is another page received then refresh the view if (loaded % pagesize === 0) { scope.$digest(); } }) .fail(function(err) { console.log('streaming error' + err.thrown ? (err.thrown.message):''); }) .done(function() { scope.$digest(); }); } }; }); 

My controller:

 'use strict'; angular.module('myStreamingApp') .controller('MyCtrl', function($scope, Stream) { $scope.data = []; Stream.search('tasks', $scope); }); 

All seams work. After a while, the system slows down and the http call does not end after updating the browser. In addition, the browser (chrome) crashes when too many records are loaded. Perhaps I am mistaken because passing the area to the factory search function is not "correct", and I suspect that calling $ digest in this area gives me problems. Any ideas on this are welcome. Especially if you have an idea for its implementation, where the factory (or service) can return the promise, and I could use

 $scope.data = Stream.search('tasks'); 

in the controller.

Answer 1

I went a little deeper and came up with the following solution. This might help someone:

factory (named Stream) has a search function that passes parameters for an Ajax request and a callback function. A callback is called for each page of data loaded by the stream. The callback function is called using delayed play, so the area can be automatically updated from every page. To access the search function, I use a service (called Search) that initially returns an empty data array. As the stream goes, factory calls the callback function passed by the service, and the page is added to the data.

Now I can call the search service form in the controller and assign the return value to the data array of the regions.

Service and factory:

 'use strict'; angular.module('myStreamingApp') .service('Search', function(Stream) { return function(params) { // initialize the data var data = []; // add the data page by page using a stream Stream.search(params, function(page) { // a page of records is received. // add each record to the data _.each(page, function(record) { data.push(record); }); }); return data; }; }) .factory('Stream', function($q) { return { // the search function calls the oboe module to get the JSON data in a stream search: function(params, callback) { // the defer will be resolved immediately var defer = $q.defer(); var promise = defer.promise; // counter for the received records var counter = 0; // I use an arbitrary page size. var pagesize = 100; // initialize the page of records var page = []; // call the oboe unction to start the stream oboe({ url: '/api/' + params.schema + '/search', method: 'GET' }) // once the stream starts we can resolve the defer. .start(function() { defer.resolve(); }) // for every node containing an _id .node('{_id}', function(node) { // we push the node to the page page.push(node); counter++; // if the pagesize is reached return the page using the promise if (counter % pagesize === 0) { promise.then(callback(page)); // initialize the page page = []; } }) .done(function() { // when the stream is done make surethe last page of nodes is returned promise.then(callback(page)); }); return promise; } }; }); 

Now I can call the service from the controller and assign a service response to the scope:

 $scope.mydata = Search({schema: 'tasks'}); 

August 30, 2014 Patch

I created the angular -oboe module with the solution above, a little structured. https://github.com/RonB/angular-oboe