No hacks needed
I would recommend using the async module as it simplifies these things.
async provides async.eachSeries as a replacement for async for arr.forEach and allows you to pass the done callback function when it is complete. It will handle each item in the series, as forEach does. In addition, it will conveniently cause errors in your callback, so you do not need to have handler logic inside the loop. If you need / require parallel processing, you can use async.each .
There will be no blocking between the async.eachSeries call and the callback.
async.eachSeries(feedsToFetch, function(feedUri, done) { // call your async function feed(feedUri, function(err, feedArticles) { // if there an error, "bubble" it to the callback if (err) return done(err); // your operation here; articles = articles.concat(feedArticles); // this task is done done(); }); }, function(err) { // errors generated in the loop above will be accessible here if (err) throw err; // we're all done! console.log("all done!"); });
Alternatively, you can build an array of asynchronous operations and pass them to async.series . A series will process your results in a series (not parallel) and call a callback when each function is complete. The only reason to use this over async.eachSeries would be if you prefer the familiar arr.forEach syntax.
// create an array of async tasks var tasks = []; feedsToFetch.forEach(function (feedUri) { // add each task to the task array tasks.push(function() { // your operations feed(feedUri, function(err, feedArticles) { if (err) throw err; articles = articles.concat(feedArticles); }); }); }); // call async.series with the task array and callback async.series(tasks, function() { console.log("done !"); });
Or you can Roll Your Own ™
You may be feeling too ambitious or you may not want to rely on async addiction. Maybe you're just bored, like me. Anyway, I intentionally copied the async.eachSeries API to make it easy to understand how this works.
As soon as we delete the comments here, we have only 9 lines of code that can be reused for any array that we want to process asynchronously! It will not modify the original array, errors can be sent to a “short circuit” iteration, and a separate callback can be used. It will also work on empty arrays. Just a little functionality in just 9 lines :)
// void asyncForEach(Array arr, Function iterator, Function callback) // * iterator(item, done) - done can be called with an err to shortcut to callback // * callback(done) - done recieves error if an iterator sent one function asyncForEach(arr, iterator, callback) { // create a cloned queue of arr var queue = arr.slice(0); // create a recursive iterator function next(err) { // if there an error, bubble to callback if (err) return callback(err); // if the queue is empty, call the callback with no error if (queue.length === 0) return callback(null); // call the callback with our task // we pass `next` here so the task can let us know when to move on to the next task iterator(queue.shift(), next); } // start the loop; next(); }
Now let me create an example of an asynchronous function to use with it. We will fake a delay with setTimeout of 500 ms here.
// void sampleAsync(String uri, Function done) // * done receives message string after 500 ms function sampleAsync(uri, done) { // fake delay of 500 ms setTimeout(function() { // our operation // <= "foo" // => "async foo !" var message = ["async", uri, "!"].join(" "); // call done with our result done(message); }, 500); }
Ok, let's see how they work!
tasks = ["cat", "hat", "wat"]; asyncForEach(tasks, function(uri, done) { sampleAsync(uri, function(message) { console.log(message); done(); }); }, function() { console.log("done"); });
Output (500 ms delay before each output)
async cat ! async hat ! async wat ! done