NodeJS and Asynchronous Hell

Question

I just came to this terrible situation when I have an array of strings, each of which represents a possibly existing file (for example, var files = ['file1', 'file2', 'file3'] . I need to skip these file names and try to find out if it exists in the current directory and if so, stop the loop and forget the rest of the remaining files, so basically I want to find the first existing file of these files and return to the hard-coded message if nothing is found.

This is what I have:

 var found = false; files.forEach(function(file) { if (found) return false; fs.readFileSync(path + file, function(err, data) { if (err) return; found = true; continueWithStuff(); }); }); if (found === false) { // Handle this scenario. } 

This is bad. It blocks (readFileSync), so it slows down.

I cannot just provide callback methods for fs.readFile , it is not so simple because I need to take the first element found ... and callbacks can be called in any random order. I think one way would be to have a callback that increments the counter and stores a list of found / not found information and when it reaches the files.length counter, then checks for the found / not found information and decides what to do next.

This is painful. I see superior performance in the case of I / O, but this is unacceptable. What is my choice?

Answer 1

Do not use synchronization files in a normal server environment - everything is single-threaded, and this completely blocks events while waiting for the results of this io-related loop. Utility CLI = possible, excellent, server = only at startup.

The general library for managing asynchronous flow is https://github.com/caolan/async

 async.filter(['file1','file2','file3'], path.exists, function(results){ // results now equals an array of the existing files }); 

And if you want to say avoid additional calls to path.exists, then you can easily write a function "first" that performed the operation until any test was completed. Like https://github.com/caolan/async#until - but you are interested in leaving.

Answer 2

An asynchronous library is absolutely what you are looking for. It provides almost all types of iterations that you want to use in asynchronous mode. However, you do not need to write your own "first" function. Async already provides a “some” feature that does just that.

https://github.com/caolan/async#some

 async.some(files, path.exists, function(result) { if (result) { continueWithStuff(); } else { // Handle this scenario } }); 

If you or someone else is reading this in the future, you do not want to use Async, you can also make your own basic version of some.

 function some(arr, func, cb) { var count = arr.length-1; (function loop() { if (count == -1) { return cb(false); } func(arr[count--], function(result) { if (result) cb(true); else loop(); }); })(); } some(files, path.exists, function(found) { if (found) { continueWithStuff(); } else { // Handle this scenario } }); 

Answer 3

You can do this without third-party libraries using a recursive function. Pass it an array of file names and a pointer initially set to zero. The function should check for the presence of the specified file (by pointer) of the file name in the array, and in its callback it must either do other material (if the file exists), or increase the pointer and the call itself (if the file does not work does not exist).

Answer 4

Use async.waterfall to control an asynchronous call in node.js, for example: by enabling async-library and using the waterfall call in async:

  var async = require('async'); async.waterfall( [function(callback) { callback(null, taskFirst(rootRequest,rootRequestFrom,rootRequestTo, callback, res)); }, function(arg1, callback) { if(arg1!==undefined ) { callback(null, taskSecond(arg1,rootRequest,rootRequestFrom,rootRequestTo,callback, res)); } } ]) 

Answer 5

(Edit: removed the synchronization clause because it is not a good idea, and we don’t want anyone to copy / paste it and use it in production code, right?)

If you insist on using asynchronous files, I think an easier way to implement this than you described is as follows:

 var path = require('path'), fileCounter = 0; function existCB(fileExists) { if (fileExists) { global.fileExists = fileCounter; continueWithStuff(); return; } fileCounter++; if (fileCounter >= files.length) { // none of the files exist, handle stuff return; } path.exists(files[fileCounter], existCB); } path.exists(files[0], existCB);