Handling Asynchronous Callbacks with Multiple Calls

Question

I am learning node.js with learnyounode . I have a problem with JUGGLING ASYNC . The problem is described as follows:
You are given three URLs as command line arguments. You must make http.get() calls to retrieve data from these URLs, and then print them in the same order as their order in the argument list. Here is my code:

 var http = require('http') var truecount = 0; var printlist = [] for(var i = 2; i < process.argv.length; i++) { http.get(process.argv[i], function(response) { var printdata = ""; response.setEncoding('utf8'); response.on('data', function(data) { printdata += data; }) response.on('end', function() { truecount += 1 printlist.push(printdata) if(truecount == 3) { printlist.forEach(function(item) { console.log(item) }) } }) }) } 

Here are the questions that I don’t understand: I am trying to save completed data in response.on('end', function(){}) for each URL using a dictionary. However, I do not know how to get the URL for this http.get() . If I can make a local variable inside http.get() , that would be great, but I think that whenever I declare a variable as var url , it will always point to the last url. Since it is global, and it continues to be updated through the cycle. What is the best way to store this completed data as a value with a key equal to the url?

Answer 1

Here is how I could solve the problem.

 #!/usr/bin/env node var http = require('http'); var argv = process.argv.splice(2), truecount = argv.length, pages = []; function printUrls() { if (--truecount > 0) return; for (i = 0; i < pages.length; i++) { console.log(pages[i].data + '\n\n'); } } function HTMLPage(url) { var _page = this; _page.data = '### [URL](' + url + ')\n'; http.get(url, function(res) { res.setEncoding('utf8'); res.on('data', function(data) { _page.data += data; }); res.on('end', printUrls); }); } for (var i = 0; i < argv.length; i++) pages.push(new HTMLPage(argv[i])); 

It adds queries to the array at the beginning of each query, so once this is done, I can iterate well through the answers, knowing that they are in the correct order.

When working with asynchronous processing, I find it much easier to think of each process as something specific beginning and end. If you want the order of the queries to be preserved, a record must be made at the creation of each process, and then you will return to that record at the end. Only then can you ensure that you have things in the correct order.

If you were desperate to use your method above, then you can define a variable inside your get callback closure and use this to store URLs, so you won’t get the last URL rewriting your variables. If you all go this way, you will significantly increase your overhead when you have to use your process.argv urls to access each answer in that order. I would not recommend this.

Answer 2

I went about this a little differently. I create an array of functions that call http.get and immediately call them with their specific context. Streams record an object in which the key is the port of the server to which this stream belongs. When the final event is triggered, it adds a completed array to this server - when this array is full, it iterates through and echoes in the original order provided by the servers.

There is no right way, but there may be a dozen or more ways. I wanted to share mine.

 var http = require('http'), request = [], dataStrings = {}, orderOfServerInputs = []; var completeResponses = []; for(server in process.argv){ if(server >= 2){ orderOfServerInputs[orderOfServerInputs.length] = process.argv[server].substr(-4); request[request.length] = function(thisServer){ http.get(process.argv[server], function(response){ response.on("data", function(data){ dataStrings[thisServer.substr(-4)] = dataStrings[thisServer.substr(-4)] ? dataStrings[thisServer.substr(-4)] : ''; //if not set set to '' dataStrings[thisServer.substr(-4)] += data.toString(); }); response.on("end", function(data){ completeResponses[completeResponses.length] = true; if(completeResponses.length > 2){ for(item in orderOfServerInputs){ serverNo = orderOfServerInputs[item].substr(-4) console.log(dataStrings[serverNo]); } } }); }); }(process.argv[server]); } } 

Answer 3

An Instant Call Function (IIFE) expression may be the solution to your problem. This allows us to bind a specific value to the function, in your case, the URL that receives the response. In the code below, I bind the variable i to index , and therefore, depending on which URL gets the response, this print list index will be updated. See this website for more information.

 var http = require('http') var truecount = 0; var printlist = []; for(var i = 2; i < process.argv.length; i++) { (function(index){ http.get(process.argv[index], function(response) { response.setEncoding('utf8'); response.on('data', function(data) { if (printlist[index] == undefined) printlist[index] = data; else printlist[index]+= data; }) response.on('end', function() { truecount += 1 if(truecount == 3) { printlist.forEach(function(item) { console.log(item) }) } }) }) })(i) }