How to transfer calls of async function to synchronization function in Node.js or Javascript?

Question

Suppose you support a library that provides the getData function. Your users will call it to get the actual data:
var output = getData();
Under the hood, the data is saved in a file, so you deployed getData using the Node.js built-in fs.readFileSync . Obviously, the getData and fs.readFileSync functions are synchronization functions. Once you were told to switch the underlying data source to a repo, for example MongoDB, which can only be accessed asynchronously. You were also told not to bother your users, the getData API cannot be changed to just return a promise or require a callback parameter. How do you meet both requirements?

An asynchronous function using a callback / promise is JavasSript DNA and Node.js. Any non-trivial JS application is probably infused with this coding style. But this practice can easily lead to the so-called reverse pyramid of death. Even worse, if any code in any caller in the call chain depends on the result of the async function, this code should also be included in the callback function, imposing a restriction on the coding style of the caller. From time to time, I find it necessary to encapsulate the asynchronization function (often provided in a third-party library) into the synchronization function to avoid massive global re-factoring. Finding a solution to this issue usually ended with Node Fibers or npm packages derived from it. But Fibers simply cannot solve the problem that I am facing. Even the example presented by Fieber's author illustrates the flaw:

 ... Fiber(function() { console.log('wait... ' + new Date); sleep(1000); console.log('ok... ' + new Date); }).run(); console.log('back in main'); 

Actual output:

 wait... Fri Jan 21 2011 22:42:04 GMT+0900 (JST) back in main ok... Fri Jan 21 2011 22:42:05 GMT+0900 (JST) 

If the Fiber function really switches the asynchronous function to synchronization mode, the output should be:

 wait... Fri Jan 21 2011 22:42:04 GMT+0900 (JST) ok... Fri Jan 21 2011 22:42:05 GMT+0900 (JST) back in main 

I created another simple example in JSFiddle and am looking for code to get the expected result. I will make a solution that only works in Node.js, so you can require any npm package, even though it doesn't work in JSFiddle.

Answer 1

deasync turns an asynchronous function into a synchronization implemented using a locking mechanism, triggering a Node.js event loop at the JavaScript level. As a result, deasync only blocks subsequent code from running without blocking the entire thread and does not wait to wait. With this module, here is the answer to the jsFiddle call:

 function AnticipatedSyncFunction(){ var ret; setTimeout(function(){ ret = "hello"; },3000); while(ret === undefined) { require('deasync').runLoopOnce(); } return ret; } var output = AnticipatedSyncFunction(); //expected: output=hello (after waiting for 3 sec) console.log("output="+output); //actual: output=hello (after waiting for 3 sec) 

(disclaimer: I am a co-author of deasync . The module was created after the publication of this question and did not find a valid offer.)

Answer 2

There is also an npm sync module. which is used to synchronize the query execution process.

If you want to run parallel requests in a synchronized manner, the node limits this because it never waits for a response. and the synchronization module is ideal for such a solution.

Sample code

 /*require sync module*/ var Sync = require('sync'); app.get('/',function(req,res,next){ story.find().exec(function(err,data){ var sync_function_data = find_user.sync(null, {name: "sanjeev"}); res.send({story:data,user:sync_function_data}); }); }); /*****sync function defined here *******/ function find_user(req_json, callback) { process.nextTick(function () { users.find(req_json,function (err,data) { if (!err) { callback(null, data); } else { callback(null, err); } }); }); } 

link link: https://www.npmjs.com/package/sync

Answer 3

If the Fiber function really turns the asynchronous mode function into synchronism

Yes. Inside the fiber, the function waits until ok . Fibers do not do synchronous asynchronous functions, but allow you to write synchronous code that uses asynchronous functions, and then will be executed asynchronously inside Fiber .

From time to time, I find it necessary to encapsulate an asynchronous function in a synchronization function to avoid massive global re-factoring.

You can not. It is not possible to make asynchronous code synchronously. You will need to anticipate this in your global code and write it with an asynchronous style from the start. Whether you exchange global code for fiber, use promises, promise generators, or simple callbacks, depending on your preference.

My goal is to minimize the impact on the caller when the data collection method is changed from synchronization to asynchronous

Both promises and fibers can do this.

Answer 4

Performing Node.js synchronization is important in several aspects, such as the database. But the actual advantage of Node.js is asynchronous code. Since it is non-blocking by one thread.

we can synchronize it using important functionality of Fiber () Use wait () and defer () we call all methods using await (). then replace the callback functions with defer ().

Normal Async Code. This uses CallBack features.

 function add (var a, var b, function(err,res){ console.log(res); }); function sub (var res2, var b, function(err,res1){ console.log(res); }); function div (var res2, var b, function(err,res3){ console.log(res3); }); 

Sync the above code with Fiber (), wait () and defer ()

 fiber(function(){ var obj1 = await(function add(var a, var b,defer())); var obj2 = await(function sub(var obj1, var b, defer())); var obj3 = await(function sub(var obj2, var b, defer())); }); 

Hope this helps. Thank you.

Answer 5

You should use the promises:

 const asyncOperation = () => { return new Promise((resolve, reject) => { setTimeout(()=>{resolve("hi")}, 3000) }) } const asyncFunction = async () => { return await asyncOperation(); } const topDog = () => { asyncFunction().then((res) => { console.log(res); }); } 

I prefer the definitions of arrow functions. But any line of the form "() => {...}" can also be written as "function () {...}"

Therefore, topDog is not async, despite the async function call.

EDIT: I understand that many times you need the asynchronous function inside the synchronization function to be inside the controller. There is a trick for these situations:

 const getDemSweetDataz = (req, res) => { (async () => { try{ res.status(200).json( await asyncOperation() ); } catch(e){ res.status(500).json(serviceResponse); //or whatever } })() //So we defined and immediately called this async function. } 

Using this with callbacks, you can make a wrapper that doesn't use promises:

 const asyncOperation = () => { return new Promise((resolve, reject) => { setTimeout(()=>{resolve("hi")}, 3000) }) } const asyncFunction = async (callback) => { let res = await asyncOperation(); callback(res); } const topDog = () => { let callback = (res) => { console.log(res); }; (async () => { await asyncFunction(callback) })() } 

By applying this trick to an EventEmitter, you can get the same results. Define an EventEmitter listener where I defined the callback, and emitted an event in which I called the callback.

Answer 6

You should not look at what is happening around the call that the fiber creates, but rather what is going on inside the fiber. When you are inside the fiber, you can program the synchronization style. For example:

 function f1 () {
     console.log ('wait ...' + new Date);
     sleep (1000);
     console.log ('ok ...' + new Date);   
 }

 function f2 () {
     f1 ();
     f1 ();
 }

 Fiber (function () {
     f2 ();
 }). run ();

Inside the fiber, you call f1 , f2 and sleep , as if they were in sync.

In a typical web application, you will create Fiber in your HTTP request manager. Once you do this, you can write all the request processing logic in a synchronization style, even if it calls asynchronous functions (fs, databases, etc.).

Answer 7

I cannot find a script that cannot be solved with node-fibers. The example you provided with node-fibers behaves as expected. The key is to run all the relevant code inside the fiber, so you do not need to start a new fiber in random positions.

Let's look at an example: Suppose you use some framework that is the entry point of your application (you cannot change this structure). This structure loads nodejs modules as plugins and calls some methods for plugins. Suppose that this structure allows only synchronous functions and does not use fibers itself.

There is a library that you want to use in one of your plugins, but this library is asynchronous, and you also do not want to change it.

The main thread cannot be obtained if no fiber is working, but you can still create plugins using the fibers! Just create a wrapper record that runs the entire structure inside the fiber so you can execute from plugins.

Downside: If an infrastructure uses setTimeout or Promise internally, it will exit the fiber context. This can be circumvented by mocking setTimeout , Promise.then and all event handlers.

This way you can get the fiber until Promise is allowed. This code accepts the async (Promise return) function and resumes the fiber when the promise is resolved:

framework-entry.js

 console.log(require("./my-plugin").run()); 

asynchronous-lib.js

 exports.getValueAsync = () => { return new Promise(resolve => { setTimeout(() => { resolve("Async Value"); }, 100); }); }; 

my-plugin.js

 const Fiber = require("fibers"); function fiberWaitFor(promiseOrValue) { var fiber = Fiber.current, error, value; Promise.resolve(promiseOrValue).then(v => { error = false; value = v; fiber.run(); }, e => { error = true; value = e; fiber.run(); }); Fiber.yield(); if (error) { throw value; } else { return value; } } const asyncLib = require("./async-lib"); exports.run = () => { return fiberWaitFor(asyncLib.getValueAsync()); }; 

my-entry.js

 require("fibers")(() => { require("./framework-entry"); }).run(); 

When starting node framework-entry.js it throws an error: Error: yield() called with no fiber running . If you run node my-entry.js , it works as expected.

Answer 8

  const wrap = fn => async (...args) => { const res = await fn(...args) // do stuff return res } 

Answer 9

Currently, this generator model can be a solution in many situations.

Here is an example of sequential console requests in nodejs using the async readline.question function:

 var main = (function* () { // just import and initialize 'readline' in nodejs var r = require('readline') var rl = r.createInterface({input: process.stdin, output: process.stdout }) // magic here, the callback is the iterator.next var answerA = yield rl.question('do you want this? ', r=>main.next(r)) // and again, in a sync fashion var answerB = yield rl.question('are you sure? ', r=>main.next(r)) // readline boilerplate rl.close() console.log(answerA, answerB) })() // <-- executed: iterator created from generator main.next() // kick off the iterator, // runs until the first 'yield', including rightmost code // and waits until another main.next() happens 

Answer 10

I struggled with this node.js first, and async.js is the best library I have found to help you deal with this. If you want to write synchronous code with node, the approach would be this way.

 var async = require('async'); console.log('in main'); doABunchOfThings(function() { console.log('back in main'); }); function doABunchOfThings(fnCallback) { async.series([ function(callback) { console.log('step 1'); callback(); }, function(callback) { setTimeout(callback, 1000); }, function(callback) { console.log('step 2'); callback(); }, function(callback) { setTimeout(callback, 2000); }, function(callback) { console.log('step 3'); callback(); }, ], function(err, results) { console.log('done with things'); fnCallback(); }); } 

this program will ALWAYS produce the following ...

 in main step 1 step 2 step 3 done with things back in main 

Answer 11

Javascript is a single-threaded language, you do not want to block the entire server! Asynchronous code eliminates race conditions by making explicit dependencies.

Learn to love asynchronous code!

Take a look at promises for asynchronous code without creating a pyramid of reverse hell. I recommend the promQ library for node.js

 httpGet(url.parse("http://example.org/")).then(function (res) { console.log(res.statusCode); // maybe 302 return httpGet(url.parse(res.headers["location"])); }).then(function (res) { console.log(res.statusCode); // maybe 200 }); 

http://howtonode.org/promises

EDIT: This is by far the most controversial answer, node now has a yield keyword that allows it to process asynchronous code as if it were synchronous. http://blog.alexmaccaw.com/how-yield-will-transform-node