Cannot Require Node Modules in WebWorker (NWJS)

I am trying to do what I thought would be simple. I use nwjs (formerly called Node -Webkit), which, if you do not know, basically means that I am developing a desktop application using Chromium and Node, where the DOM is in the same area as Node. I want to offload the work of a web worker so that the graphical interface does not freeze when I send text to Ivona Cloud (using ivona-node), which is a text API. Sound is returned to pieces as it is created and recorded in MP3. ivona-node uses fs to write mp3 to disk. I got it working in dom, but web workers are needed so as not to damage the user interface. Thus, I have two Node modules that I need to use in webworker, ivona-node and fs.

The problem is that in webworker you cannot use require. So I tried packing ivona-node and fs with a browser (there is a package called browserify-fs for this that I used), and importScripts () is required for the replacement. Now I get var errors in Node modules.

Note. I don’t think that the native_fs_ method will work to write mp3 to disk in chunks (The stream), as it should be, and I get errors in the Ivona package (actually, first of all) that I don’t know how to fix. I include all the information to reproduce this.

Here is the error I get in the console: Uncaught SyntaxError: Unexpected token var VM39 ivonabundle.js: 23132

  • Steps to play in NWJS:

npm install ivona- node

npm install browserify-fs

npm install -g browserify

  • Now I looked at main.js for ivona-node and index.js for browser-fs:

browserify main.js> ivonabundle.js

browserify index.js> fsbundle.js


package.json ...

{ "name": "appname", "description": "appdescr", "title": "apptitle", "main": "index.html", "window": { "toolbar": true, "resizable": false, "width": 800, "height": 500 }, "webkit": { "plugin": true } } 

index.html ...

 <html> <head> <title>apptitle</title> </head> <body> <p><output id="result"></output></p> <button onclick="startWorker()">Start Worker</button> <button onclick="stopWorker()">Stop Worker</button> <br><br> <script> var w; function startWorker() { if(typeof(Worker) !== "undefined") { if(typeof(w) == "undefined") { w = new Worker("TTMP3.worker.js"); w.postMessage(['This is some text to speak.']); } w.onmessage = function(event) { document.getElementById("result").innerHTML = event.data; }; } else { document.getElementById("result").innerHTML = "Sorry! No Web Worker support."; } } function stopWorker() { w.terminate(); w = undefined; } </script> </body> </html> 

TTMP3.worker.js ...

 importScripts('node_modules/browserify-fs/fsbundle.js','node_modules/ivona-node/src/ivonabundle.js'); onmessage = function T2MP3(Text2Speak) { postMessage(Text2Speak.data[0]); //var fs = require('fs'), // Ivona = require('ivona-node'); var ivona = new Ivona({ accessKey: 'xxxxxxxxxxx', secretKey: 'xxxxxxxxxxx' }); //ivona.listVoices() //.on('end', function(voices) { //console.log(voices); //}); // ivona.createVoice(text, config) // [string] text - the text to be spoken // [object] config (optional) - override Ivona request via 'body' value ivona.createVoice(Text2Speak.data[0], { body: { voice: { name: 'Salli', language: 'en-US', gender: 'Female' } } }).pipe(fs.createWriteStream('text.mp3')); postMessage("Done"); } 
+7
javascript requirejs web-worker node-webkit
source share
1 answer

There are two things that I want to note first:

  • Including node modules in a web worker

To enable the ivona-node module, I had to modify its code a bit. When I try to scroll it, I get the error message: Uncaught Error: Cannot find module '/node_modules/ivona-node/src/proxy' . When checking the generated bundle.js I noticed that it does not include the proxy module code, which is located in the proxy.js file in the src ivona-node folder. I can load the proxy module by changing this line HttpsPA = require(__dirname + '/proxy'); as follows: HttpsPA = require('./proxy'); . After that, ivona-node can be downloaded from the client side through browserify . Then I came across another error when trying to follow an example. It turned out that this code:

 ivona.createVoice(Text2Speak.data[0], { body: { voice: { name: 'Salli', language: 'en-US', gender: 'Female' } } }).pipe(fs.createWriteStream('text.mp3')); 

ceases to be correct, it causes an error: Uncaught Error: Cannot pipe. Not readable. Uncaught Error: Cannot pipe. Not readable. The problem here is in the http module. The browserify module has wrapped many of the npm built-in modules, which means that they are available when you use require() or use their functionality. http is one of them, but as you can mention here: strem-http , it tries to combine node api and behavior as closely as possible, but some functions are not available because browsers do not provide almost as much control over requests. The fact of the class http.ClientRequest is very important, this class in the nodejs environment creates nodejs that produce this Stream.call(this) statement, which allows you to use the pipe method in the request, but in browserify if you call https.request , the result is a Writable stream, this is a call inside ClientRequest : stream.Writable.call(self) . So we explicitly have a WritableStream even with this method:

 Writable.prototype.pipe = function() { this.emit('error', new Error('Cannot pipe. Not readable.')); }; 

Responsibility for the above error. Now we need to use a different approach to save data from ivona-node , which will leave me in the second problem.

  1. Create file from web user

It is well known that having access to the FileSystem from a web application is associated with many security issues, so the problem is how we can access the FileSystem from a web worker. The first approach uses the HTML5 API file system . This approach is inconvenient because it works in the sandbox, so if we have a desktop application, we want to access the OS FileSystem. To achieve this, we can transfer the web worker data to the main thread, where we can use all the functionality of nodejs FileSystem. The web worker provides functionality called Transferable Objects , you can get additional information here and here , which we can use to transfer data received from the ivona-node module in the web worker to the main stream, and then use require('fs') just like node-webkit provide us. This is the step you can follow:

  • install browserify

     npm install -g browserify 
  • install ivona-node

     npm install ivona-node --save 
  • go to node_modules/ivona-node/src/main.js and change this line:

    HttpsPA = require(__dirname + '/proxy');

    :

    HttpsPA = require('./proxy');

  • create your bundle.js .

    Here you have alternatives, create bundle.js to allow require() , or put some code in a file with some logic of what you want (you can actually include all the web worker code), and then create bundle.js . In this example, I will create bundle.js only to access require() and use importScripts() in the web worker file

    browserify -r ivona-node > ibundle.js

  • Together

    Change the web worker code and index.html to get the data in the web worker and send it to the main stream (in index.html)

this is the web worker code (MyWorker.js)

 importScripts('ibundle.js'); var Ivona = require('ivona-node'); onmessage = function T2MP3(Text2Speak) { var ivona = new Ivona({ accessKey: 'xxxxxxxxxxxx', secretKey: 'xxxxxxxxxxxx' }); var req = ivona.createVoice(Text2Speak.data[0], { body: { voice: { name: 'Salli', language: 'en-US', gender: 'Female' } } }); req.on('data', function(chunk){ var arr = new Uint8Array(chunk); postMessage({event: 'data', data: arr}, [arr.buffer]); }); req.on('end', function(){ postMessage(Text2Speak.data[0]); }); } 

and index.html:

 <html> <head> <title>apptitle</title> </head> <body> <p><output id="result"></output></p> <button onclick="startWorker()">Start Worker</button> <button onclick="stopWorker()">Stop Worker</button> <br><br> <script> var w; var fs = require('fs'); function startWorker() { var writer = fs.createWriteStream('text.mp3'); if(typeof(Worker) !== "undefined") { if(typeof(w) == "undefined") { w = new Worker("MyWorker.js"); w.postMessage(['This is some text to speak.']); } w.onmessage = function(event) { var data = event.data; if(data.event !== undefined && data.event == 'data'){ var buffer = new Buffer(data.data); writer.write(buffer); } else{ writer.end(); document.getElementById("result").innerHTML = data; } }; } else { document.getElementById("result").innerHTML = "Sorry! No Web Worker support."; } } function stopWorker() { w.terminate(); w = undefined; } </script> </body> </html> 
+3
source share

All Articles