Can't Require Node Modules In WebWorker (NWJS)
I'm trying to do something I thought would be simple. I'm using nwjs (Formerly called Node-Webkit) which if you don't know basically means I'm developing a desktop app using Chromium & Node where the DOM is in the same scope as Node. I want to offload work to a webworker so that the GUI doesn't hang when I send some text off to Ivona Cloud (using ivona-node) which is a text to speech API. The audio comes back in chunks as it's generated and gets written to an MP3. ivona-node uses fs to write the mp3 to the drive. I got it working in the dom but webworkers are needed to not hang the UI. So I have two node modules I need to use in the webworker, ivona-node and fs.
The problem is that in a webworker you can't use require. So I tried packaging ivona-node and fs with browserify (There's a package called browserify-fs for this which I used) and replacing require with importScripts(). Now I'm getting var errors in the node modules.
Note: I don't think the method of native_fs_ will work for writing the mp3 to disk in chunks (The stream) as it should be and I'm getting errors in the Ivona package as well (Actually first and foremost) that I don't know how to fix. I'm including all information to reproduce this.
Here's an error I'm getting in the console: Uncaught SyntaxError: Unexpected token var VM39 ivonabundle.js:23132
npm install ivona-node
npm install browserify-fs
npm install -g browserify
browserify main.js > ivonabundle.js
browserify index.js > fsbundle.js
package.json...
{
"name": "appname",
"description": "appdescr",
"title": "apptitle",
"main": "index.html",
"window":
{
"toolbar": true,
"resizable": false,
"width": 800,
"height": 500
},
"webkit":
{
"plugin": true
}
}
index.html...
<html>
<head>
<title>apptitle</title>
</head>
<body>
<p><output id="result"></output></p>
<button onclick="startWorker()">Start Worker</button>
<button onclick="stopWorker()">Stop Worker</button>
<br><br>
<script>
var w;
function startWorker() {
if(typeof(Worker) !== "undefined") {
if(typeof(w) == "undefined") {
w = new Worker("TTMP3.worker.js");
w.postMessage(['This is some text to speak.']);
}
w.onmessage = function(event) {
document.getElementById("result").innerHTML = event.data;
};
} else {
document.getElementById("result").innerHTML = "Sorry! No Web Worker support.";
}
}
function stopWorker() {
w.terminate();
w = undefined;
}
</script>
</body>
</html>
TTMP3.worker.js...
importScripts('node_modules/browserify-fs/fsbundle.js','node_modules/ivona-node/src/ivonabundle.js');
onmessage = function T2MP3(Text2Speak)
{
postMessage(Text2Speak.data[0]);
//var fs = require('fs'),
// Ivona = require('ivona-node');
var ivona = new Ivona({
accessKey: 'xxxxxxxxxxx',
secretKey: 'xxxxxxxxxxx'
});
//ivona.listVoices()
//.on('end', function(voices) {
//console.log(voices);
//});
// ivona.createVoice(text, config)
// [string] text - the text to be spoken
// [object] config (optional) - override Ivona request via 'body' value
ivona.createVoice(Text2Speak.data[0], {
body: {
voice: {
name: 'Salli',
language: 'en-US',
gender: 'Female'
}
}
}).pipe(fs.createWriteStream('text.mp3'));
postMessage("Done");
}
There are two things that I wan to point out first:
In order to include the module ivona-node I had to change a little its code. When I try to browserify it I get an error: Uncaught Error: Cannot find module '/node_modules/ivona-node/src/proxy' . Checking the bundle.js generated I notice that it doesn't include the code of the module proxy which is in the file proxy.js in the src folder of ivona-node . I can load the proxy module changing this line HttpsPA = require(__dirname + '/proxy'); by this: HttpsPA = require('./proxy'); . After that ivona-node can be loaded in the client side through browserify . Then I was facing another error when trying to follow the example. Turn out that this code:
ivona.createVoice(Text2Speak.data[0], {
body: {
voice: {
name: 'Salli',
language: 'en-US',
gender: 'Female'
}
}
}).pipe(fs.createWriteStream('text.mp3'));
is no longer correct, it cause the error: Uncaught Error: Cannot pipe. Not readable. Uncaught Error: Cannot pipe. Not readable. The problem here is in the module http . the module browserify has wrapped many built-in modules of npm , which mean that they are available when you use require() or use their functionality. http is one of them, but as you can reference here: strem-http, It tries to match node's api and behavior as closely as possible, but some features aren't available, since browsers don't give nearly as much control over requests. Very significant is the fact of the class http.ClientRequest , this class in nodejs environment create an OutgoingMessage that produce this statement Stream.call(this) allowing the use of the method pipe in the request, but in the browserify version when you call https.request the result is a Writable Stream, this is the call inside the ClientRequest : stream.Writable.call(self) . So we have explicitly a WritableStream even with this method:
Writable.prototype.pipe = function() {
this.emit('error', new Error('Cannot pipe. Not readable.'));
};
The responsible of the above error. Now we have to use a different approach to save the data from ivona-node , which leave me to the second issue.
Is well know that having access to the FileSystem from a web application have many security issues, so the problem is how we can have access to the FileSystem from the web worker. One first approach is using the HTML5 FileSystem API. This approach has the inconvenient that it operate in a sandbox, so if we have in a desktop app we want to have access to the OS FileSystem. To accomplish this goal we can pass the data from the web worker to the main thread where we can use all the nodejs FileSystem functionalities. Web worker provide a functionality called Transferable Objects , you can get more info here and here that we can use to pass the data received from the module ivona-node in the web worker to the main thread and then use require('fs') in the same way that node-webkit provide us. These are the step you can follow:
install browserify
npm install -g browserify
install ivona-node
npm install ivona-node --save
go to node_modules/ivona-node/src/main.js and change this line:
HttpsPA = require(__dirname + '/proxy');
by this:
HttpsPA = require('./proxy');
create your bundle.js .
Here you have some alternatives, create a bundle.js to allow a require() or put some code in a file with some logic of what you want (you can actually include all the code of the web worker) and then create the bundle.js . In this example I will create the bundle.js only for have access to require() and use importScripts() in the web worker file
browserify -r ivona-node > ibundle.js
Put all together
Modify the code of the web worker and index.html in order to receive the data in the web worker and send it to the main thread (in index.html)
this is the code of web worker (MyWorker.js)
importScripts('ibundle.js');
var Ivona = require('ivona-node');
onmessage = function T2MP3(Text2Speak)
{
var ivona = new Ivona({
accessKey: 'xxxxxxxxxxxx',
secretKey: 'xxxxxxxxxxxx'
});
var req = ivona.createVoice(Text2Speak.data[0], {
body: {
voice: {
name: 'Salli',
language: 'en-US',
gender: 'Female'
}
}
});
req.on('data', function(chunk){
var arr = new Uint8Array(chunk);
postMessage({event: 'data', data: arr}, [arr.buffer]);
});
req.on('end', function(){
postMessage(Text2Speak.data[0]);
});
}
and index.html:
<html>
<head>
<title>apptitle</title>
</head>
<body>
<p><output id="result"></output></p>
<button onclick="startWorker()">Start Worker</button>
<button onclick="stopWorker()">Stop Worker</button>
<br><br>
<script>
var w;
var fs = require('fs');
function startWorker() {
var writer = fs.createWriteStream('text.mp3');
if(typeof(Worker) !== "undefined") {
if(typeof(w) == "undefined") {
w = new Worker("MyWorker.js");
w.postMessage(['This is some text to speak.']);
}
w.onmessage = function(event) {
var data = event.data;
if(data.event !== undefined && data.event == 'data'){
var buffer = new Buffer(data.data);
writer.write(buffer);
}
else{
writer.end();
document.getElementById("result").innerHTML = data;
}
};
} else {
document.getElementById("result").innerHTML = "Sorry! No Web Worker support.";
}
}
function stopWorker() {
w.terminate();
w = undefined;
}
</script>
</body>
</html>
上一篇: json.org java库去哪了？