{ "_args": [ [ { "raw": "workerpool@^2.2.1", "scope": null, "escapedName": "workerpool", "name": "workerpool", "rawSpec": "^2.2.1", "spec": ">=2.2.1 <3.0.0", "type": "range" }, "/home/travis/build/lukesargeant/ember-sparkline/node_modules/broccoli-babel-transpiler" ] ], "_from": "workerpool@>=2.2.1 <3.0.0", "_id": "workerpool@2.3.0", "_inCache": true, "_location": "/workerpool", "_nodeVersion": "8.6.0", "_npmOperationalInternal": { "host": "s3://npm-registry-packages", "tmp": "tmp/workerpool-2.3.0.tgz_1506785316329_0.8559376900084317" }, "_npmUser": { "name": "josdejong", "email": "wjosdejong@gmail.com" }, "_npmVersion": "5.4.2", "_phantomChildren": {}, "_requested": { "raw": "workerpool@^2.2.1", "scope": null, "escapedName": "workerpool", "name": "workerpool", "rawSpec": "^2.2.1", "spec": ">=2.2.1 <3.0.0", "type": "range" }, "_requiredBy": [ "/broccoli-babel-transpiler" ], "_resolved": "https://registry.npmjs.org/workerpool/-/workerpool-2.3.0.tgz", "_shasum": "86c5cbe946b55e7dc9d12b1936c8801a6e2d744d", "_shrinkwrap": null, "_spec": "workerpool@^2.2.1", "_where": "/home/travis/build/lukesargeant/ember-sparkline/node_modules/broccoli-babel-transpiler", "author": { "name": "Jos de Jong", "email": "wjosdejong@gmail.com", "url": "https://github.com/josdejong" }, "bugs": { "url": "https://github.com/josdejong/workerpool/issues" }, "dependencies": { "object-assign": "4.1.1" }, "description": "Offload tasks to a pool of workers on node.js and in the browser", "devDependencies": { "gulp": "^3.9.1", "gulp-util": "^3.0.7", "istanbul": "^0.4.4", "mocha": "^2.5.3", "uglify-js": "2.8.29", "webpack": "^1.13.1" }, "directories": {}, "dist": { "integrity": "sha512-JP5DpviEV84zDmz13QnD4FfRjZBjnTOYY2O4pGgxtlqLh47WOzQFHm8o17TE5OSfcDoKC6vHSrN4yPju93DW0Q==", "shasum": "86c5cbe946b55e7dc9d12b1936c8801a6e2d744d", "tarball": "https://registry.npmjs.org/workerpool/-/workerpool-2.3.0.tgz" }, "gitHead": "b53796c14498ff5ed2ab3c630ae13264432ebebe", "homepage": "https://github.com/josdejong/workerpool", "keywords": [ "worker", "web worker", "cluster", "pool", "isomorphic" ], "main": "./index.js", "maintainers": [ { "name": "josdejong", "email": "wjosdejong@gmail.com" } ], "name": "workerpool", "optionalDependencies": {}, "readme": "# workerpool\n\nJavaScript is based upon a single event loop which handles one event at a time. Jeremy Epstein [explains this clearly](http://greenash.net.au/thoughts/2012/11/nodejs-itself-is-blocking-only-its-io-is-non-blocking/):\n\n> In Node.js everything runs in parallel, except your code.\n> What this means is that all I/O code that you write in Node.js is non-blocking,\n> while (conversely) all non-I/O code that you write in Node.js is blocking.\n\nThis means that CPU heavy tasks will block other tasks from being executed. In case of a browser environment, the browser will not react to user events like a mouse click while executing a CPU intensive task (the browser \"hangs\"). In case of a node.js server, the server will not respond to any new request while executing a single, heavy request.\n\nFor front-end processes, this is not a desired situation.\nTherefore, CPU intensive tasks should be offloaded from the main event loop onto dedicated *workers*. In a browser environment, [Web Workers](http://www.html5rocks.com/en/tutorials/workers/basics/) can be used. In node.js, [child processes](http://nodejs.org/api/child_process.html) are available. An application should be split in separate, decoupled parts, which can run independent of each other in a parallelized way. Effectively, this results in an architecture which achieves concurrency by means of isolated processes and message passing.\n\n**workerpool** offers an easy way to create a pool of workers for both dynamically offloading computations as well as managing a pool of dedicated workers. **workerpool** basically implements a [thread pool pattern](http://en.wikipedia.org/wiki/Thread_pool_pattern). There is a pool of workers to execute tasks. New tasks are put in a queue. A worker executes one task at a time, and once finished, picks a new task from the queue. Workers can be accessed via a natural, promise based proxy, as if they are available straight in the main application.\n\n**workerpool** runs on node.js, Chrome, Firefox, Opera, Safari, and IE10+.\n\n\n## Features\n\n- Easy to use\n- Runs in the browser and on node.js\n- Dynamically offload functions to a worker\n- Access workers via a proxy\n- Cancel running tasks\n- Set a timeout on tasks\n- Handles crashed workers\n- Small, less than 4 kB minified and gzipped\n\n\n## Install\n\nInstall via npm:\n\n npm install workerpool\n\n\n## Load\n\nTo load workerpool in a node.js application (both main application as well as workers):\n\n```js\nvar workerpool = require('workerpool');\n```\n\nTo load workerpool in the browser:\n\n```html\n\n```\n\nTo load workerpool in a web worker in the browser:\n\n```js\nimportScripts('workerpool.js');\n```\n\n\n## Use\n\n### Offload functions dynamically\n\nIn the following example there is a function `add`, which is offloaded dynamically to a worker to be executed for a given set of arguments.\n\n**myApp.js**\n```js\nvar workerpool = require('workerpool');\nvar pool = workerpool.pool();\n\nfunction add(a, b) {\n return a + b;\n}\n\npool.exec(add, [3, 4])\n .then(function (result) {\n console.log('result', result); // outputs 7\n })\n .catch(function (err) {\n console.error(err);\n })\n .then(function () {\n pool.terminate(); // terminate all workers when done\n });\n```\n\nNote that both function and arguments must be static and stringifiable, as they need to be send to the worker in a serialized form. In case of large functions or function arguments, the overhead of sending the data to the worker can be significant.\n\n\n### Dedicated workers\n\nA dedicated worker can be created in a separate script, and then used via a worker pool.\n\n**myWorker.js**\n```js\nvar workerpool = require('workerpool');\n\n// a deliberately inefficient implementation of the fibonacci sequence\nfunction fibonacci(n) {\n if (n < 2) return n;\n return fibonacci(n - 2) + fibonacci(n - 1);\n}\n\n// create a worker and register public functions\nworkerpool.worker({\n fibonacci: fibonacci\n});\n```\n\nThis worker can be used by a worker pool:\n\n**myApp.js**\n```js\nvar workerpool = require('workerpool');\n\n// create a worker pool using an external worker script\nvar pool = workerpool.pool(__dirname + '/myWorker.js');\n\n// run registered functions on the worker via exec\npool.exec('fibonacci', [10])\n .then(function (result) {\n console.log('Result: ' + result); // outputs 55\n })\n .catch(function (err) {\n console.error(err);\n })\n .then(function () {\n pool.terminate(); // terminate all workers when done\n });\n\n// or run registered functions on the worker via a proxy:\npool.proxy()\n .then(function (worker) {\n return worker.fibonacci(10);\n })\n .then(function (result) {\n console.log('Result: ' + result); // outputs 55\n })\n .catch(function (err) {\n console.error(err);\n })\n .then(function () {\n pool.terminate(); // terminate all workers when done\n });\n```\n\nWorker can also initialize asynchronously:\n\n**myAsyncWorker.js**\n```js\ndefine(['workerpool/dist/workerpool'], function(workerpool) {\n\n // a deliberately inefficient implementation of the fibonacci sequence\n function fibonacci(n) {\n if (n < 2) return n;\n return fibonacci(n - 2) + fibonacci(n - 1);\n }\n\n // create a worker and register public functions\n workerpool.worker({\n fibonacci: fibonacci\n });\n\n});\n```\n\n## Examples\n\nExamples are available in the examples directory:\n\n[https://github.com/josdejong/workerpool/tree/master/examples](https://github.com/josdejong/workerpool/tree/master/examples)\n\n\n## API\n\nThe API of workerpool consists of two parts: a function `workerpool.pool` to create a worker pool, and a function `workerpool.worker` to create a worker.\n\n### pool\n\nA workerpool can be created using the function `workerpool.pool`:\n\n`workerpool.pool([script: string] [, options: Object]) : Pool`\n\nWhen a `script` argument is provided, the provided script will be started as a dedicated worker.\nWhen no `script` argument is provided, a default worker is started which can be used to offload functions dynamically via `Pool.exec`.\nNote that on node.js, `script` must be an absolute file path like `__dirname + '/myWorker.js'`.\n\nThe following options are available:\n- `minWorkers: number | 'max'`. The minimum number of workers that must be initialized and kept available. Setting this to `'max'` will create `maxWorkers` default workers (see below).\n- `maxWorkers: number`. The default number of maxWorkers is the number of CPU's minus one. When the number of CPU's could not be determined (for example in older browsers), `maxWorkers` is set to 3.\n\nA worker pool contains the following functions:\n\n- `Pool.exec(method: Function | string, params: Array | null) : Promise.<*, Error>`
\n Execute a function on a worker with given arguments.\n - When `method` is a string, a method with this name must exist at the worker and must be registered to make it accessible via the pool. The function will be executed on the worker with given parameters.\n - When `method` is a function, the provided function `fn` will be stringified, send to the worker, and executed there with the provided parameters. The provided function must be static, it must not depend on variables in a surrounding scope.\n\n- `Pool.proxy() : Promise.`
\n Create a proxy for the worker pool. The proxy contains a proxy for all methods available on the worker. All methods return promises resolving the methods result.\n\n- `Pool.stats() : Object`
\n Retrieve statistics on workers, and active and pending tasks.\n\n Returns an object containing the following properties:\n\n ```js\n {\n totalWorkers: 0,\n busyWorkers: 0,\n idleWorkers: 0,\n pendingTasks: 0,\n activeTasks: 0\n }\n ```\n\n- `Pool.terminate([force: boolean [, timeout: number]])`\n\n If parameter `force` is false (default), workers will finish the tasks they are working on before terminating themselves. When `force` is true, all workers are terminated immediately without finishing running tasks. If `timeout` is provided, worker will be forced to terminal when the timeout expires and the worker has not finished.\n\n- `Pool.clear([force: boolean])`
\n *Deprecated: use `Pool.terminate` instead*
.\n Clear all workers from the pool. If parameter `force` is false (default), workers will finish the tasks they are working on before terminating themselves. When `force` is true, all workers are terminated immediately without finishing running tasks.\n\nThe function `Pool.exec` and the proxy functions all return a `Promise`. The promise has the following functions available:\n\n- `Promise.then(fn: Function.)`
\n Get the result of the promise once resolve.\n- `Promise.catch(fn: Function.)`
\n Get the error of the promise when rejected.\n- `Promise.cancel()`
\n A running task can be cancelled. The worker executing the task is enforced to terminate immediately.\n The promise will be rejected with a `Promise.CancellationError`.\n- `Promise.timeout(delay: number)`
\n Cancel a running task when it is not resolved or rejected withing given delay in milliseconds. The timer will start when the task is actually started, not when the task is created and queued.\n The worker executing the task is enforced to terminate immediately.\n The promise will be rejected with a `Promise.TimeoutError`.\n\nExample usage:\n\n```js\nvar workerpool = require('workerpool');\n\nfunction add(a, b) {\n return a + b;\n}\n\nvar pool1 = workerpool.pool();\n\n// offload a function to a worker\npool1.exec(add, [2, 4])\n .then(function (result) {\n console.log(result); // will output 6\n })\n .catch(function (err) {\n console.error(err);\n });\n\n// create a dedicated worker\nvar pool2 = workerpool.pool(__dirname + '/myWorker.js');\n\n// supposed myWorker.js contains a function 'fibonacci'\npool2.exec('fibonacci', [10])\n .then(function (result) {\n console.log(result); // will output 55\n })\n .catch(function (err) {\n console.error(err);\n });\n\n// create a proxy to myWorker.js\npool2.proxy()\n .then(function (myWorker) {\n return myWorker.fibonacci(10)\n })\n .then(function (result) {\n console.log(result); // will output 55\n })\n .catch(function (err) {\n console.error(err);\n });\n\n// create a pool with a specified maximum number of workers\nvar pool3 = workerpool.pool({maxWorkers: 7});\n```\n\n\n### worker\n\nA worker is constructed as:\n\n`workerpool.worker([methods: Object.])`\n\nArgument `methods` is optional can can be an object with functions available in the worker. Registered functions will be available via the worker pool.\n\nExample usage:\n\n```js\n// file myWorker.js\nvar workerpool = require('workerpool');\n\nfunction add(a, b) {\n return a + b;\n}\n\nfunction multiply(a, b) {\n return a * b;\n}\n\n// create a worker and register functions\nworkerpool.worker({\n add: add,\n multiply: multiply\n});\n```\n\nAsynchronous results can be handled by returning a Promise from a function in the worker:\n\n```js\n// file myWorker.js\nvar workerpool = require('workerpool');\n\nfunction timeout(delay) {\n return new Promise(function (resolve, reject) {\n setTimeout(resolve, delay)\n });\n}\n\n// create a worker and register functions\nworkerpool.worker({\n timeout: timeout\n});\n```\n\n### Utilities\n\nFollowing properties are available for convenience:\n\n- **platform**: The Javascript platform. Either *node* or *browser*\n- **isMainThread**: Whether the code is running in main thread or not (Workers)\n- **cpus**: The number of CPUs/cores available\n\n\n## Roadmap\n\n- Implement functions for parallel processing: `map`, `reduce`, `forEach`,\n `filter`, `some`, `every`, ...\n- Implement graceful degradation on old browsers not supporting webworkers:\n fallback to processing tasks in the main application.\n- Implement session support: be able to handle a series of related tasks by a\n single worker, which can keep a state for the session.\n\n\n## Sources of inspiration\n\n- https://github.com/learnboost/cluster\n- https://github.com/adambom/parallel.js\n- https://github.com/padolsey/operative\n- https://github.com/calvinmetcalf/catiline\n- https://github.com/Unitech/pm2\n- https://github.com/godaddy/node-cluster-service\n- https://github.com/ramesaliyev/EasyWebWorker\n\n\n## Build\n\nFirst clone the project from github:\n\n git clone git://github.com/josdejong/workerpool.git\n cd workerpool\n\nInstall the project dependencies:\n\n npm install\n\nThen, the project can be build by executing the build script via npm:\n\n npm run build\n\nThis will build the library workerpool.js and workerpool.min.js from the source\nfiles and put them in the folder dist.\n\n\n## Test\n\nTo execute tests for the library, install the project dependencies once:\n\n npm install\n\nThen, the tests can be executed:\n\n npm test\n\nTo test code coverage of the tests:\n\n npm run coverage\n\nTo see the coverage results, open the generated report in your browser:\n\n ./coverage/lcov-report/index.html\n\n\n## License\n\nCopyright (C) 2014-2017 Jos de Jong \n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n", "readmeFilename": "README.md", "repository": { "type": "git", "url": "git://github.com/josdejong/workerpool.git" }, "scripts": { "build": "gulp", "coverage": "istanbul cover _mocha -- test; echo \"\nCoverage report is available at ./coverage/lcov-report/index.html\"", "test": "mocha test --timeout 10000", "watch": "gulp watch" }, "version": "2.3.0" }