Node.js: The Road to Workers

# Node.js: The Road to Workers

Anna Henningsen <br />
she/her <br />
@addaleax

<br />

Node.js TSC <br />
Active Node.js contributor since December 2015

---

# How did I get into Node.js core?

```js
setTimeout(1000, 1000, function() { console.log('hi!'); });
```

---
template: how-did-i

(don’t ask me why)

```
> TypeError: callback.call is not a function
    at ontimeout [as _onTimeout] (timers.js:195:34)
    at Timer.listOnTimeout (timers.js:92:15)
```

---
template: how-did-i

```diff
@@ -177,6 +177,10 @@ exports.enroll = function(item, msecs) {

exports.setTimeout = function(callback, after) {
+  if (typeof callback !== 'function') {
+    throw new TypeError('"callback" argument must be a function');
+  }
+
   after *= 1; // coalesce to number or NaN

if (!(after >= 1 && after <= TIMEOUT_MAX)) {
```

---

# Why am I talking about this?

- I didn’t know much about what was happening inside Node.js
- A lot of people in the community don’t
- There’s a hidden world here

---

<img src="50-speedup.png" />

💖 If you like something, tell the people who made it 💖

---

# Backing up a little bit…

<div style="float:right">
  <div>
    <img src="Pebble_stack.jpg" style="height:400px" />
  </div>
  
  <span style="color:grey;font-size:50%">Image: © CC-BY 3.0 Zzubnik on Wikimedia Commons</span>
</div>

## How is Node.js structured?

- One Node.js application is …

- One Process
  - One Thread*
  - One Event Loop
  - One JS Engine instance
  - One Node.js instance

- None of this has to be that way!

---

# What’s the problem

<div style="float:right">
  <div>
    <img src="Queue-to-the-Louvre-cropped.jpg" style="height:300px" />
  </div>
  
  <span style="color:grey;font-size:50%">Image: © CC-BY-SA 3.0 Edal on Wikimedia Commons</span>
</div>

- Typical Node.js application: *One* piece of code runs at a time
  - This makes life easy
- Hogging the CPU keeps I/O from happening
- CPU-heavy applications may have increased latency

- The golden rule of Node.js performance: <br/>
**<span style="color:gold;padding:10px">Don’t block the event loop</span>**

---

# Existing solutions

- Multiple processes
- `cluster` API

Drawbacks:

- 👎 No shared memory
- 👎 Communication over JSON

Advantages:

- 👍 More isolation, stability
- 👍 Identical APIs

---

# The solution: Worker threads

- One Node.js application is …

- One Process
  - **<span style="color: #aaaaff">Multiple</span> threads**
  - One event loop per thread
  - One JS engine instance per thread
  - One Node.js instance per thread

---

# Implementation idea: <br /> Embed Node.js into itself

<div style="float:left">
  <div>
    <img src="Klein_bottle.svg" style="height:300px" />
  </div>
  
  <span style="color:grey;font-size:50%">Image: © CC-BY-SA 3.0 Tttrung on Wikimedia Commons</span>
</div>

- Embedding is supported – in theory
- In practice, before Node 10: *Eep*.
  - Lots of global state
  - No resource clean-up on exit
  - CLI options were global

---

# Prior art

- 2 PRs by Petka Antonov
- Mostly just stalled out due to lack of reviewers

- Lightweight implementation à la WebWorkers by Audrey Tang (唐鳳)
  (https://github.com/audreyt/node-webworker-threads/)

---

# A New Attempt

- Around October 2017: Ayo.js, a fork of Node.js
- Prime development ground for Worker support
- Worked out a full threading implementation for Node.js

---

# Porting to Node.js

<img src="20876-head.png" style="width: 90%" />
<img src="20876-middle.png" style="width: 90%" />
<img src="20876-merged.png" style="width: 90%" />

---

# Porting to Node.js

<div style="float:right">
  <div>
    <img src="Hurdles_Bislett_Games_2008.jpg" style="height:270px" />
  </div>
  
  <span style="color:grey;font-size:50%">Image: © CC-BY 2.0 Ragnar Singsaas</span>
</div>

- Major issues:
  - Windows support blocked workers for 2+ months
  - npm package name `worker` unavailable
  - New module: Scoped name à la `@nodejs/workers`?
- Surprisingly, less issues with review for the main PR
- Available in Node 10! 🎉

---

# The result

<div style="float:right" id="workers-design">
<table style="border-spacing: 5px">
<tr class="ngroup">
<td class="nitem" colspan="2">Your code</td>
<td class="nitem" colspan="2">Your code</td>
<td class="nitem" colspan="2">Your code</td>
</tr>
<tr class="ngroup">
<td class="nitem" colspan="2">Node.js</td>
<td class="nitem" colspan="2">Node.js</td>
<td class="nitem" colspan="2">Node.js</td>
</tr>
<tr class="ngroup">
<td class="nitem">V8</td><td class="nitem">libuv</td>
<td class="nitem">V8</td><td class="nitem">libuv</td>
<td class="nitem">V8</td><td class="nitem">libuv</td>
</tr>
<tr class="ngroup">
<td class="nitem" colspan="2">Thread</td>
<td class="nitem" colspan="2">Thread</td>
<td class="nitem" colspan="2">Thread</td>
</tr>
<tr class="ngroup">
<td class="nitem" colspan="6">Process</td>
</tr>
</table>
</div>

- Not quite the Web `Worker` API
- `require()` works
- Almost all Node.js core APIs work

## Communication

- 👍 Transferring `ArrayBuffer`s works
- 👍 `SharedArrayBuffer`s and `Atomics` work
- 👍 `MessagePort`, `MessageChannel` close to the browser
- `workerData` to pass startup data

---

# API

- `const { Worker, parentPort } = require('worker_threads');`
- `new Worker(filename) || new Worker(code, { eval: true })`
- `worker.on('message'), worker.postMessage(data)`
- `parentPort.on('message'), parentPort.postMessage(data)`

---

# Example

```js
const { Worker } = require('worker_threads');

```sh
$ node --experimental-worker test.js
{ pong: 'ping' }
```

---

# Example

```js
const data = new Int32Array(new SharedArrayBuffer(32));
data[0] = 2016;
const worker = new Worker(`
const data = require('worker_threads').workerData;
for (let i = 2; data[0] != 1; ++i)  // Do the prime dance
  while (data[0] % i === 0)
    data[0] /= i, data[i]++;
data[1] = 1;  // Change data[1] and notify the main thread on it
Atomics.notify(data, 1);
`, { eval: true, workerData: data });
Atomics.wait(data, 1, 0);  // Wait for the notify() call above
console.log(data);
```

---

# Status

- <span style="background-color: #ffaaaa; color: black; padding: 5px">Experimental</span>
- We’re looking for feedback! → https://github.com/nodejs/worker/issues/6
- Do you have: Real world usage? Cool experimentation?
- DevTools support!
- Web Locks API?
- Take care when using add-ons
- `.terminate()` is kind of experimental-within-experimental

## Hopefully?

- Passing native handles around (e.g. sockets)
- Deadlock detection
- More isolation?

---

# Other approaches

- Alibaba and Microsoft (Napa.js) have been working on similar things
  - 😲 Shared JS objects 😲
  - Require Engine modifications

# Please don’t …

- … think this makes everything magically faster
- … use this for parallelizing I/O
- … think spawning workers is cheap (yet?)
  - About the same as for child processes
  - We’re working on it!
  - Generally: Use worker pools

---

# Thank you!

If you have any questions: <br />
Look for long blue hair!

Also:
- https://twitter.com/addaleax/
- anna@addaleax.net
- https://github.com/nodejs/help/

Slides @ https://addaleax.net/workers-verona/