# Understanding S.js

S.js is one of the fastest stream libraries around.  It was written by Adam Haile in 2013 and has inspired a lot of interesting work in UI and Reactive programming.  It however uses a very different model to most stream libraries, users of knockout.js may find it familiar, stream relationships/dependencies are automatically detected by usage.

I'm part of the team working on a new UI library: Vella.  And Vella will be largely powered by S.js.  As a team we agreed there's some rough edges of S.js that may or may not be required for our use case.  We also agreed, it is important to understand the S.js source back to front so we can responsibly maintain our library which depends on S for all of its core operations.

So this post's goal is to document the process of understanding the codebase with very little pre-existing knowledge of the space or ecosystem.  Thankfully, it is just one file, and it is only ~700 odd lines of code.  So it is doable!  Let's begin.

## Usage API

Before we look at the internals let's look at the external API and imagine how it might work behind the scenes.  That way we can reference any unanswered questions or mysteries later on.

You can create an event stream in S using the `S.data` constructor.  The resulting stream is a getter/setter.

```js
const name = S.data('James')

name() // 'James'

name('Fred')

name() // Fred
```

So far, not _too_ disimilar to other stream libraries like `flyd` or `mithril-stream`.

We can also subscribe to a stream's changes a few different ways, let's start with what might be the most familiar.


```js
const name = S.data('James')

S.on(name, () => console.log('Name:', name() ))

name('Fred')
name('Barney')
name('Pierre-Yves')

// logs
// Name: James
// Name: Fred
// Name: Barney
// Name: Pierre-Yves
```

We can listen to multiple streams at once using `on` as well because, `S.on` accepts a list.


```js
const name = S.data('James')
const age = S.data(10000)

S.on([name, age], () => console.log('Name:', name(), 'Age:', age() ))

name('Fred')
name('Barney')
age(10001)
name('Pierre-Yves')

// logs
// Name: James Age: 10000
// Name: Fred Age: 10000
// Name: Barney Age: 10000
// Name: Barney Age: 10001
// Name: Pierre-Yves Age: 10001
```

This is still familiar territory hopefully.

But what if we want to subscribe to a stream and produce a value instead of just logging or performing a side effect?

S calls this a `computation`.  Turns out, all we need to do is return a value and store the resulting stream.

```js

const name = S.data('James')
const age = S.data(10000)

// Stream<[String, String, String, Number]>
const log = 
    S.on([name, age], () => ['Name:', name(), 'Age:', age() ])

S.on(log, () => console.log(...log()))

name('Fred')
name('Barney')
age(10001)
name('Pierre-Yves')

// logs
// Name: James Age: 10000
// Name: Fred Age: 10000
// Name: Barney Age: 10000
// Name: Barney Age: 10001
// Name: Pierre-Yves Age: 10001
```

At this point, you might be wondering why `S.on` doesn't pass in the values of the streams `name` and `age` like e.g. `stream.map` would in other libraries.  That's because S automatically detects subscriptions, and so instead of `S.on` we can simply use `S` like so:


```js

const name = S.data('James')
const age = S.data(10000)

// Automatically tracks that `log` depends on `name` and `age`
// S.on doesn't automatically track dependencies and can be used
// for some edge cases where automatic dependency tracking could
// cause problems.
const log = 
    S(() => ['Name:', name(), 'Age:', age() ])

// When log emits, our S function will execute
// and console.log will run.
S(() => console.log(...log()))

name('Fred')
name('Barney')
age(10001)
name('Pierre-Yves')

// logs
// Name: James Age: 10000
// Name: Fred Age: 10000
// Name: Barney Age: 10000
// Name: Barney Age: 10001
// Name: Pierre-Yves Age: 10001
```

Both `S` and `S.on` return a computation stream.  The difference is that `S` automatically tracks dependencies.  In general, it's idiomatic to prefer the usage of `S` over `S.on`.

`S` and `S.on` can also take an initial value, which can let us reproduce a scan of a stream like so.

```js
const num = S.data()
const sum = S( x => x + num(), 0)

sum() // 0
num(1)
sum() // 1 (0 + 1)
num(2)
sum() // 3 (1 + 2)
num(3)
sum() // 6 (3 + 3)
```

Other libraries might use a method like:

```js
stream.scan( 
    (previous, next) => previous + next
    , seed
    , stream 
)
```

Because in S it is idiomatic to automatically track dependencies, we both do not need the `next` value, and we don't need the `stream` argument.  Which is why the signatures are so different even if the result behaviour is the same.

---

Now if you were following along in a REPL, you may have seen an error message warning that without a `root` any created streams will never be cleaned up.  This is one of those magic rough edges I mentioned in the beginning.  What is a `root`?  And how do you clean up streams in `S`?

Well `S` has a really cool super power that other stream libraries tend not to have.  Because stream usage is automatically tracked, stream's can also be automatically cleaned up when the parent context is disposed.  And because the parent context follows the same rules: all streams are automatically cleaned up.  But there's one caveat.  What if there is no parent context?  If there's no parent context you can't have automatic clean up.

Enter `S.root`.

```js
S.root( dispose => {
    const name = S.data('James')

    setTimeout( () => {
        // dispose this stream context in 10 seconds.
        dispose()
    }, 10000 )
})
```

`S.root` fills the need of a top level parent context for top level streams and computations.  Now - I'm not sure if `root` is really necessary.  And `root` and the cleanup mechanism, at the time of writing this sentence, still kind of mystifies me.  But by the end of this post we'll have figured out exactly the requirements, mechanisms and tradeoffs of `S.root` and automatic clean up.

Speaking of `cleanup`, S does automatically handle it, _but_ we can subscribe to the cleanup phase via `S.cleanup`.


```js
S( () => {
    name()

    S.cleanup( lastTime => {
        if( lastTime ){
            console.log('name stream is being cleaned up')
        }
    })
})
```

This let's us deregister event listeners, or do some other kind of clean up side effect when a stream is about to be GC'd.

There are a few other mechanisms and methods on the S API, but that's a pretty good walkthrough of the core functionality, enough for us to get started.

## The Source Code

There's not an exact methodology we are going to use to explore the source code.  But a high level process will be, we start top to bottom, as we encounter something that doesn't make sense, we explore it immediately.  When we finally understand that current mystery, we go back to our original place in the top to bottom scan.

If along the way, we find additional mysteries or points of confusion, we add them to our stack of things to search.  We'll explore things on that stack before returning to our scan.

That way, we're building up mastery.  We won't proceed until we've understand (to some degree) a particular subsection, or abstraction.  We're going to be slow but thorough.

The code we will be referencing for this post is on the project's github [here](https://github.com/adamhaile/S/blob/634e5fb5c1a6726a8eaf0ade1d13a719effea3ed/src/S.ts).  We're linking to a specific git ref, so that when the code updates this blog post will not get out of sync with the referenced code.

### Type Exports

The top of the file starts with some type exports.  This is a pretty common practice in type driven design.  If we explore and understand these types before reading the source, hopefully we can have a better grasp of the following code.  The types are a way to read the intention of the author and the library.

Here is the entire interface.

```ts

export interface S {
    // Computation root
    root<T>(fn : (dispose? : () => void) => T) : T;

    // Computation constructors
    <T>(fn : () => T) : () => T;
    <T>(fn : (v : T) => T, seed : T) : () => T;
    on<T>(ev : () => any, fn : () => T) : () => T;
    on<T>(ev : () => any, fn : (v : T) => T, seed : T, onchanges?: boolean) : () => T;
    effect<T>(fn : () => T) : void;
    effect<T>(fn : (v : T) => T, seed : T) : void;

    // Data signal constructors
    data<T>(value : T) : DataSignal<T>;
    value<T>(value : T, eq? : (a : T, b : T) => boolean) : DataSignal<T>;

    // Batching changes
    freeze<T>(fn : () => T) : T;

    // Sampling a signal
    sample<T>(fn : () => T) : T;

    // Freeing external resources
    cleanup(fn : (final : boolean) => any) : void;

    // experimental - methods for creating new kinds of bindings
    isFrozen() : boolean;
    isListening() : boolean;
    makeDataNode<T>(value : T) : IDataNode<T>;
    makeComputationNode<T, S>(fn : (val : S) => T, seed : S, orphan : boolean, sample : true) : { node : INode<T> | null, value : T };
    makeComputationNode<T, S>(fn : (val : T | S) => T, seed : S, orphan : boolean, sample : boolean) : { node : INode<T> | null, value : T };
    disposeNode(node : INode<any>) : void;
}
```
Let's work through this bit by bit.

```ts
root<T>(fn : (dispose? : () => void) => T) : T;
```

This signature just means, root is a function that accepts a function.  The `<T>` `=> T` and `:T` means, whatever the function `fn` returns, root will _also_ return a value of the same type.

So `S.root( () => 0)` would return `0`, because the `fn` `() => 0` returned `0`.

Why is that useful?  I'm not sure.

Next up computation constructors, let's start with the variations of `S( () => ... )`

```ts
<T>(fn : () => T) : () => T;
<T>(fn : (v : T) => T, seed : T) : () => T;
```

Let's start with the end there, the `: () => T` means, this expression returns a function that returns type `T`.  That is our getter function mentioned earlier.

Our `S` expression accepts a function `fn`, which simply returns a value of type `T`.

The second signature just says, you can also provide a `seed` value, that must be of the same type `T` than `fn` returns.

This is all in line with the usage code we've seen before.

```js
S( x => x + num(), 0)
```

`num` is a number stream, being added to `x`, also a `number` and `S` is taking a seed value of `number`.  So our type `T` is `number`, and the type signature enforces all those types need to line up - which is a good thing.  That will avoid a lot of common bugs e.g.

```js
1 + "0" // "10"
```

The `S.on` signatures are very similar except they accept a list of streams and there is a `onchanges` boolean I didn't mention earlier.  `onchanges` is just an option that forces `on` to only react to subsequent events, not the existing value in the referenced streams.

```ts
on<T>(ev : () => any, fn : () => T) : () => T;
on<T>(ev : () => any, fn : (v : T) => T, seed : T, onchanges?: boolean) : () => T;
```

Note, our `ev` is a function that returns `any`.  That's not really accurate, that's just the type system not being sufficient to model the behaviour of `on` so they are default to `any`.  This isn't taking into account that `ev` could be a list of streams of different types.

This isn't a huge deal, but is common in typescript code, you can't accurately or easily model a lot of common scenarios in Javascript, like the type of a list that represents a a dynamic tuple.  Maybe it is possible in the future or even now with a special incantantion.  But it's not simple.  So we tend to see `any` appear.  `any` just tells typescript to accept any value as valid.

`effect` is next, we didn't cover that yet in the usage guide.  It is not part of the public documentation, so we won't spend much time on it unless it becomes important to understand later usage code.


But it looks like `S` and `effect` have the exact same type signature _except_ there is no returned stream.  We can infer from these signatures that `effect` is just calling `S` internally and not returning the result.  Let's have a look at the types, and then the source.

```ts
      <T>(fn : () => T) : () => T; // S(...)
effect<T>(fn : () => T) : void;

      <T>(fn : (v : T) => T, seed : T) : () => T; // S( ... )
effect<T>(fn : (v : T) => T, seed : T) : void;
```

And here is the source for `effect`.

```ts
S.effect = function effect<T>(fn : (v : T | undefined) => T, value? : T) : void {
    makeComputationNode(fn, value, false, false);
}
```

And the source for `S`

```ts
// Public interface
var S = <S>function S<T>(fn : (v : T | undefined) => T, value? : T) : () => T {
    // ...

    var { node, value: _value } = makeComputationNode(fn, value, false, false);

    if (node === null) {
        return function computation() { return _value; }
    } else {
        return function computation() {
            return node!.current();
        }
    }
};
```

So `S` internally calls `makeComputationNode` and returns a getter.  `effect` internally calles `makeComputationNode` but doesn't return a getter.  So we can see we can definitely make reasonable inferences about source code by simply reading types.

We'll return to `S` source after dissecting the types.

---

Next comes the `data` and `value` constructors.

```ts
// Data signal constructors
data<T>(value : T) : DataSignal<T>;
value<T>(value : T, eq? : (a : T, b : T) => boolean) : DataSignal<T>;
```

I didn't cover `S.value` yet, but `S.value` is just `S.data` that diffs event values and only emits if the value changed.  `S.value` also offers an optional equality function `eq`.  Note they both return a `DataSignal<T>`.  From that we can assume value internally calls the same or shared code as data, but adds some additional diffing logic.

But what is a `DataSignal`

As far as the types are concerned, it is a getter setter function.

```ts
export interface DataSignal<T> {
    () : T;
    (val : T) : T;
}
```

---

We never covered `freeze` or `sample`.  I'd like to cover `freeze` later when we get into `RootClock`, but at a high level, it lets us perform writes that do not trigger the dependent computations to be updated until after the `freeze` block.  If that doesn't mean anything or make sense yet, no problem we'll get to it.  As the comment puts it, it's relevant for batching changes to multiple streams.

```ts
// Batching changes
freeze<T>(fn : () => T) : T;
```

`sample` allows us to retrieve the value of a stream without triggering the recording of an automatic dependency.

```ts
// Sampling a signal
sample<T>(fn : () => T) : T;
```

For example the following sample will rerun any time `a` or `b`'s value changes.

```js
S( () => {
    console.log(a(), b())
})
```

But this sample will only update when `a` changes:

```js
S( () => {
    console.log(a(), S.sample(b))
})
```

Both examples will log the same thing, but the latter doesn't create a dependency from `b`, it just reads the current value and that's it.  If you find you have a lot of `sample` calls, you might want to use `S.on` instead of `S` for example.

```js
S.on(a, () => {
    console.log(a(), b())
})
```

The above only logs when a changes, even though we call `b`, thanks to `on`'s explicit dependencies.

---

`cleanup`.  The most mysterious process in `S` as far as I'm concerned.  It's also perhaps it's best feature.  Most other libraries require some manual clean up of streams.  It can be a chore.

The type for the cleanup listener, isn't all that useful to gain our understanding of the internals, but that makes sense, it's only a subscription.

```ts
// Freeing external resources
cleanup(fn : (final : boolean) => any) : void;
```

We've got a function that accepts a boolean which if `true` informs us this is the last time cleanup will be called.  Now a thing that confuses me about this is, why would cleanup ever be called more than once?  That seems like a strange design decision.  We can also see `cleanup` returns `void` which makes sense.  Depending on a cleanup value could cause some strange problems.

After `cleanup` in the types, we see some `experimental` annotations.  We won't dive into them yet as they may not be stable and therefore not required for us to understand the current published stable version of `S`.

But here they are:

```ts
// experimental - methods for creating new kinds of bindings
isFrozen() : boolean;
isListening() : boolean;
makeDataNode<T>(value : T) : IDataNode<T>;
makeComputationNode<T, S>(fn : (val : S) => T, seed : S, orphan : boolean, sample : true) : { node : INode<T> | null, value : T };
makeComputationNode<T, S>(fn : (val : T | S) => T, seed : S, orphan : boolean, sample : boolean) : { node : INode<T> | null, value : T };
disposeNode(node : INode<any>) : void;
```

The most interesting method here to me is `disposeNode`.  That implies there are some occasions where manually disposing a node is useful.  Maybe we can dig into the issues later to figure out how that arose?

### S( ... ) _the function itself_

```ts
// Public interface
var S = <S>function S<T>(fn : (v : T | undefined) => T, value? : T) : () => T {

    if (Owner === null) console.warn("computations created without a root or parent will never be disposed");

    var { node, value: _value } = makeComputationNode(fn, value, false, false);

    if (node === null) {
        return function computation() { return _value; }
    } else {
        return function computation() {
            return node!.current();
        }
    }
};
```

This is the centrepiece of the library.  And it's nice to see it's not many lines.  We've already covered the signature.  It's a little weird the signature has to be reannotated but that's just Typescript I guess.

The first line is already interesting.

```ts
if (Owner === null) console.warn("computations created without a root or parent will never be disposed");
```

Where was `Owner` defined?  This begins our journey into S' globals!  In order to have automatic dependency tracking, you sort of need globals.  And that's something I'm not very comfortable with but can accept is a requirement for this S model.

So let's find `Owner`.

We'll, all the globals are defined halfway down the page... which is even stranger.   But here goes:

```ts
// "Globals" used to keep track of current system state
var RootClock    = new Clock(),
    RunningClock = null as Clock | null, // currently running clock 
    Listener     = null as ComputationNode | null, // currently listening computation
    Owner        = null as ComputationNode | null, // owner for new computations
    LastNode     = null as ComputationNode | null; // cached unused node, for re-use
```

We don't know what any of these things are, but we can see that all the Globals are either 1 of 2 nullable types: `Clock` or `ComputationNode`.  We don't know what either of those things are yet, but it's good to know there's not too many global nouns to be aware of.

Good thing is, returning back to our `S` source code, the next line forces us to find out what a `ComputationNode` is.

```ts
var { node, value: _value } = makeComputationNode(fn, value, false, false);
```

Without jumping into `makeComputationNode` yet.  We can infer a lot.  Firstly, this function / type is super important.  It's already been referenced several times in this post, and it's central to the function that shares the name of the library, and is the entry point to the entire system.

So, when we read `makeComputationNode` we can infer we should pay special attention to the inner workings.  Every detail is likely relevant to our quest.

We can also see `makeComputationNode` returns not just a node, but a `value`.  Why return a value instead of storing it on the node itself.  We shall see...

Let's dive in!

### `makeComputationNode`

`makeComputationNode` is a big function.  And it references other functions and external state.  So we're going to work through it in piecemeal, jump around and then return for a macro view later on when we have a more complete understanding.

First line:

```ts
var node     = getCandidateNode(),
```

What is a candidate node?  Well, I'm goign to _assume_ it's some process of inferring which node we are currently in the context of using the global state variables we covered earlier.  And if there's no node, we maybe throw, or create a new empty node?

Let's find out...

```ts
function getCandidateNode() {
    var node = LastNode;
    if (node === null) node = new ComputationNode();
    else LastNode = null;
    return node;
}
```

I was _so_ close.  So we have a global `LastNode`.  If that node is `null`, we make a new one.  If it isn't `null`, we set it to be null thereby claiming it for ourselves!  At the end we return the node.

Now how and why could `LastNode` be null?  Presumably when we're at the start of a tick?  If that is the case, I'd prefer there was some kind of global state that represents the current phase, and we checked that and acted accordingly, instead of checking if things are `null`.  But that's me.

But maybe there's other reasons, we'll find out.  So let's jump to the second line of `makeComputationNode`:

```ts
owner    = Owner,
```

So what's `Owner`?  I'm guessing it's the `ComputationNode` that controls whether or not this node will be disposed?  Or maybe if the `Owner` node updates, we need to rerun our computation?  A mystery to solve!

Next line:

```ts
listener = Listener,
```

I'm not sure what a listener is in the context of S, but I'm guessing it is the function itself.  Let's scan the library.  Let's look at the type signature:


```ts
Listener     = null as ComputationNode | null, // currently listening computation
```

Okay, so I was wrong, it's a computation node, just like the `Owner`, and the node we are making in this function.  So this process is largely recursive...

What does `Listener` mean though?  What is a `currently listening computation`?  We'll find out I guess.

```ts
toplevel = RunningClock === null;
```

This line is thankfully easy to infer, if there's no `RunningClock` we must be at the top of dependency tree.  It's just a local human readable flag.  But we still don't know what a `Clock` is.

```ts
Owner = node;
Listener = sample ? null : node;
```

We've now set the Owner to be this new node.  And the listener to be null if we're sampling, or the current node if we aren't.

So every time we call `makeComputationNode`, the Owner switches.  It's a bit like changing the coordinate system.  The rest of the process now sees our new node as the root where subsequent computations will be dependencies of.

I'm not sure why a computation node can sample, I thought you could only sample when retrieving a node not making one.  So that's a bit of a mystery to me.

```js
if (toplevel) {
    value = execToplevelComputation(fn, value);
} else {
    value = fn(value);
} 
```

If we're top level, we use this special `execTopLevelComputation` fn, instead of directly applying it otherwise.  There might be some setup/teardown global management that happens for the top level computation that doesn't otherwise happen.

Is the top level computation `S.root( () => ... )` itself?  I'm not sure yet.  Let's look at `execTopLevelComputation`

```ts
function execToplevelComputation<T>(fn : (v : T | undefined) => T, value : T | undefined) {
    RunningClock = RootClock;
    RootClock.changes.reset();
    RootClock.updates.reset();

    try {
        return fn(value);
    } finally {
        Owner = Listener = RunningClock = null;
    }
}
```

---

```ts
function makeComputationNode<T>(
    fn : (v : T | undefined) => T, value : T | undefined, orphan : boolean, sample : boolean
) : { node: ComputationNode | null, value : T } {
    var node     = getCandidateNode(),
        owner    = Owner,
        listener = Listener,
        toplevel = RunningClock === null;
        
    Owner = node;
    Listener = sample ? null : node;

    if (toplevel) {
        value = execToplevelComputation(fn, value);
    } else {
        value = fn(value);
    } 

    Owner = owner;
    Listener = listener;

    var recycled = recycleOrClaimNode(node, fn, value, orphan);

    if (toplevel) finishToplevelComputation(owner, listener);

    makeComputationNodeResult.node = recycled ? null : node;
    makeComputationNodeResult.value =  value!;

    return makeComputationNodeResult;
}
```