# ideas

## return to. revision of. and summary of useful app infrastructure ideas

There are two things here. A JS utils facility. And a server side rendering facility.

### JS Utils Facility

This is mostly for involved JavaScript in settings where involved user input is required. Such as in Project Ant and in Chrome Extensions and Electron Apps.

The utils include:
- module loader
- task runner that supports synchronous and asychronous, serial and parallel execution and arbitrary composition of tasks.
- runtime type insurance
- services for input event recording and synthetic and authentic ( via privileged APIs such as debugging protocol of webview tag APIs ) event dispatch and event sequence playback.
- arbitrary capability composition by multiple prototype delegation
  - prototypes are used instead of classes and composition is used instead of "inheritance"

**Note:** There is no rendering facility in the JS Utils. Any requiredments for dynamic HTML render go to a server endpoint like this:

`GET /render/<model_name>/<model_id>/into/<view_name>`

Or, where the model details are held in the client like this:

```
POST /render

<model_name>
<model_details>

<view_name>
<view_details>

<slot_map>
```

As a set of alternates to the second option any fields that are provided in the request body can be moved to the request URI by reference ( such as name ), if the dereference of that name is possible on the server.



### Server Rendering Facility

#### Technical Preface

- Write our own HTML renderer.
- Very simple. We write our views in a reduced set of HTML. This set is very easy to parse.
- One item per line. No angle brackets.
- Items can be either tags, attributes or text
- Tags start lines with
- Attributes start lines with -
- Text starts line 

```
// syntax initial sketch
main/
- class free joy smile
- data-hello excellent
div#happy/
- class big small 
p/
so this is a paragraph
/p
/div
/main


// syntax simplified sketch
form
method put
action /nice
input 
value awesome
type text
name thebest
.
textarea
!cris is really so cool
.
button
..
```

tags start line with nothing special
they are a single word on the line
tags end with a `.` or multiple `.` on a line to close multiple tags
attributes are either a single word followed by space separated words
or for just the presence attribute a single word followed without space by `!`

as in

```
div
conteneditable!
!hello
.
```

will produce

`<div contenteditable>hello</div>`

Text starts with `!`

So our templates are written like this

The syntax for variable binding shall follow.

Incidentally, the second statement in the above sketch will produce the following HTML when passed into the flow

```html
<form method=POST action=/nice>
  <input value=awesome type=text name=thebest>
  <textarea>cris is really so cool</textarea>
  <button></button>
</form>
```

---

#### Interpolation and Inference

We don't need to write everything.

For example

A URI on a line by itself that points to an image file extension will become an image tag.
Same for other media types, including JavaScript. They all become their respective tags.
Since we do not worry about stylesheets ( since styles are applied in a separate flow ) we do not 
produce style tags. 

An other URI on its own line becomes an anchor tag

A URI followed by text becomes an anchor tag if there are spaces in the URI the continuation operator must be used and the text put on the next line.

POST or GET followed by a URI becomes a form tag

charset followed by a strings becomes the relevant meta tag

We can add a whole bunch of these inference macros. The language can be customized with a separate "inference" file that describes the intferences it supports.

#### Continuation

`~` is the continuation operator. It is important for how we insert slots into the HTML.

Slots are identifiers. Slots can later be linked to slots in a model via a slot map.

For example to include a slot in an attribute on a div look at this

```
div
class ~
:spoken~
red bright full
```

Will produce ( when spoken is bound to "balloon" )

`<div class="balloon red bright full"></div>`

We choose the contunuation operator since it makes parsing easy. And it is also easy to write.

To parse we just need to check if a line ends in the continuation operator. Then we keep adding lines so long as that is true.

##### Continuation Corner Cases

What if we want to actually have a line that ends in ~ such as a text line like

```
!hello my name is Cris your writing is so funny~~
```

In our language ~ is the last character on a line. No character can follow it. So how can the following line be produced?

!hello my name is Cris your writing is so funny~
~~
~

This is because you cannot continue an empty line. 

So what this means is that:


`!` start a text line

`hello my name is Cris your writing is so funny` add that to the text line

`~` continue text line on next line

`~` add that to the text line ( since the continuation character cannot be the first character since that would be redundant ) 

`~` continue text on next line

`~` add that to the text line


And that is the last character of the text line producing the original intent.

Okay that is good what if we want to just break our text into multiple lines for the sake of typing and viewing the document for editing?

Obviously the whitespace will be ignored by the HTML ( unless inside a pre tag ). So how do we include whitespace?

````
!This is a text line that will soon break~
 ~
The line broke and it~
 broke again and~
 again
 ```
 
Notice the single spaces that start the line? These activate the newline whitespace so that it is preserved. If the first character of the continued line was not whitespace the newline would not be activated. 

#### One Point

All render happens on the server. This is so that we have a single service that can render across the following media:

- Web browser handheld device
- Web browser keyboard device
- Email
- Web crawler ( that does not process or does not always process JavaScript or processes some unreliable specified possibly unknown and inconsistent-across-versions-or-crawlers subset of JavaScript )
- "Low capability" devices
- Other HTML rendering devices

This gives us some immediate advantages:

- Unified, controllable experience
- Consistent, controllable speed
- Freedom from JS frameworks
- One conceptual model for building application works across diverse media
- Our applications views are very clearly HTML documents
- SEO can be utilised effectively
- Content can be discovered organically
- Links really do work and make sense
- Style encapsulation vim `style` attribute styles.

#### Style Encapsulation

If you consider the last point, style encapsulation, and combine it with the other point of having this facility -- the resue of HTML views, then you pretty much get the benefits of Web Components ( without all the JS API ) -- being componentized HTML and style encapsulation, without any JS. Pretty cool. 

The only thing necessary is to write a basic "style applier". This will take a file of simple single ID or class selectors in standard CSS format and apply it to the HTML as style attributes.

At first it will not support combinators such as "descendent combinator", "sibling combinator" and so on. Nor will it at first support arbitrary attribute selectors. These things could be added later but why bother? IDs and classes will provide most of the facility we need. 

the crucial thing to note is that this style applier works local to a view. So we pass it a view and a set of simplified ( the subset of CSS we just described and shall support ) stylesheets and it applies those styles to that piece of HTML only. So there are no issues with the same class existing anywhere else in the final rendered document. Styles are scoped to views. End of story. 


It also provides some opportunities:

- No AJAX
- Nothing but net ( nothing but native input elements ).
- No iframes
- No "transformative" JavaScript like CustomElements or other goodies.
- No assumption of JavaScript

Here are some consequences of these things:

#### No AJAX

- Everything is GET or POST. 
- Everything is done via:
  - forms
  - anchor download attributes
  - anchor ping attributes
- Everything is encoded in browser default transport formats: multipart/form-data application/x-www-url-encoded
- No JSON

**Note:** this does not mean that there can be no:
- server-client separation
- JSON providing APIs

It just means that we need to have a service, likely exposed through a single endpoint that renders models and views together, like this:

`HTTP GET /render/<model_name>/<model_id>/into/<view_name>`

And further, as per my previous ideas, you can decouple models and views by using "slot maps".

Slot maps are just JSON files that link selectors for slots in a view to slot names in a model.

A slot map will be automatically selected by the render service once it knows the model and view names. Since any slot map is clearly uniquely defined by the model and view between which it provides a mapping of slots.


#### Nothing but net ( nothing but native input elements )

This presents some unique opportunities perhaps workably stated through questions:

- how do we customize the look of native input elements?
- how do we provide useful facilities to enter information like tags and dates?
- how do we do reCAPTCHA and other anti spam measures?

My default position on questions like these is:

**We don't**

We prefer simplicity and server solutions over adding more to the documents we send to the client.

Also, if it does work to adopt an alterate positions here are some specific considerations that could work:

- CSS can be used to some extent to alter the appearance of native input elements
- Useful facilities to enter tags and dates can either be throughtfully constructed without javascript using only native input elements composed in delightful and thrilling compositions. Or, once this is achieved, a small JS file can be used to upgrade the "native" experience to one that uses JavaScript if it is available. 
- The same standard and upgrade approach can be used for a lot of things. Many of the types of captcha can be implemented using standard HTML forms and images, for example.

## Some thoughts on the absolute simplest human interface framework for a web app that uses APIs and HTML components

- must have HTML components
  - but may not need to do rendering on the client. Could be done on the server.
  - simplest way is to just use IFRAMES. And IFRAME can:
    - reload itself without reloading the frames it is embedded in
    - perform GET requests ( Anchor Tags ( href attribute ) and Forms ( action attribute ) )
    - GET requests can also be performed by:
      - SCRIPT tags ( src attribute )
      - LINK tags ( href attribute )
      - Other tags in HTML5 such as:
        - `link[rel="import"]` ( href attribute )
    - perform POST requests ( Anchor Tags ( ping attribute ) Forms ( action attribute ) )
  - in this simplest way POST data is sent in the type of 'application/www-form-url-encoded' or 'multipart/form'
  - in this simplest way JavaScript does not need to be used:
    - if you use CSS to define the size of the IFRAMEs so they can accomodate the components they will embed
    - if you use the `_target` attribute of anchors and forms to specify the IFRAME where the result of the request will appear
  - the only issues with all this are:
    - do we get search engine indexing ? in other words, do search engines index iframes?
      - it appears we do not, so an alternative is
      - every iframe is accompanied by an anchor link that points to the page it embeds
      - and every iframe is only ever used for 1 type of component ( since iframe SRC attributes DO NOT update on navigations of that IFRAME after being initially set )
      - Unfortunately, it seems this sort of javascript-free paradise suffers from the issue that indexing is not perfect
  - html components are used and recognized only the server. 
- must be server side rendered, then indexing will be perfect, and all clients will recieve the same page. 
  - component compositing is actually pretty easy:
     - we can cache in memory rendered sub components just as we cache models from the datastore
     - achieving the composit render is pretty much just string concatenation of cached components
     - simple and fast, really no different to sending back a JSON representation 
     - in this case we have:
        - an application-independent database service, and 
        - an application-specific view component service:
          - basically an adapater, for the application and for each application using those views, around the database
          - another way to see the application-specific view component service, is as a sort-of content delivery network
  - in this case we use javascript only for upgrade of functionality. 
  - but crucially, not for upgrade of the way we deliver functionality 
- this server side rendering seems ideal, because we achieve consistency of user experience, and search engine indexing. 
- however, what about cases where a server is not available to render our views? such as in Chrome extensions or in Electron apps?
- in this case it seems to make sense to have a client framework for web components that matches the semantics of the one understood by the server. So that the same pieces of HTML can be used in either. In other words, there is a 'rendering service' for HTML and whether it exists on the client, or on the server, doesn't matter for the application, or for the human. 
- because the semantics of such a componentized html render will be constrained by the JavaScript version of it, since the JavaScript version will seek to leverage pre-existing APIs such as those for Web Components and Shadow DOM v1, it works to design this JavaScript first, and also with the server requirement in mind.
- so a possible workable plan is:
  - Once the JavaScript components are done, build a server based service capable of producing identical rendered output, thos without any JavaScript executing on the client. 
  
## what if we do not use HTML components

- in that case a much simpler framework is possible. 
- we do not get to reuse HTML but we do not have to worry about rendering. 
- we end up writing more HTML and possibly more pages, but it is sipler HTML and we get to have a much simpler javascript framework.
- the advantage of NOT having HTML components is we get to automatically have search engine indexing, and we look like a static site, but with some upgrades. 
- in fact, it can be seen that the need for componentized views, actually introduces a LOT of additional requirements and involvement, that, logically, is only worth it, if the effort expended is less in aggregate than the other effort that such expenditure is trying to avoid: namely, the other effort of the writing of additional pages. It would actually be possible to quantify this somehow. In any case, it is clear that intuitively, if the target site has only a few pages, then the effort of adding componentized views is wasted. 
- this neat consideration of trade offs does not consider the need for templating:
  - data is returned from a server, in some format, and either it must already be in HTML, or
  - it must be put into HTML
  - the issue that detracts from our neat consideration of trade offs is that:
    - in either case, whether the rendering content already being in HTML, or
    - the returned content requiring being put into HTML
    - WE MUST have componentized HTML at least to some extent since
      - in the first case the server has created a piece of HTML
      - and in the second case the client is aware of how to create, or put into, a piece of HTML 
    - but there is an, albeit, likely-to-some strange-sounding solution
  - never use formats. All data that comes back from the server is ONE SINGLE SLOT.
  - In  other words, the ONLY composition of views, is that which is:
    - specified by the HTML
    - and occurs on the client, or the server. 
  - 1 slot = 1 request
  - A template that looks something like: `<a href="<slot url=scheme://slotprovider/path/to/slot>"><slot url=scheme://slotporivder/path/to/slot2></a>`
  - Okay, but isn't this torturous to write?
  - The API is basically _over-normalized_ 
  - It just doesn't make a lot of sense to write like that. 
  
## Okay, so HTML components _are_ necessary, how to do them in the simplest way possible?

- An interesting question. Worth more thought.
- One of the problems with views is that they are bound to models in their commonest incarnation.
- A view is typically a piece of templated HTML. The templates do two horrible things to the HTML:
  - the make it no longer really like HTML, since it is now filled with template literals.
  - the bind it to the model it is intended to template
- The question is: why does this have to be so?
- The answer is: it does not.

# Alternatives to Model Bound HTML views

- The most obvious alternative is a single view, that can be used with multiple models.
- How might this be implemented?
- With a map, of course, from model slot paths to HTML pieces.
- The obvious way is to use a type of augmented CSS selector, one which can specify the HTML piece completely ( as CSS selectors can )
- And then which goes on to specify which part of that HTML piece is used, `:innerText, :attr-href` and so on.
- If this is the case, why not use something like stylesheets for these maps?
- Stylesheets are used to apply _styles_ to elements. 
- Why can't "datasheets" be used to apply _data_ to elements. Or "modelsheets" or "slotsheets" or something like that, anyway.

### Examples of Cascading Data Sheets 

Which are the maps between models' slots and views' parts.

- We can use the content attribute ( which is only ever used in CSS for :before and :after ), to apply this. 
- For instance:

```css
// datasheet.cds
#view > #name > #first {
  content : model.firstName;
}
#view > #name > #last {
  content : model.lastName;
}
```

- And everything was good in the universe!
- It turns out that we can't use content unless we are using `:before` or `:after`
- A further example, consider:

```html
<title>CDS test</title>                                                                                          
<style>                                                                                                          
  #view > #name > #first {                                                                                       
    --var-part : innerText;                                                                                      
    --var-key : model.firstName;                                                                                 
  }                                                                                                              
  #view > #name > #last {                                                                                        
    --var-part : innerText;                                                                                      
    --var-key : model.lastName;                                                                                  
  }                                                                                                              
</style>                                                                                                         
<h1>CDS test</h1>                                                                                                
<div id=view>                                                                                                    
  <div id=name>                                                                                                  
    <div id=first></div>                                                                                         
    <div id=last></div>                                                                                          
  </div>                                                                                                         
</div>                                                                                                           
```

Then the part and key properties can be accessed like this:

```js
const declaration1 = document.styleSheets[ 0 ].rules[ 0 ].style;
const declaration2 = document.styleSheets[ 0 ].rules[ 1 ].style;

console.log( "First name part ", declaration1.getPropertyValue( "--var-part" ), " first name key ", declaration1.getPropertyValue( "--var-key" ) );
console.log( "First name part ", declaration2.getPropertyValue( "--var-part" ), " first name key ", declaration2.getPropertyValue( "--var-key" ) );
```

And it would not be very hard to specify and to write a simple function that iterated over the chosen stylesheet rules, and applied them to the model. Something like

```js
function template( model, view, datasheet ) {
  // iterate over datasheet rules
  // use view.cloneNode( true ), to clone a view piece.
  // or you could parse it from a HTML string using DOMParser parseFromString or Element insertAdjacentHTML
  // use view.querySelector( rule.selectorText ) to get the HTML piece
  // use getPropertyValue( "--var-part" ) to get the part of that HTML piece
  // use getPropertyValue( "--var-key" ) to get the key of the model slot
  // resolve the model slot key by splitting it across '.' and finding it in the model
  // apply the slot value to the specified piece, and
  // repeat the process!
  // returning the templated view
}
```

## So - Cascading Data Sheets

- are a way to stop views depending on models. 
- And probably to do some other cool things, too. 
  - Hello, scopes! Thanks to the cascade
  
## Sugar

Lots of sugar can be added as well. Like

```html
<link rel="alternate stylesheet" href=views-for-user.css title=person-user-datasheet>
<article id=person-view>
  <!-- ... !-->
</article>
<script>
  fetch( new Request( `/user/${ getOwnId( ) }` ) ).
    then( response => response.json( ) ).
    then( model => template( model, document.querySelector( '#person-view' ), getDatasheet( 'person-user' ) ) );
</script>
```

The great thing is, *all alternate stylesheets* are parsed into the `document.styleSheets` list and given their corresponding title attribute. So it's really easy to find the data sheet you want. And once you found it, iterating through its cssRules and getting their style property ( for the `CSSStyleDeclaration` interface ) is so easy as well. Once you're at that point, getting the already parsed data is so simple: you've got the `selectorText` and `part` and `key` values already available. No need to explicitly do anything, the browser has already parsed all this for you. 

# more goodness

There other thing is, once an app specifies a view using a tag name and a model type and ID using the model attribute the renderer, on the client or server, automatically fetches the view HTML file, the model, in whatever format works, like json, or JavaScript object, or Python object or Python dictionary or whatever, and also the data sheet corresponding to this model type and view pairing, in whatever format works such as CSS for client render, or JSON for server renderer. On the server the data sheets are stored as objects.

# project ant connection

These data sheets are also what we use to store or at least represent the mapping of structure to semantics ( data types and semantic categories ). Indexed by the full app view locator or degenerate locators, these data sheets represent semantics typically availably at that app view. We say typically because dealing with apps at such a level of sophisticated control, there is a lot of drift, non determinacy and probability associated with whether the semantics are available or not. We keep score of whether, the mapping selectors are available, over the times the app view was loaded, and whether the semantics were reported correct or incorrect, by a human or other capable agent, over the times the app view was loaded as well. We also keep track of time. These simple score let us identify maps that it could work to improve, and contribute to our view of a maps relevance.

# connection with web components

Actually at first glance it seems this way of terminating views is incompatible with web components. Because we are advocating a separation between model attribute and view parts, while web components makes th