Developing Ink Components

components.ink is trying to expand on and explore some of the ideas written in:
components.ink is my experiment in creating tools for interactive scientific writing, and developing web-components that improve that experience so that you, as a scientist and communicator, can inject interactivity into your writing. My goal is primarily to support scientific writing. I have been experimenting on my own website, and taking some of my scientific articles, my experience in collaborative scientific textbooks and rethinking them for the interactive web. There are, of course, many existing tools out there that are also are trying to do similar things.

A smattering of notable examples:

• Tangle: explorable explanations made easy
• Distill: a modern medium for presenting research
• eLIFE: improve research communication through open science and technology innovation
• dokieli: decentralised authoring, annotations and social notifications
• Idyll Lang: a toolkit for creating data-driven stories and explorable explanations

These are some projects shaping my thinking behind components.ink. In this document, I want to run through some of my motivations, experiments and comparisons to a few other tools.

A scalable, interactive document

My first interactive document that I wrote was in December 2013 with my sister and my dad! We interpreted and dissected a government report on the probability of an oil spill. Beyond the content of the article, I think there were a few things that were interesting about presenting this work: (1) an interactive figure linked to text, equations and tables; and (2) it is a scalable document - going from a picture to a mathematical proof with infinite sums in one document. Interactive figures have been explored numerous times; explorabl.es shows a number of these sorts of documents.

The idea of a scalable document, I think is slightly more novel, and is trying to challenge how to present ideas that satisfy the interplay between two very different personas. In this case: (1) my dad, a computer scientist and (2) my sister, a social activist. Putting a mathematical proof detracted from the single paragraph of content that my sister cared about - but it also shows a depth of analysis that helps to back up the article.

Having more to say, including all sorts of details is something that I have fought with in my scientific articles. I work with collaborators for hours on whittling down a paragraph to a single statement, or weed out intermediate steps that the reader has to reconstruct on their own. There is value to the brevity: clarity of thought; however, it comes at the expense of context. I would have written a shorter letter, but I did not have the time. When re-thinking our medium of creating and consuming, I believe that we should also not have to compromise on the trade-off between clarity and context.

Moving on from Tangle

I developed my early interactive documents using tangle.js, it was easy to integrate, however, slightly harder to keep up to date as I continued to iterate on the writing. The separation of the logic of the interactivity (in Javascript) and the prose (in html markup) meant I was jumping around and getting lost all the time. I think this is especially true when I wanted to introduce three new ways of looking at the same model. The model and logic was the same, I wanted a new instance of the model in its own scope, so I could show how it works with different inputs or fixing a single variable - independent of the main document! but the easiest way to add this was by adding _example3 to my variable names in Tangle. There was not the idea of scopes or namespaces in my writing, something that is taken for granted in every modern programming language. This document has become one of my main test-cases for developing components.ink. Some of the interactivity that I required were:

• Non-linear sliders, with custom formatting
• Scopes to present multiple ideas as instances of the same model
• Logic and variables declared "simply" and "close" to where they are used in the text
• Easily include equations, figures, and charts that can interact with the variables driving the document

After a few iterations, refactoring the article using ink components has reduced the complexity to 42% of the size, in terms of lines-of-code written. There are still a number of outstanding issues though!

Extensible Markdown, xmd

My first approach was to boldly state I would invent a "new language" that would extend markdown to include some of these ideas. I started by writing a "spec" for the .xmd language, which is on GitHub, in collaboration with Lindsey under the GeoSci project. I spent a few months figuring out how to write parsers in Python and the pros and cons of extending a markdown parser or writing one from scratch. An excerpt of the specification is below, I wanted it to be trivial to insert, Jinja/Handlebars style, all sorts of variables as well as extend the markdown to use new commands.

After getting some proof of concepts, integrating Python extensions as simple classes and having an editor experience in Google Docs, I gave up when I hit my head against bracket-parsing and debugging my article.

The silly thing was that I was in a weird place between extending markdown and writing my own parser. Neither of those things were actually getting closer to my objectives for interactive documents, and frankly, I am not very good at writing parsers. I spent my time doing things I really didn't want to do.

I decided to move on from creating a domain specific language for the following reasons:

• Parsers are hard, and I kept on screwing it up
• I would have to implement syntax highlighters and helpers in everyones favorite code editors
• There is the burden of good documentation and debugging experiences for users of the new language

Building out the infrastructure for a new language is a lot of work, and much of that work is only tangentially related to making really good interactive documents. I gave up after trying for about a month or so. However, there are other projects such as Idyll Lang that are successfully committing to extending markdown. They note in their paper there are a lot of advantages after you get over some of the hurdles of parsing, language nuances, and compiler bugs. Including:

• Limited verbosity, as the language is domain specific!
• Multiple compile targets (e.g. html, $\LaTeX$)
• Access to the intermediate representation and state can lead to really interesting analysis of the documents, as well as potential for WYSIWYG editors.
• Easier integrations into upstream providers (e.g. Jupyter, R Markdown, Svelte etc.)

There is a lot of development that has gone into Idyll and the pieces around it (editors, components, publishing, etc.), which make it a really awesome tool and community. To be honest, I feel there is a lot of duplication in the work I have done with components.ink and Idyll. If we look at how to start a movement it is probably better for interactive writing generally for me to help join on with the Idyll work now that I have discovered it. However, I still have some experimenting to do and I think a few unique ideas. Some of which I am contributing back (e.g. display transforms and compiler bugs)!!

I have decided to push on with my own project and see what happens. Next up I will do a bit of comparison to Idyll. I have decided to use web-components over React. To get started with the library you can go to components.ink!

Comparing Ink and Idyll

The helloWorld example of Idyll is a reactive number display hooked up to an html range input, dragging the slider updates the formatted display.

Ink Components:

Clearly, xml is verbose and repetitive and ink-repetitive and </redundant> and the extra tags are distracting when writing - which could be argued as completely antagonistic to the writing process. However, other than that the two implementations are almost identical at first glance!

Comparison of Some Features

Expressions

Expressions that can be executed in Idyll uses value=`backticks`, Ink prepends a semicolon to the attribute :value="semicolon".

Variables

Derived and Data in Idyll are new components, in Ink they are just variables.
I was thinking about this for a while, and I think the variable should really just be set dynamically based on a function, I am not sure why we need another type (Derived) to do this?

Equations

Idyll has equations in their Presentation section with no reactivity, Ink allows you to nest a display inside an equation. I kept wanting to update equations by setting the :math="'x = ' + Math.round(x*10)/10" property, however, this kept dropping decimal places when it was an integer. Agh. Because of how the shadow-dom works, I could just copy the textContent, and then use ink-displays instead, directly in the XML.

Charts

Idyll has simple charts out of the box but no two-way binding of elements, Ink allows you to have drag-nodes.

Conditional Visibility

Ink takes a slightly different approach to conditionals, and I opted to go for reactive visibility through :visible, rather than if/else statements in Idyll.

Other Cool Ideas™

Transforms

One of the things that I couldn't do was transformed displays without creating a Idyll component. Adding a transform without replacing the format string allowed things like display "free" when value is 0, or do an emoji-sliding-scale™. I am curious how this changes the approach in nonlinear range sliders.

June 2019: This has now been added to Idyll!

Demos & Code

For a lot of the examples I have created a demo component that allows execution of the internals as well as the ability to copy the source code directly. Allowing the executed code to be identical to that which is being displayed cuts down on the verbosity.

Bind

The ability to update multiple variables in a single action, hover, click, drag, etc. This is possible, I am sure, by creating your own component in Idyll. The name, value, and bind syntax I think is a pretty elegant advanced feature that allows some interesting things especially in the interactions with charts or not updating variables directly (e.g. multiple coordinate systems).

Scopes

Mixing multiple scopes of variables is a wicked cool idea. I have introduced the ink-scope component that keeps variable and dynamic execution in different IFrames on the page. There is a lot to improve with being able to reference from other scopes, but I think it is the start of a cool idea.

Periodic

Having a periodic boolean flag on dynamic text input (e.g. ) that allows you to keep dragging while the number cycles through a range.

Format

Format can be set of the variable and cascades as the default formatting on any display.

Source

This is a whole can of worms. I want the ability to "import ideas" in the same way that you would import an image. This means adding a src attribute to the component so that they can be dynamically imported at run-time (or server-side rendering). At the moment, I think this exploration is the reason I am curious on forging ahead with my own implementation. There is dependency on the API format, which I am still thinking about.
Eventually this idea leads to equation and figure banks, that can be improved independently. More on importing ideas is here.

I think some of these ideas could be used to improve Idyll, and am hoping I can contribute ideas to that project if not code directly.

Summary

I have started contributing components.ink, which is aimed at interactive scientific writing and reactive documents. It is a very similar approach to Idyll Lang, however, it implements everything using web-components rather than introducing a new domain specific language. There are potentially a few advantages, however, if you are in charge of the compiler and therefore the language - really you can add whatever magic you want! At the very least, these ideas can be used to expand and improve Idyll, however, I will probably continue to iterate on components.ink as it's own project. 🚀