DEV Community

Re-Exploring Reactivity and Introducing the Observer API and Reflex Functions

Oxford Harrison on July 01, 2023

In the last few years, I have spent an insane amount of time thinking through reactivity on the frontend! In all that we've achieved in the space, ...
Read full post
Collapse
 
godswillumukoro profile image
Godswill Umukoro •

Hi Oxford, this is a crazy thought. Thoroughly enjoyed the analysis behind each of the APIs! Like, there's much I never really figured out before now.

At the end of the day, you realize that much of the sophistication around reactivity isn't really for the application, **nor for the developer. They just cater to "the approach"! For example, who really needs "functional" primitives? Is it particularly the runtime, the application, or the developer?

Not that these don't have a use case, but we seem to have made what's supposed to be the exception the default!

I think the Observer API and Reflex Functions will be a huge leap for reactivity!

Collapse
 
oxharris profile image
Oxford Harrison • • Edited

Hi Godswill, I'm excited to hear that, and to have you try things out with the polyfill!

Collapse
 
efpage profile image
Eckehard • • Edited

We have seen many new approaches over the last few years, that are intendeded to make life easier. Many solve one problem by creating two new ones. And - any new approach brings a new layer of complexity to the game - which is not desireable at all.

Let me give an example from Svelte. I really like the Svelte-approach, but - for me - it has a conceptual weakness. Here is an example, that displays an array

<script>
    const colors = ['red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet'];
</script>

{#each colors as color}
    <div> {color} </div>
{/each}
Enter fullscreen mode Exit fullscreen mode

With their approach, they had to implement new control elements like #each, #if, #else, that introduce their own syntax and new rules. But we already have JS working on this site, so why can´t we use this?

In my oppinion, it would be far better if they could use JS directly like this:

<script>
      for (i of [0, 25, 50, 75, 100]) {
         <div> {i} </div>
      }
</script>
Enter fullscreen mode Exit fullscreen mode

There are good reasons why they did it differently, but here we get a bunch of new concepts, that need to be explained and which are a potential source of confusion and errors. Using the conventional syntax of Javascript would have removed this whole layer (possibly bringing other problems).

I still have no clear opinion about the benefit of Reflex Functions, but we always should ask:

  • does the new concept make our code shorter or easier to understand, or does it reduce the number of elements or tools used?
  • does it help, to use our existing tools more efficiently?
  • is it generally applicable in a wide range of tasks and application? If you think, your approach meets this criteria, I would agree, that this could be a huge step forward.
Collapse
 
oxharris profile image
Oxford Harrison • • Edited

I couldn't agree more that along with what seems like a solution often comes some more complexity! Template languages and custom DSLs have never seemed like a good-enough answer to me! They just double our syntax space!

Wow. I think you just led us to one thing that Reflex Functions addresses! (How come that wasn't immediately obvious?)

This is what you're looking for, which is possible with Reflex Functions today:

<script>
  const colors = ['red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet'];
  for (let color of colors) {
    // valid JS code
  }
</script>
Enter fullscreen mode Exit fullscreen mode

It doesn't look like there's Reflex Functions present at all. That's right; this is plain JavaScript that would work as is! Reflex Functions comes in when you need "reactivity" on top of this plain logic, in which case, you could imagine flipping a "reactivity" feature for your script in something like a Boolean attribute (which I'll talk about next):

<script reflex>
</script>
Enter fullscreen mode Exit fullscreen mode

That would signify that the contents of the script should be evaluated within a Reflex Function! So under the hood, that would translate to:

function** eval() {
  const colors = ['red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet'];
  for (let color of colors) {
    // valid JS code
  }
}
let [ , reflect ] = eval();
Enter fullscreen mode Exit fullscreen mode

from which point, you have the reflect() function for reflecting updates!

Point is, Reflex Functions can be your compile target - in this case, for your <script> elements; "reflex" scripts!

In fact, something like "reflex" scripts might be by far the most common way people will experience Reflex Functions! (Or may be not. But point is, this is a primitive that can be under the hood of anything.)

Now, lest you're imagining having Svelte work this way today, "reflex" scripts are here already (and hopes to be a companion proposal to Reflex Functions) at OOHTML (This might be really worth your time!)

With "reflex" scripts, you don't need a compile step at all as the case may be in Svelte today; everything hits the browser running! And here's a good way to see that:

a list element that receives an array to render from a running application, such that when the list is updated by the application, the loop runs again to re-render:

<body>

  <div id="list">

    <script reflex>
      for (let color of window.color) {
        // valid JS code
      }
    </script>

  </div>

</body>
Enter fullscreen mode Exit fullscreen mode 
// If application were to update the array, the loop re-runs
window.colors = [ ... ]; // Or via the Observer API: Observer.set(window, 'colors', [ ... ])
Enter fullscreen mode Exit fullscreen mode

Or if you were to manually reflect your update:

// Your get a reference to your script
let [ reflexScript ] = document.querySelector('#list').scripts; // Or: document.querySelector('#list script[reflex]')
window.colors = [ ... ];
reflexScript.reflect(['window', 'colors']);

But all of that is taken care of in the OOHTML implementation

You may want to see how reactivity works with Loops, and other control flow constructs:

Thread Thread
 
efpage profile image
Eckehard •

Hy Oxford,

maybe I missed a point, but I have great problems understanding the concept of Reflex Functions.

As far as I understood, they allow to rerun parts of the code, depending on the change of external data, right? This gives me a lot of questions:

  • What happens, if some global data changed, that are not part of the Reflex-System. How do you care, that the result of a function is still correct?
  • Pure functions should not have any external dependencies. How can they benefit from Reflex-functions?
  • If you measure the time for a page update, code execution is only a very small fraction. Even unnecessary DOM updates do not seem to take any time, as long as the visual representation of the page does not change. The most time is consumed to rebuild the page view itself. Are there any examples where the Reflex-system really saved some time?

Any hint´s are welcome

Thread Thread
 
oxharris profile image
Oxford Harrison • • Edited

Hi Eckehard,

Would it help to link you to the "Usecases" section of the README?

Could at least give us some context. If you pick one of the usecase examples, I could clarify anything about it!

Does that help? Or have a specific case in mind?

Meanwhile, to answer ques #1 off the top of my head...

If you have a global dependency in your Reflex Function:

function** render() {
  console.log(globalThis.property);
}
let [ , reflect ] = render();
Enter fullscreen mode Exit fullscreen mode

or in a script element that compiles to the same 
<script reflex>
  console.log(globalThis.property);
</script>

then the idea is that you should observe that property for change and get the function to reflect it:

// Observe and reflect
Observer.observe(globalThis, 'property', change => {
    reflect(['globalThis', 'property']);
});

// The change
globalThis.property = 'New value'; // Or using the poyfill: Observer.set(globalThis, 'property', 'New value');
Enter fullscreen mode Exit fullscreen mode

and in the script example 
// Observe and reflect
Observer.observe(globalThis, 'property', change => {
    scriptElement.reflect(['globalThis', 'property']);
});

So, in other words, the observer API makes it automatic!

And from your question, if a global change happens and isn't part of the "Reflex System
" (by which I think you mean: a global thing that isn't referenced at all in a Reflex Function), then it rightly has no effect within Reflex Functions! Changes that aren't a dependency don't need to be observed and reflected, and even if reflect() is called - reflect(['globalThis', 'property2']), it won't have an effect!

As a tip, if you were to know the exact external dependencies that needs to be observed, the ReflexFunction.inspect() method shows you just that!

Meanwhile, the fact that Reflex Functions don't concern themselves with change detection on the outside world is by design! It allows them to fit with different ways things change on the UI; e.g. via events, etc. They just want to do one thing well: accept a textual representation of what has changed on the outside scope and scan their own scope top-down to re-run dependent statements!

Reactivity with Reflex Functions is all based on static source text analysis, and that's fundamentally different from the "callbacks-and-closures network" nature of the "functional" approach to reactivity! This text-synthesis approach is cheaper - because it happens as part of the source code parsing process within the engine - and saves much runtime overheads on your code. It is also the secret to its "literal-syntax and linear-flow" advantage as discussed in the main post!

To answer #2: Reflex Functions don't rely on being pure! It's normally a mutable world in programming and Reflex Functions are designed to embrace that! But if you'd rather follow an immutable principle, they work as perfectly too!

Thread Thread
 
efpage profile image
Eckehard • • Edited

It allows them to fit with different ways things change on the UI; e.g. via events, etc.

Assume, there is a global variable, that is incremented on every render cycle. Assume, this has an effect "upstream" in your code, e.g. a part of your code is only executed, until the counter is below 10. If you only rerun the downstream part of your code, how should the change be detected?

I do not really understand, what you mean by "up" and "down" the scope? As long, as an application runs, we can say: "After" execution is "before" execution. If the execution of code has and effect on a global variable, this changes the "state" of the whole application. So, it is unsure, if this has an effect on the result.

Data drive vs event driven design

Today, application design if often driven by data, this makes change detection tricky. You only see, that data has changed, but you do not know why this change happend.

If you follow an event driven approach, there is always a "reason", why data changed. If a user hits a button, or if he/she inputs data, this is an event. Even a change in the database can cause an event. This makes it very easy to detect changes, as you only need to check a small amount of data, that are in the scope of the event.

Collapse
 
ninjin profile image
Jin •

Svelte has a much more serious problem - any exception breaks the entire reactivity system completely. This is the fundamental difference between tasks and invariants: the task can and should fall as early as possible, but the invariant should continue working despite the errors in the middle. Semantically, functions in JS are tasks. An invariant can call tasks within itself, but it is not a task itself.

Collapse
 
ninjin profile image
Jin •

Unfortunately, what you are proposing is not a revolution, but a dead-end branch of development. Here are just a few problems that I see right off the bat:

  • Now JS already has 4 colors of functions, which is already a lot, but your offer will inevitably double their number. This creates a lot of problems when trying to write universal code.
  • You rely on the compiler, but since JS is not focused on reactive programming at all, such a compiler will produce undesirable behavior: various forms of glitches and unexpected side effects.
  • In fact, there is no problem with observing the changes of our objects. A much more serious task of reactive runtime is ensuring timely recalculation of invariants in the optimal order. In your solution, I see unnecessary recalculations and the observability of an irrelevant state.

I told more about all this here: dev.to/ninjin/main-aspects-of-reac...
It's very sad that you didn't mention this material, but you didn't even hear about it.

Regarding your statement that there is no way for JS to fully use reactive programming without a significant change in semantics - this is not true. Familiarize yourself with the concept of channels and reactive memoization, which are the basis of object-based reactive programming. This is a much more flexible approach in terms of (de)composition.

Read more about it in this article: dev.to/ninjin/designing-the-ideal-...

As a spoiler, look at this, for 9 years now, the code on the $moll framework has been working optimally without glitches without any reactive compilers:

class App {

    @mem left( next = 5 ) { return next }
    @mem right( next = 5 ) { return next }

    @mem index() { return this.left() + this.right() }

    @mem log() {
        const data = fetchJSON( `/data/${ this.index() }` )
        console.log( data.result )
    }

    @act inc() {
        this.left( this.left() + 1 )
        this.right( this.right() + 1 )
    }

    @act dec() {
        this.left( this.left() - 1 )
        this.right( this.right() - 1 )
    }

}
Enter fullscreen mode Exit fullscreen mode
Collapse
 
oxharris profile image
Oxford Harrison • • Edited

Hi Jin, thanks ofcourse for your response.

I could attempt addressing some things you raised, but I still feel I'm unable to make sense of everything! Maybe some context and specificity would help. (Especially if what you're getting at is a counter proposal, which is itself welcomed.)

If it's what I think I understand,

I think point #1 might be misplaced. I don't think anyone evaluates proposals this way unless what is being proposed is something that is already significantly addressed by existing features.

Point #2 exposes a significant gap between what you might have thought this was and what it really is! We don't rely on compilers, and in fact the whole point is to no more rely on compilers for reactivity!

I'd need more context for point #3. Also the links and the sample code aren't particularly helpful! (For example, is that decorator syntax in your code? And if so, have you found a way to use decorators without a compiler? Also, what is your code doing?)

Collapse
 
godswillumukoro profile image
Godswill Umukoro •

It appears that Jin consistently steers conversations towards promoting his $mol framework. However, I remain unconvinced by his approach.

Collapse
 
efpage profile image
Eckehard •

Please check out this post on medium: Why State Management is All Wrong. Reactivity often is bound tightly to the View-layer of an app. This is ok, if Javascript is just used to create some "reactivity" in the UI. But modern applications often do much more than just creating some nice visual effects.

In traditional programming, the View-layer was a realtively thin layer ontop of a massive application logic. State changes had to be reflected in the UI very fast, so there have been concepts to achieve this efficiently. Most languages featured an event driven approach and a class based system wide communication.

The Web is not that different. There are more bottlenecks, but we should be more specific about that. What precisely causes performance issues? Waiting to fetch some data from a server may be slow, but this is - in the first line - not depending on the framework or technology you use. It is more a result of a bad system design and an overwhelming size of the boilerplate.

On the other hand: using reactivity in an app, that runs completely in the browser, will hardly be very slow, if the UI is not forced to redraw the same content again and again.

It often seems to me, that some of the tools created to make the web faster and our life easier achieve just the opposite...

Collapse
 
oxharris profile image
Oxford Harrison • • Edited

Thanks for the last sentence; and I can tell you that much of our underlying equations on the web the past decade will eventually be revisited! It's just a matter of time!

And just as Brad Lemley puts it there, State Management is Wrong! Thanks for that link! I might not be fully convinced how the proffered solution answers the question, but I couldn't agree more that:

"'state management' solutions are so convoluted and weird: they focus on managing state instead of implementing functionality. The 'state management' approach is backwards and fundamentally flawed: functionality produces state — not the other way around."

I wonder why we've treated State as a separate component in our frontend architectures, when that should really be a bi-product of functionality! I look at any piece of UI code today and everything is talking about "data flow" instead of "control flow" (the latter being the way we naturally write programs).

That's why you enter Svelte scripts and it feels different because it's more about the control flow this time and less about the data flow! Then you enter Reflex Functions and its full blown "control flow" on top of which reactivity is based: Imperative Reactive Programming! ("Imperative" being just as you would write your code if there wasn't anything called reactivity!)

People will gain clarity on how a state-first thinking makes life very complicated, but not so fast, I'm sure!

Collapse
 
efpage profile image
Eckehard •

State first thinking is a natural result of stateless programming. Instead of managing state changes, you just rebuild the whole view tree and let your VDOM handle the state changes. This is a completely view centric approach, that dominates the whole application.

Brad Lemley just asks the right questions.

Reactive programming is an approach to make the life of a web designer easier. But I´m pretty sure, it is not viable as a general programming concept for large scale applications.

Managing complexity was the holy grail of computer science for a long time. Putting all variables in a central value store was surely never a solution for that problem.