Sam Sartor
Git(Hub|Lab)
# Hornpipe #1 - Yet Another Reason GUIs are Hard 2023-1-10
Hey! This page is a draft. Don't expect much from it.

Do we need yet another blog post about how hard it is to build user-friendly applications in Rust? Raph Levien has already covered windowing, rendering, text shaping, layout, accessability, translation, GPU acceleration, widget trees, virtual DOMs, and about 12 other hard problems I have no idea how to solve. But there is something els bothering me, something not so often talked about: Ctrl+Z.

Around 4 years ago while working on SketchUp Web (an interesting mix of legacy C++ and shiny VueJS) I got the idea to try making a JavaScript-style reactivity system in Rust. I wasn’t so concerned with rendering or layout, since there were plenty of people working on that. All I wanted was a way to mutate random stuff in closures and then have everything else update automatically. I wrote and published a simple little crate. But then I had a dangerous idea.

Since I am tracking those changes anyway, maybe I should put them into some kind of undo/redo stack? Then I’m not forced to go mangle a bunch of mutable pointers when someone presses Ctrl+Z, legacy C++ style.

That turned out to be hard.

Anyway, in the years since I’ve managed to prototype and then throw away all kinds of concurrent datastructures, multi-version concurrency systems, channels, whole new type systems, a pretty nasty DSL, even a busted-up tracing garbage collector. Until this point it has all been for naught, but prototype #7 (named after a fun type of dance) has shown enough promise that I am willing to talk about it.

I don’t yet know if Hornpipe will get any real use. If I’ll have time to fix all the bugs or find contributors to help keep it maintained. But I think the ideas are pretty interesting! So without further ado,

The Spectrum

The way I see it, all software applications fall on a certain spectrum, from those that are very stateful to those that are completely stateless. Compilers and command-line utilities are traditionally stateless. So are most school assignments! Statelessness is usually described as a property of code, often code that is side-effect-free or purely functional. But to me it has more to do with the task the code is trying to perform.

A stateless task is one that allows all input to be received before any output is produced, while a stateful task requires output to be produced while many inputs are still unknown.

In the simplest cases, stateless Rust applications have this sort of structure:

fn main() {
    let input = read(...);
    let output = work(&input);
    write(..., output);
}

The program receives a complete input, loans it to some functions that produce output, and then finishes the task. Rust is very good at this! Although the input needs to be shared throughout the application, it can be made immutable. Output often needs gradual assembly via mutation, but it can also be owned by the functions doing the assembly.

On the other hand, stateful applications can be more difficult. Graphical user interfaces are the classic example, and the sole focus of this blog post. But keep in mind other examples like industrial control systems, operating systems, embedded devices, and video games.

Immediate-mode

The simplest way to write a stateful GUI is in the immediate mode. Example frameworks are imgui, elm, and egui. Any such project has a fundamental sort of structure:

fn main() {
    let mut state = empty();
    loop {
        let event = read(...);
        state = render(&event, state);
    }
}

This application has to produce output (to the screen) before all the inputs are available (from the mouse, keyboard, or chair). That is what forces it to maintain and update state.

To give some more concrete examples, here is a basic application someone might actually want to use. It consists of a simple list of counters. The user should be able to increment counters, decrement counters, add new counters, and to remove existing counters (optional).

The obvious state in this application is the “model”, which can be easily written out like so:

[[counter]]
value = 42
color = "#6fa8dc"

[[counter]]
value = 12
color = "#f6b26b"

[[counter]]
value = 7
color = "#93c47"

But to some extent the state also consists of the GUI elements themselves. Whether or not the application logic cares at all, the “view” state needs to be known to the renderer, window manager, input subsystem, and so on so that pixels can be blatted and events can be dispatched.

Immediate-mode GUIs recreate the entire view state on every frame, in order to maintain the absolute best separation between view state and model state. Then because the view never borrows any part of the model, the application feels almost stateless, like a command-line utility. The event and previous model are a complete input, while the view state and new model are a complete output:

fn render(event: &Event, old_model: Vec<Counter>) -> (Vec<Counter>, Vec<Widget>) {
    // Init the new state.
    let mut new_view = Vec::new();
    let mut old_model = Vec::new();

    for Counter { mut value, color } in old_model {
        // Add the current counter text to the view state.
        new_view.push(Text::new(value));

        // Add a + button to the state and check if would have
        // been pressed by the last click event.
        let add = Button::new("+");
        if add.is_clicked_by(event) {
            value += 1;
        }
        new_view.push(add);

        // Add a - button to the state and check if would have
        // been pressed by the last click event.
        let sub = Button::new("-");
        if sub.is_clicked_by(event) {
            value -= 1;
        }
        new_view.push(sub);
        new_model.push(Counter { value, color });
    }

    // The button to create a new counter.
    let add = Button::new("+");
    if add.is_clicked_by(event) {
        new_model.push(Counter {
            value: 0,
            color: new_random_color(),
        });
    }
    new_view.push(add);

    // Return the updated state.
    (new_model, new_view)
}

This code should feel familiar if you’ve used a purely-functional GUI framework like elm (give me minute before I talk about relm). But it isn’t fundamentally different from mutation-happy frameworks like imgui and egui. They simply bundle the last event and the new view state together into a single mutable object, in order to reduce boilerplate:

fn render(ui: &mut Ui, model: &mut Vec<Counter>) {
    for Counter { value, color } in model {
        ui.add(Text::new(value).with_color(color));
        if ui.add(Button::new("+")).clicked() {
            *value += 1;
        }
        if ui.add(Button::new("-")).clicked() {
            *value -= 1;
        }
    }
    if ui.add(Button::new("+")).clicked() {
        model.push(Counter {
            value: 0,
            color: new_random_color(),
        });
    }
}

I’m not bothering to compile, run, lint, or even think particularly hard about these examples. You’ll have to forgive any obvious bugs.

Retained-mode

There are 3 weaknesses of purely immediate-mode GUIs which I can easily identify:

  1. Poor Performance - recreating the whole view state every iteration can require a lot of computation, even if the actual changes are very small. Increment 1 counter out of 1000 and you still need to do font shaping for all of them.
  2. Lack of Statefulness - sometimes view state is unavoidable. Input elements should receive or loose focus, maintain text selections, remember if they are expanded or collapsed. Where do you put that information, especially if you are authoring custom widgets?
  3. Ad-hoc Structure - the programmer is responsible for maintaining all the relationships between objects in the model state and corresponding elements in the view state. When a “+” button is clicked, how do we really know which counter to increment?

In search of solutions to these problems, consider the far opposite corner of the design space: retained mode. Classic examples are GTK, QT, and the HTML DOM. For each element of a retained-mode GUI, the framework “retains” an object across frames. But as a consequence the programmer has to track down and mutate the properties of each GUI element they want to change, every time they want to change it. Put simply, this sucks. In old-school JQuery it looks vaguely bearable for very very simple GUIs:

<script>
    counts = [0, 0, 0];

    function add(index, offset) {
        return () => {
            counts[index] += offset;
            $(`#value${index}`).html(counts[index]);
        };
    }

    function newCounter {
        const i = counts.length;
        counts[i] = 0;
        $('#counters').append(`
            <div id="value${i}">0</div>
            <button onclick="add(${i}, 1)">+</button>
            <button onclick="add(${i}, -1)">-</button>
        `);
    }
</script>

<div id="counters">
    <div id="value1">0</div>
    <button onclick="add(1, 1)">+</button>
    <button onclick="add(1, -1)">-</button>
    <div id="value2">0</div>
    <button onclick="add(2, 1)">+</button>
    <button onclick="add(2, -1)">-</button>
    <div id="value3">0</div>
    <button onclick="add(3, 1)">+</button>
    <button onclick="add(3, -1)">-</button>
</div>
<button onclick="newCounter">+</button>

But Rust is fundamentally opposed to any code that needs to arbitrarily mutate objects owned by some other module. I’m not even going to show you what that looks like. It looks bad.

Virtual DOMs are the most popular solution in existance today (unless you have the misfortune to write C++ or Java for a living). A VDOM is a sort of hybrid of retained mode and immediate mode. We break the user interface into many stateful retained-mode widgets, but each acts like a immediate-mode GUI in miniature! When the state of a particular widget changes, it rerenders in entirety. But then the output is diff-ed to identify child widgets, so that their retained state can automatically drop into place. So many frameworks take this approach: React, Relm4, Dioxus, Iced, Yew, KAS, etc.

But not necessarily VueJS…

You will be mutated. Immutability is futile.