Terminal with GUI Primitives
Listen to this post
Worflow consisting of multiple projects running in parallel
Recently @theo shared an article desribing how we as developers are working on multiple projects in paralel and how the current tools are not built for it. Which further prompted him to start using Cmux.
In this blog I describe how we can take the idea further!
Lets first establish a shared context of understanding between me and you 🫵.
Terminal
Most terminals have single level of hierarchy in the form of tabs. Each tab has two states:
- Shell mode A command-line shell (such as Bash or Nushell) which accepts user input and executing commands.
- Full-screen (raw) mode A program (such as Vim or btop) takes control of the terminal display using raw input/output, managing the entire screen and handling input directly.
The important distinction is that the terminal itself supports these modes but does not come with the program like Bash/Vim builtin. It exposes these primitives and on top of which other program are built.
IDE
As the name suggests, its an “Integrated” Developement Environment. So the important distinction being that it comes with most of the programs builtin. There are plugins which provide some extension, however the core experiences like the code editor, terminal, file explorer, agent pane are all first class citizens.
How Cmux solves the problem ?
Now we come back to Cmux. It sits in between a Terminal and an IDE.
First it provides a double level of hierarchy.
- Workspaces: which cleanly map to projects from Theo’s diagram
- Panes: which are like tabs
In Theo’s diagram, we still have the Browser and IDE. So what Cmux guys add another ability to the tabs. Now a tab an also be a web viewer. And if you are a terminal user, you can get away with most of what an IDE does inside a terminal. Problem mostly solved! Now it has the double hierarchy required for the workflow while covering all the bases.
How can we make it better ?
Based on the shared context we established, Cmux is an ugly child of a Terminal and an IDE. Let me explain.
A terminal is all about exposing primitives and an IDE is all about integrating everything into one. Cmux exposes the primitives of a terminal, while working like an IDE by also making an exception and including the web viewer in the host.
We can do better!
Proposed Design
We can improve this by going back to the roots of a terminal and exposing primitives for a GUI. Finally I decide on two new primitives for a terminal:
- Shared Pixel Buffer
- Embedded View
Shared Pixel Buffer
This primitive is inspired by the wayland compositor. The way wayland works is that there are shared pixel buffer between the compositor and each client. The client simply has to make sure that it render stuff into the pixel buffer. The client can use any method for the task. The compositor manages which pixel buffer and how much of that content is shown.
In similar fashion the compositor receives all the inputs and the decides how they are shared with the client.
We do something similar. Each hosted application can do whatever it what, that only contract we care about is the pixel buffer which would be composited when its associated tab is focused. Similarly the host receives inputs from the OS which we pass onto the focused client.
Embedded View
This is special to MacOS. In previous case, everything passed through the host. In the embedded view, the host still manages which tab if focused and their life cycle. However, it does not manage the rendering or the inputs. Its mostly acts as a mean of handshake between the hosted application and the AppKit.
My intuition to think about the two cases is as follows:
In the embedded view the host is letting the child directly talk to AppKit. Whereas in the shared pixel buffer model, the child is seperate and talks through the host.
For example in the shared pixel buffer model, the host can add shaders before it shows the shared buffer. But in the embedded view that would not be possible.