The Quest for the Perfect (Editor|Terminal|IDE)
Copyright 2021 Brian Davis - CC-BY-NC-SA
Some time in 2018 I started designing an editor to replace my usage of Vim. I use Vim for writing code (mainly Python) as well as prose (blog posts, design documents, notes, todo lists, etc). I learned Vim while working for the NOC at PSU. We used it to edit apache config files on solaris hosts. Vim's focus on the keyboard and minimal interface have always appealed to me, although I am well aware of it's quirks. I vividly remember trying to figure out how to close it the first time.
I prefer minimal tools. I use the i3 window manager, urxvt terminal emulator, and Vim for a lot of what I do on computers. Or at least, that's what I would do if I could always have only the software I want.
For work I'm required to use a Windows laptop and while Vim can run on Windows, and I have used it, its hard to ever feel like the terminal is anything more than a second class citizen on Windows.
My setup isn't perfect either. I don't like editing my vimrc or my i3 config or .Xresources. All these config files have custom syntax and no easy way to discover the settings that are possible. Installing Vim plugins feels opaque and magical. Plugin managers that download code from a bunch of github repos is not my idea of a robust system. After 15 years of using Vim I still only use a tiny fraction of it's features and that's not because I couldn't make use of them. Its because they are a pain to learn and remember.
The Quest began as a question: could I design a better system? Could I take the lessons from these very mature, successful tools and apply them with an eye towards improving cross platform compatibility, unified config management, make the interface discoverable, and make extensibility more accessible?
I filled pages of notebooks with all the commands I could think of that a text editor (or a window manager or a terminal emulator) should support. I tried to design shortcut schemes for these commands that wouldn't overlap and be easy to remember. I experimented with command line interfaces, command menus that changed based on context, and lists of commands with fuzzy search. I looked at novel TUIs, GUIs, and CLIs whenever I found them. And I built toy programs to test out my ideas.
crabapple
It started with this awesome tutorial for building a simple text editor in C. After following it I felt empowered to tackle problems in a way I hadn't before. But I wasn't thinking just about a text editor. I was imagining a common text based interface for many applications.
scale
scale was an attempt at creating a text editor in python using curses. I lost interest in a strict terminal interface quickly becase of the barriers to portability.
sycamore
SDL seemed like the next logical step. SDL is highly portable and mature. I have experience with pygame which is based on SDL so I had some (probably) applicable experience. As I dug into it I quickly realized that text layout is a Hard Problem(TM) and not one of the problems I had set out to solve. Better to look for something that has already solved it.
oasis
At this point my thinking had coalesced around four core problems I wanted to solve:
- Unified, discoverable, keyboard based interface.
- Common configuration language.
- Portability.
- Easy user extensibility.
And with that clarity I came up with a name: Oasis. As in, this program/system would be my oasis in a desert of sub-optimal tools. I went with Qt this time so that I could focus on building the interface I wanted instead of low level text layout stuff. I used Python and YAML so everything would be familiar to a wide population of programmers. And Qt + Python is portable by design.
I architected the application so that "tools" (like the editor) were widget/controller module pairs. I hoped the combination of a simple, popular language and a clean architecture would make the application easy to extend. Unfortunately I went overboard with the modules and sub-modules and pair modules. It became a burden to remember where code should go.
I spent a lot of energy building a robust modal command system, ala Vim. But once it was done I found I didn't really like using it. Designing the shortcuts to be memorable and unique was really hard and since I kept tweaking them it wasn't easy to learn and remember the interface. I turned out that the modal interface of Vim wasn't really the reason I'd kept using it. It was the inertia of having first learned a keyboard centric system, and not wanting to learn another one. If I was going to build a new keyboard based interface, and I did want to build a new one to address the shortcomings in Vim, it would have to be VERY easy to learn and remember.
I put a lot of effort into trying to keep logic and GUI code separate, thinking I wanted to be able to replace Qt with a different library if necessary. But that introduced a lot of complexity and since I was relying so heavily on Qt anyway I didn't see how I'd ever extract it from the application even after all my effort to keep it isolated.
Oasis had a lot of code dedicated to window management. I basically had a simple implementation of i3 working using QBoxLayouts. It was cool but it had a few bugs, wasn't very flexible and, I decided, not core to what I was trying to do.
I started to think about redesigns.
qute
I wrote a second Vim-like in Qt without bothering to isolate GUI code from logic. It was certainly more compact and easier to reason about but I didn't want to use it any more than I did oasis.
QtEdit
Then I made a stupidly simple text editor that wrapped QTextEdit and added Open and Save commands to the menu. That was it. It was too far on the minimal side to make me want to use it.
ed2
I took one more stab at rewriting oasis into something simpler (in terms of architecture).It was when I added a menu bar that I realized I had a possible solution to the discoverability part of my goals.
ed3
In early 2020 I started a project that I named ed3. Naming was getting tiresome at this point and I had so little confidence that this version would last any longer. I leaned into Qt, trying to get as much utility as I could. I gave it a little bit of architecture, separating widgets into modules but all the modules into one flat source folder.
I drew on my previous work for syntax highlighting and code formatting and built a wrapper for the menu bar that made it easy to add shortcuts. Added search and replace and couple dock windows for displaying a list of open files or getting python help. In a relatively short space of time and code I had a pretty capable little editor. One I actually didn't mind using.
Shortcuts rely on chords but I can also use the keyboard menu navigation which I realized works a lot like Vim's modal commands except you are visually navigating the command space. That's a surprisingly powerful idea in a very mundane package. There are a few places where I don't have hinting or menus to communicate the possible commands and that's a deficiency I'm working to address.
There's a lot of spit and polish I'd like to add. And features like code navigation and VCS integration. But the core design feels like something I can use now and continue to extend. I think that's the goal of software design: getting to a concept that solves the problem at hand while being flexible to adapt to change in the right dimensions and rigid to resist change in wrong the dimensions.
I continued to work on ed3 for about 2 years. Eventually setting up automated builds for linux and windows. I added a terminal with basic clones of posix commands and an fzf inspired file picker. The text editor got an overview navigation pane. I'm proud of the work I did and I continue to use ed3 as a daily driver on window.
But it's never been snappy. Startup time is several seconds on windows.
qed
One of the nice things about using Qt from python is that it's relatively straightforward to prototype your application in python and then port it to C++. I was in the middle of refreshing my C++ knowledge when I decided to try porting ed3. Of course I couldn't just do a straight port. I had things to tweak and my whole ActionDispatcher system from ed3 relied on too much python magic.
Getting started was pretty easy. Qt and C++ have a well beaten path to building an executable with a UI. My first major obstacle was that the fzf clone I wrote for ed3 relied on python to walk the file tree and used and generators. That was all pretty fun but not easily replicable in C++. Besides I knew that QFileSystemModel was a faster, better integrated solution and wanted to go that way.
Next I needed json and templating libraries. I've struggled in the past with integrating C++ libraries into my build so I picked them based entirely on easy of integrating. Fortunately both did what I needed with very little fuss.
With C++ I constantly find myself writing more complex abstractions, getting bogged down, tossing it and starting over with a simpler vision. I think this same process was happening with python, it just would take much longer to get bogged down. In that way I think C++ might be training me to be a better programmer? It's certainly a less forgiving teacher than python. Good grief do I miss tracebacks.
I have somewhat higher hopes for qed than I did for ed3. I would like to go ahead and write an LSP client instead of bodging together a few python specific language tools. Partly because I'm writing a significant amount of C++ and rust now. I will need something besides pygments for syntax highlighting. I've been avoiding tree-sitter but maybe. Finally I have some experimental toy scripting languages I've been playing with. It would be fun to integrate one, possibly alongside lua, python, and tinyjs.
But before I get to the fun stuff I think I need to figure out windows builds and test startup time. It's snappier than python but not as snappy as say vim.
Here's a little gif showing the fuzzy search file opener, syntax highlighting via tree-sitter, and screen splitting all via keyboard hotkeys.
The Quest for the Workshop
The oasis metaphor was a good one for the why of what I'm trying to do. Tools should engender good feelings, that's what keeps us wanting to use them. But I found a better metaphor for the what. I'm building a digital workshop. A workshop is a thoughtfully designed space filled with tools configured to enable and accelerate a craftsperson's work.
The Quest is perhaps a highly personal and never ending one. Whether this work continues to provide value to me personally is still in doubt. Whether it will ever provide value to anyone else is completely unknown. As time goes on I'll update this page with what I learn.