Thanks for sharing Greg! I've been waiting for Rust to reach 1.0 so that I can dive-in and work on side projects. Text editors to me (a web programmer) look too intimidating to build from scratch. Can you point to any resources that you used (if any) to understand how text-editors work below-the-hood?
Would totally love to build a side-project like this! Thanks!
> Text editors to me (a web programmer) look too intimidating to build from scratch.
Honestly, this was my thinking too before I started working on one! I, like you, work primarily on web applications and I had never really worked on "lower level" projects. I have always wanted to build a text-editor, though I thought it was much too big a task and didn't know where to even start.
Because of my background in web programming, I was most comfortable with Python and JavaScript. There were a couple of popular editors out there built with JS, (mainly brackets [0]) so I decided to try read through their source code. This helped a lot. Reading source code that I understood, or atleast could make a guess at understanding. I did some research into how brackets was built and found this sort-of useful architecture video [1].
Then, I heard about the Go language, and by accident came across a project called Vigo[0] which is a Vi clone written in Go. The codebase was pretty small, and I was able to easily implement a command for the editor and contribute it. From there I just spent as much time as I could in reading source code of different editors, and comparing them to eachother. Asking myself questions like "How does Vim handle buffers compared to Emacs?" - I have no idea, so I'd go look it up. It could take weeks to get an answer, and sometimes I couldn't figure it out. But I defintely learned Something in doing the research!
I also did some reading from the authors of popular editors like Stallman (Emacs)[3] and Pike (Acme)[4]
When I started building one of my own, I still didn't really (And still dont!) know what I was doing. If you look at the early commits of the project you'll see me totally rewriting pieces over and over until as I figured out better ways of doing it.
I wish I could point you to a single resource that helped me, but it was really a bunch of things that I did over time that helped me. I still don't know much about editors, and I'm sure there's people who can do a much better job than me, but I'm having fun doing it! And I think thats the most important part. Have fun in researching editors (or whatever you decide to build). If you're interested in it, it makes even the most boring and difficult ideas fun to work with. Feel free to reach out to me too if you need help, I'm happy to share what I've learned so far!
sidenote: I didn't know Rust before I started writing Iota, which made things even more interesting ;)
The definitive book on implementing an EMACS, as of a long time ago, is Craig A. Finseth's The Craft of Text Editing -or- Emacs for the Modern Worldhttp://www.finseth.com/craft/
Although I can't say I used it for my work on various versions of EMACS in the '80s, I just dived into code. But everything I read in it (or its first version, I think his bachelors thesis polished into a technical memo or the like) sounded right.
As for what to do now ... well, buffer implementation tradeoffs are different, maybe ones other than buffer gap should get another look.
I've been thinking about writing a few articles on my experiences so far in writing the editor. Sorta covering what I wrote above. I could definitely work in some stuff on architecture and design choices. When I get around to it, I'll post it on my blog, and most likely submit it here too. Hopefully you'll find something useful in it!
Back when I was learning to program with Java 1.0 (using the AWT, no Swing) I made a text editor that supported syntax highlighting. My first big mistake was to open my text editor's source code with my fledgling text editor and attempt to make a minor tweak, only to discover that saving had a bug and wiped out my entire program.
No I did not have any source control at the time. I just started over.
For a cool introduction to the internals of text editors I always recommend finding the old docs for the LCC-Win32 compiler. That compiler comes with it's own IDE, and the author (Jacob Navia) did a great job of candidly documenting his mental process while developing the editor. I know the documents are kind of hard to find, so feel free to get it from my Google Drive: https://drive.google.com/file/d/0BxF_x9mBC-n1Q0VnR3BpN21iWmc...
This document is great, it explains the internals of not just the editor but some of the compiler/linker as well. It's a shame that it was removed from newer versions of the documentation package.
> AFAIK, there are no more planned syntactic changes before
> 1.0.
I wouldn't count on this. :) The bar for breaking changes will continue to rise as we approach 1.0, but until that stable release is cut nothing is safe. Even the upcoming 1.0.0-alpha release in January will contain no concrete guarantee of syntax stability.
I actually contributed quite a bit to a fork of godit called vigo [0]. It gave me lots of insight into how text editors work.
When I went to start writing Iota, I found that there was no termbox-type library in Rust, so I created a wrapper [1]. My goal is to create a pure-Rust version, similar to termbox-go. It's a long way from that, but it will get there!
I initially found it much harder to work with Rust, as the language is more strict in a number of ways. But once I began to understand how it works, and began to grasp some of its concepts such as ownership and lifetimes I found it much more powerful than Go.
Go has a larger community right now, many more libraries and resources openly available. That's probably one of Rust's downsides right now however its catching up, especially with the launch of crates.io. I don't think it will be long in maturing following the 1.0 release.
Saying that, both are enjoyable to work with. But for me Rust wins out!
Termbox can be implemented on non-POSIX platforms, whereas terminfo/termcap do not. See https://github.com/nsf/termbox-go which runs on Windows.
NB: The C implementation doesn't support Windows, AFAIK, which means this Rust library doesn't either, at least as long as it is based on bindings and not a pure Rust implementation of termbox.
Great work, thank you. I was awaiting for something like this for a while. Thanks for sharing.
As a side note:
In the long term what I would really love to see - is Rust GUI for Neovim[1] (embedded library / service). Thus I would be able to both hack around my editor and get VIM's maturity and experience most of the times.
Nothing besides convenience. As a rule of thumb I would love to have editor I find suiting my needs and ideally it would be easy to me to contribute/adjust it to my needs.
Regarding [Neo]Vim - I don't think core is something I would touch a lot. Most likely UI/GUI stuff and plugins (vimscripts etc ..). Due to the nature of vimscripts I would prefer to hack around it in something like Python, Lua or Rust.
- I find C++ codebase too hard to follow and understand. I like C, but it lacks syntax sugar etc...
- I find Rust promising (hopefully it's something I will be able to read relatively easy and hopefully it would be easier to reason about,especially in regard to memory management).
"Iota was born out of my frustrations with existing text editors. Over the years I've tried combinations of simple text editors, IDEs and everything in between. None of them felt right to me, however. Some were too slow & bulky, others were too difficult to customise and still others were platform specific and I couldn't use them on all my machines."
Thank you! In my experience it hasn't been very unstable. Every now and again something will change that will break things, but that's become more and more infrequent. Overall, rust is great!
Its true that Rustbox uses C code right now. Its just a wrapper around the original termbox library. My goal for rustbox is to be a full Rust port of termbox, removing the reliance on C. Hadn't heard of busybox, will check it out. Thanks!
Would totally love to build a side-project like this! Thanks!