Nim: Scripting ease in a compiled language
Nim went 1.0 around the beginning of October, 2019. I’ve taken the chance to dig into it for the past three weeks, and while I can’t call myself an Nim expert as of yet, I think I can offer a decently informed opinion.
I suppose I should add a bit of context: I’ve used Go as a side-project language since 2014, mostly for Gills, PISC, and the blog you’re reading right now. It’s definitely served me well, but I’ve been growing rather tired of some of it’s… choices. It’s not been terrible, but I’ve grown to miss generics and exceptions from C#, and Go just tends to be verbose in general.
That being said, there are some things I like about Go: It compiles on both Windows and Linux, it makes executable files that are easy to deploy (at least for my amateur sysadmin sensibilities), and it gets web servers spun up relatively quickly, all things considered.
For a while, I got around Go’s verbosity by using PISC and then Lua as scripting languages for the projects I had going in Go. I see myself using Lua as a scripting layer for Gills for quite some time to come, just due to how nice a search/tag driven CMS is.
But, when Nim announced their 1.0 status, I decided to do a few small projects to see if it was the language I wanted to make my side-project language of choice for the next 5-10 years, the way that Go has been my side-project language of choice (outside of games, where I’ve mostly used Lua) for the past 4 years.
Project Zero: Porting Jumplist
So, a project that I’ve landed on as a decent way to exercise a programming language without requiring the language to have a full web stack (which is a very heavy requirement) is to implement a directory jumplist manager. I first wrote one in bash that was an abomination of sed, awk, and bash that had slightly self-modifying code. The concept was to avoid calling into windows processes in git-bash, since that seemed to bring a non-trivial amount of lag.
When I wanted to bring the same jumping experience to
cmd.exe as a result of using that a lot more at my new job, I opted to write a jumplist manager in Go and some wrapper batch scripts.
Porting this program to Nim was mostly a pleasant experience. I was able to remove about half my code, because Nim had a pre-existing insertion-ordered table, so I didn’t have to write my own insertion-ordered key-value store like I did in Go. The only grumble I had from this project that I had some difficulties figuring out some of the type errors I got at first. But, seeing half of the code I wrote disappear, and the code that I wrote being much smaller was really nice.
Project One: Porting kb_wiki
After I ported the Jumplist script, I had a pretty solid idea of my next target: The various instances of the kilobyte_wiki software. This was software I’d written in Erlang a while back. It was still running in tmux panes because I’d not figured out how to do proper erlang releases while using erlang.mk as my build system. (Side note: I recommend trying out rebar3 over erlang.mk, as with erlang.mk, you have debug a complicated makefile on top of the erlang tools. From what I’ve heard, rebar3 is more user-friendly than erlang.mk, which was difficult for this erlang and make newb to use). I also wanted to stand up an instance for tracking various articles and the best comments I’d found on Lobsters and Hacker News in a given month. In the process of porting kb_wiki from Erlang to Nim, I ended up finding a few things that kinda confused me at first glance, but that make sense in retrospect.
First, the standard library database modules only return database results as
seq[string], roughly a
List<string> if you’re from C#, or an
ArrayList<string> if you’re from Java. This is a deliberate decision to optimize for the likely case where most of the data is going back into other strings in other parts of the application most of the time, and to try to help keep the database code from having to expose different sets of types for different databases, (as I gathered from talking Araq, the creator of Nim on IRC). While it’s not the choice I’d go with for a database application where I was heavily concerned with I/O performance, it’s certainly not a terrible choice for my more standard Sqlite+Web Server type stuff I’ve done a lot of lately.
Project One and a half: Understanding Jester
Also, though it currently (in October of 2019) not look the most kept up to date, Jester is definitely very nice to use as a web framework. Here are a few things that I ran across when porting wiki from Erlang to Nim:
- You’ll want to make sure to read up on the
settings: macro if you want to set what ports and hostnames your web server is binding to. I use it in kbwiki, if you want a small example.
cond expressions in the router that evaluate to
false just won’t match, so you’ll get 404 errors, which can make debugging tricky. I recommend making them if statements if you’re trying to debug why a route isn’t matching, and then use your debugging method of choice to figure out what’s going on.
- For now, Jester only properly supports HTML forms that use an
enctype="multipart/form-data" in their HTTP posts. I don’t think this is a major problem, but it tripped me up when I was trying to forms to work on my website, so I figured I’d mention it here to save the next person making websites some time.
- The Jester README current states that it requires a
devel version of the Nim compiler, but I think at least the version in Nimble works fine with Nim 1.0. I am investigating this, but for now, it’s not something huge to worry about.
- Jester is a bit underdocumented, which means that understanding certain parts will require reading it’s source. I didn’t have much difficulty there, and I’d like to help with the docs for it, as I have time. This paragraph is a first attempt at helping with them.
Cool things learned from two and a half projects
So, I had already learned from my first project in Nim that it was a lot more concise than Go, but it was in this second project where that truly became evident. See, Nim is the language I’ve had the easiest experience creating executables in. If you look at the code for kb_wiki, you’ll notice the main app,
kbwiki.nim, and that it requires
views.nim. Unrelated to the main kbwiki app are
scraper.nim. They are all separate applications that can be compiled into executables using
nim c mdtest,
nim c todo or
nim c scraper, and they can use other nim files.
But, because files are that easy to re-use, and executables are the easy to make, it makes code re-use much easier, at least in the small, than I’ve ever had it be until now. In Go, creating new executables involves a lot more ceremony, in terms of making sure that a new folder structure is created and such. In C#, unless you use LinqPad, you have either MSBuild/VS projects, or you have to create a whole folder and use then
dotnet tool to create projects and explain to the C# compiler how to turn a set of source files into an executable. But, in Nim, you just write your file, install the right things via
nimble, and then you can compile it into an executable.
Project Three: Scripts and utilities at work
This nimbleness of executable creation makes Nim the first complied language I’d use for general purpose command-line tools, where the value of writing 10 lines of code isn’t dominated by the process of setting up a folder structure and project files to get everything set up.
So, I’ve made some small utilities for my day job. An program for spitting out “banners”, aka the “-” with a user-specified color in the background. A little command-line punch-clock for helping me keep track of what time I’m working (when working flex hours, it’s helpful to be able to punch in and out for personal tracking). A tool for copying these little tools from the directory they get developed in to the designated “My command-line tools live here to be on the $PATH” directory.
Project the Fourth: Scraping kb_wiki
Most recently, like I mentioned above, I wrote a scraper that can be pointed at instances of
kbwiki and download their content. The motivation for the project was pretty simple: I had two instances of the Erlang version of kbwiki floating around, and I wanted to get the content hosted on them in Sqlite files.
The Erlang versions of the wiki use
dets tables, because that’s what was easiest to get access to when I was writing the wiki in the first place. So, at first, I looked into trying to connect my Erlang code to SQLite. But, after digging into that for about 90 minutes, I was running into issues around gcc compiler versions on my linux box. Not wanting to spend until the wee hours of the morning trying to fight both Erlang, GCC, and Sqlite, I decided to switch from trying to dump the content into Sqlite files from Erlang, to writing a Nim scraper that basically gloms over the HTML, logs in as an admin, and then downloads all of the markdown for the articles. That took me just under two and a half hours (give or take) to finish, and then I was able to scrap down the wikis in a matter of seconds.
After that, I moved https://idea.junglecoder.com and https://drift.junglecoder.com from running on Erlang to running on Nim. And I didn’t have to fuss with certificates or the like, due to them running behind nginx. I also put up https://feed.junglecoder.com, where I’ve been keeping a feed of what seem to me to be notable comments, articles, and such.