JAMES LONG

RIP Over-Engineered Blog

April 1, 2016

Many of you know that I like building my own dynamic blog engine. If these 3 posts are any indication, I majorly refactor it about every 8 months. At least for the last few years I used the same database layer: a simple redis instance that stores posts and metadata. Before my own engine, I used many other systems like Jekyll but I got fed up with them. (I’ve looked at many other static site generators, but I still get fed up with them.)

Static site generators are nice, but they don’t really reduce complexity. They just shift it to compile-time instead of run-time. And for certain use cases (mine in particular), implementing them at compile-time is doable but unnecessarily complex. If you think of your data as a graph, you need to generate every single view of this graph every single time, requiring several plugins and ad-hoc fixes. With a large number of posts, compiling becomes slow and live preview isn’t really live any more. Worst of all is when your generator breaks because of an Xcode update and now you can’t deploy your post.

I totally get why people like static site generators. For infrequent posting, it’s nice to throw it onto github pages. But running a cheap linode or digital ocean server is super easy, and if you’re willing to do that it’s actually much easier to run a server. I recommend getting some sysops experience anyway.

You know the amount of times my site has legitimately crashed in 3-4 years? Once. And that’s because nginx logs filled up the disk space, so it would have happened anyway if I was using a static site generator.

On the other side of the spectrum, you have services like Medium but I like owning my content.

All of this makes more sense in context. I had a vision for my blog. I wanted to build something where I could easily write rich, interactive programming tutorials. Something like Khan Academy but suited to specific concepts like React. I was never sure what that looked like, but I knew I needed to write my own code. I thought about writing a compiler that converted markdown into React components, and the ability to extend markdown syntax with custom React components for a specific tutorial. But that never happened.

Along the way I wrote several great interactive tutorials like Removing User Interface Complexity, or Why React is Awesome, Taming the Asynchronous Beast with CSP Channels in JavaScript, and Writing Your First Sweet.js Macro. These were huge for me, especially the React one because it had a noticeable impact on React adoption in the early days. It’s been great for my career and led to many opportunities for speaking and meeting people.

Those tutorials were hacked together though, and I always meant to build features into my blog to make writing them easier. But I never did that, and I’m not going to. In fact, each tutorial currently is its own isolated little app, which is actually nice.

That leads me to this past year. I haven’t written any interactive tutorials, and my blog has become a complex beast that I only use to learn new libraries by integrating them. I’ve been completely frank that my blog is over-engineered, and if you look at one of earlier refactorings the technologies listed is like a list of bookmarks you’d find in my pinboard.in: react, js-csp (CSP channels), mori (persistent data structures), sweet.js (macros), etc. I can’t help but laugh out how shamelessly I followed interesting tech.

It’s time to move on, though. These days I just want to push words, and the occasional interaction is just a <script> tag away. My goal for this rewrite is to just work with a list of static markdown files. Unfortunately, this means my blog no longer uses React. :( I love React, but simple static pages are great for things like this. Just focus on the content. (I’m also now using React at work and other side projects, so I’m learning things there instead.)

Does that mean I’m going to use a static site generator? Heck no. If you look at my latest code, you’ll find a simple server that renders a few dynamic pages (written in nunjucks, which is ironic because 4 years ago I wrote nunjucks for my blog engine). Although it’s a dynamic site, the server indexes posts from a local directory. It will scan for markdown files and load them into memory, and re-index them whenever a file changes.

So I get to just write markdown files and rsync them to my server to publish them. I get all the benefits of static site generation without actually going through that hassle.

For live previews, I run a local instance of my blog and it’ll watch the local directory and reload the browser whenever a file changes, just like you would fire up a static site generator, except I don’t have to compile everything so startup is quick. This setup means I can use fantastic programs like iA Writer to manage my posts:

As you can see, I use YAML front matter for metadata like Jekyll. Safe to say, this is quite an improvement over, uh, whatever this is:

In fact, the ability to use programs like iA Writer or Ulysses to write posts was the biggest motivation for me to do this. These programs make writing fun, and my hack-job of an editor made it painful. Watching my friends use these programs made me jealous (Walter tried to convince me to do this years ago).

Visual proof of the simple publishing system: rsync. I also haven’t mentioned that I’m managing these files with git as well.

All this to say, you can have your cake and eat it too. My vision of a tutorial builder didn’t pan out, so I’m finally reaping the rewards of using simple files to blog. However, you don’t need to throw out servers completely; a stupidly simple dynamic server will achieve the same as a complex static site generator. And a basic node server can handle a lot of traffic.

The last missing piece: one of the best features about dynamic blogs is the ability to drag-and-drop images right into a post. I had that in my editor and I will miss it. I know some markdown editors support that, but I need to upload the image to my server so it exists when I publish the post. I fixed this by hacking together a simple service which lets me drag-and-drop images and automatically upload them to my server. The site gives you back a URL and a markdown image tag ready to use. This works really well and completely solves the problem.

Now for the sweet, sweet joy of deleting code. Bye React, Redux, React Router, Universal JavaScript, and everything else! Have you ever wanted real numbers for technical debt? Here you go! Just look at all that red!

(This includes a few things like CodeMirror, so it is exaggerated.)

Moral of the story: use the right tool for the job. Unless you really want to learn a different tool.