One of our rules in software development is the separation of concerns. This rule applies more widely in many areas of life. I am more creative when I can focus on one aspect of the project at a time, and then move on to another aspect of the project. So is with writing. I want the ability to focus on the content, embrace the simplicity of markdown, and have the blog software sort out the rest.
There is something beautiful when one can focus on the content alone, without having to worry about how it looks, because formatting is taken care of afterwards.
Why Build a New Blog from Parts, If Not from Scratch?
Legitimate question. There are already a lot of blog sites and a lot of blog software, many have solid features, and some even had almost the exact features I wanted. So why do this? In July of 2023 when I decided that I would take a sabbatical, I wanted to work on a bit of code again to make sure that I am not one of those VPs who doesn't know how to read code anymore, never mind write it. Technical people like to be managed by technical managers.
Don't take me wrong. As I'm writing this I still believe that as VP of Engineering at work one should not write production code once the organization has more than 5 developers. One can still write code, but only for proof of concept and if one's time permits, since a VP has their own job to do and not their engineers' job.
Back to the why: given that I had a chance to work on something of my own choosing, I decided that I will work on a project that I wanted to do for a while now, and one that I can complete in a few months at most. My team at work has been using static site generators for our documentation for years, and I always thought that my website is a perfect candidate to be converted to a statically generated site.
But probably the main reason for picking this project was that going forward I want to write more, and I want to write markdown in a comfortable editor, like iA Writer.
Blog Software Requirements
Once I figured out why I want to do this project, I decided to write down the requirements for the project. It ended up being only a handful of Post-It notes. Here are the main points:
- Author the article in markdown, not MDX and no complex templates.
- Keep the quote styling from my previous blog.
- Follow the system's appearance: light and dark.
- Responsive, should look good even on an iPhone SE.
- Code syntax highlighting.
- Text to diagram renderer, like Mermaid.
- Support for LaTeX or similar.
- Emoji support, like GitHub and Slack.
- Have the ability to include a box looking like a Post-It note to call out some content.
- Page bundles to keep the content together that belongs to the article.
- Site search without remote calls using an index generated at build time.
- Build it from components, like React components.
- Styling with TailwindCSS.
- Pre-render, generate the whole site at build time.
A Brief Overview of Capabilities
The goal to keep the article in markdown was my most important one and it has been achieved, though in order to support the capabilities that I wanted I needed to introduce a few new conventions into markdown: it became a new markdown flavor. In putting this site together I greatly benefited from many projects. I will be touching on those in a later section.
🥁 Emoji short codes are supported, like the ones that GitHub, Slack, and Discord uses. See Emojipedia for details. In the text include the short code
:phone: and it renders ☎️.
Insert a quote in markdown using the block quote syntax. Mark the author's name with double block quote.
After processing, the above markdown will render like this:
Writing is nature’s way of letting you know how sloppy your thinking is.
Richard Guindon, cartoonist
The note is another repurposed blockquote. Starting the blockquote with the word
[Note] changes the rendering.
📝 Blockquote Rendered as Post-It
Start a blockquote with [Note] followed by a title and renders as a post-it note.
Let's write ”Hello, World!” to show this works.
To add line numbers to the code block, include a meta parameter after the language identifier.
Math Formula Rendering
The rendering is done with KaTeX
This is inline.
Formula in a separate paragraph:
is rendered as:
Diagrams with Mermaid
"A picture is worth a thousand words." To add diagrams to the text, I decided to use Mermaid. Writing a code block with language set
mermaid like this:
will be rendered as:
Hugo & Hextra
The main reason for deciding against Hugo was that the article's content text started to not resemble markdown anymore. We must exempt the front matter from this, since all of the static site generators expect some front matter, but the rest of the content had too much template stuff in it.
Next.js & Nextra
It looked very promising. Unfortunately, as of November 2023 Nextra was still using the Pages Router, and as a result, I would've had to become familiar with technology that is already obsolete. That and a few other issues having to do with styling, adding additional features, and I concluded that I am better off going without Nextra, though I decided to stay with Next.js, the App Router, TypeScript, and TailwindCSS.
Nextra wasn't the only template I considered as a starting point to work with Next.js. Similarly, with Hugo, Hextra wasn't the only one considered. Unfortunately, the others I have looked at had much fewer features that I wanted, so I had to look into extending them, and that's when I usually would run into issues.
Then I ran into issues using Tailwind with 11ty. After a few hours of attempting to make it work, I moved on to see how I can get Mermaid to work with 11ty. It turns out that there is no server-side rendering, build-time diagram generation option, so that was out. I didn't want the site to have to make a remote call to render the diagram, I wanted it pre-rendered during the build, so I moved on to the next technology.
Gatsby.js claims to be the best React-based framework, and I have no reason to doubt it. For my use case though, I had to pass, because I couldn't get some things to work with Gatsby either, so after a couple days of investigation I took a pass.
After about a month of investigation I made the decision and recorded it in my dev journal: "build a static site generator and base it on unified.js."
Having created a lot of projects over the course of about a month or so I concluded that the default Next App options are pretty good:
- Use the
- App router
To these I added the following:
unified.jsfamily of parsers and plugins
- prettier for code and TailwindCSS
At some point during my investigation I came across the unified.js project, or more like a family of projects consisting of parsers and plugins. After looking into many different technology choices these stood out. So I based my SSG project on unified. Major thanks to every contributor to unified and its ecosystem of parsers and plugins. Their work is pretty good!
The project uses the following packages:
The above are parsers or plugins for the processing pipeline and each is acting on a type node in the tree.
Unfortunately, I ran into trouble with the
rehype-mermaid plugin. When I included it in the project, I got error messages coming from deep in the call stack. Even after several hours of investigation, reading and debugging into source code I could not figure out what it was. So I implemented a workaround. I wrote two plugins:
rehype-mermaid2 to split up markdown processing for Mermaid code blocks. Then I added a separate project just to convert Mermaid code blocks into SVG. Decidedly not my preferred solution, but it works, and now it renders Mermaid diagrams during the build, and thus the site is generated in its entirety at build time.
To support page bundles I wrote a plugin to move files into the public folder at the appropriate location for runtime. And I also wrote a plugin to implement the [Note] feature. None of them are robust enough for others to use yet. I intend to make them a bit better and share them in the future.
One of the ironies of this project, in light of an earlier article of mine, is that I ran into trouble setting up Jest and StoryBook, so automated testing using frameworks fell by the wayside. These libraries had issues with the version of the packages I have been using and I didn't want to go back to previous versions of the dev stack.
While I had a good understanding of how a static site generator should work, I never worked on one before, only my teams did. There is a "distance" between a solid theoretical, architectural, and design understanding of a problem and the concrete details required to implement something. Getting to those details is not easy for someone coming into it cold. Once I had a working environment setup, and I was able to see my code run and debug into it, then I started making fast progress, but getting to that point took longer than it should've.
This leads me to my final remark: we as a profession have a serious "startup" problem: it is very difficult for a novice, or novice to a given area, to get started being productive and writing the first line of code. Sometimes even writing a "Hello, World!" is too difficult for a novice. The guidance that's available on the web assumes certain a level of knowledge that a novice rarely has. And universities don't teach that either. We need to do a better job helping someone get started, because that someone is us more often than we care to admit it.