Embracing Simplicity

One of our rules in software development is the separation of concerns. This rule applies more widely in many areas of life. I am more creative when I can focus on one aspect of the project at a time, and then move on to another aspect of the project. So is with writing. I want the ability to focus on the content, embrace the simplicity of markdown, and have the blog software sort out the rest.

There is something beautiful when one can focus on the content alone, without having to worry about how it looks, because formatting is taken care of afterwards.

Why Build a New Blog from Parts, If Not from Scratch?

Legitimate question. There are already a lot of blog sites and a lot of blog software, many have solid features, and some even had almost the exact features I wanted. So why do this? In July of 2023 when I decided that I would take a sabbatical, I wanted to work on a bit of code again to make sure that I am not one of those VPs who doesn't know how to read code anymore, never mind write it. Technical people like to be managed by technical managers.

Don't take me wrong. As I'm writing this I still believe that as VP of Engineering at work one should not write production code once the organization has more than 5 developers. One can still write code, but only for proof of concept and if one's time permits, since a VP has their own job to do and not their engineers' job.

Back to the why: given that I had a chance to work on something of my own choosing, I decided that I will work on a project that I wanted to do for a while now, and one that I can complete in a few months at most. My team at work has been using static site generators for our documentation for years, and I always thought that my website is a perfect candidate to be converted to a statically generated site.

But probably the main reason for picking this project was that going forward I want to write more, and I want to write markdown in a comfortable editor, like iA Writer.

Blog Software Requirements

Once I figured out why I want to do this project, I decided to write down the requirements for the project. It ended up being only a handful of Post-It notes. Here are the main points:

• Author the article in markdown, not MDX and no complex templates.
• Keep the quote styling from my previous blog.
• Follow the system's appearance: light and dark.
• Responsive, should look good even on an iPhone SE.
• Code syntax highlighting.
• Text to diagram renderer, like Mermaid.
• Support for LaTeX or similar.
• Emoji support, like GitHub and Slack.
• Have the ability to include a box looking like a Post-It note to call out some content.
• Page bundles to keep the content together that belongs to the article.
• Site search without remote calls using an index generated at build time.
• Build it from components, like React components.
• Styling with TailwindCSS.
• Pre-render, generate the whole site at build time.

A Brief Overview of Capabilities

The goal to keep the article in markdown was my most important one and it has been achieved, though in order to support the capabilities that I wanted I needed to introduce a few new conventions into markdown: it became a new markdown flavor. In putting this site together I greatly benefited from many projects. I will be touching on those in a later section.

Emojis

🥁 Emoji short codes are supported, like the ones that GitHub, Slack, and Discord uses. See Emojipedia for details. In the text include the short code :phone: and it renders ☎️.

Quotes

Insert a quote in markdown using the block quote syntax. Mark the author's name with double block quote.

> Writing is nature’s way of letting you know how sloppy your thinking is.
>
> > Richard Guindon, cartoonist

After processing, the above markdown will render like this:

Writing is nature’s way of letting you know how sloppy your thinking is.

Richard Guindon, cartoonist

Notes

The note is another repurposed blockquote. Starting the blockquote with the word [Note] changes the rendering.

> [Note] Blockquote Rendered as Post-It
>
> Start a blockquote with [Note] followed by a title and renders as a post-it note.

📝 Blockquote Rendered as Post-It

Start a blockquote with [Note] followed by a title and renders as a post-it note.

Syntax Highlighting

Let's write ”Hello, World!” to show this works.

```python title="hello.py" caption="Fig.4. Hello, World!"
def say_hello():
    print("Hello, World!")
```

Renders as:

def say_hello():
    print("Hello, World!")

To add line numbers to the code block, include a meta parameter after the language identifier.

```python showLineNumbers{42} title="hello.py with Line Numbers" caption="Fig.6. Python Code with Line Numbers"
def say_hello():
    print("Hello, World!")
```

Renders as:

def say_hello():
    print("Hello, World!")

Math Formula Rendering

The rendering is done with KaTeX

This $\sigma(z) = \frac{1}{1 + e^{-z}}$ is inline.

This $\sigma(z) = \frac{1}{1 + e^{-z}}$ is inline.

Formula in a separate paragraph:

$$F(\omega) = \int_{-\infty}^{\infty} f(t) e^{-j\omega t} \, dt$$

is rendered as:

$F(\omega) = \int_{-\infty}^{\infty} f(t) e^{-j\omega t} \, dt$

Diagrams with Mermaid

"A picture is worth a thousand words." To add diagrams to the text, I decided to use Mermaid. Writing a code block with language set mermaid like this:

```mermaid
flowchart LR
  subgraph "Function Composition"
    direction LR
    a(A)--f-->b(B)
    b(B)--g-->c(C)
    a(A)--gof-->c(C)
  end
```

will be rendered as:

Technologies Evaluated

Hugo & Hextra

Hugo is one of the most popular open-source static site generators. Hextra is one of the most popular and capable themes for Hugo.

Hugo has the benefit of Go. Unfortunately, when it comes to adding additional capabilities to Hextra one has to contend with the sad state of Go templating support across the board. As of late 2023 support in VS Code for formatting and syntax highlighting was troublesome. For just about anything, I had to drop into Javascript so often that after a while I decided that I might as well just look for something that's written in Javascript (or TypeScript).

The main reason for deciding against Hugo was that the article's content text started to not resemble markdown anymore. We must exempt the front matter from this, since all of the static site generators expect some front matter, but the rest of the content had too much template stuff in it.

Next.js & Nextra

Next.js is the first full-stack framework recommended by React on their website. Nextra is possibly the flagship template for Next.js. Even the Next.js team use it for their doc site.

It looked very promising. Unfortunately, as of November 2023 Nextra was still using the Pages Router, and as a result, I would've had to become familiar with technology that is already obsolete. That and a few other issues having to do with styling, adding additional features, and I concluded that I am better off going without Nextra, though I decided to stay with Next.js, the App Router, TypeScript, and TailwindCSS.

Nextra wasn't the only template I considered as a starting point to work with Next.js. Similarly, with Hugo, Hextra wasn't the only one considered. Unfortunately, the others I have looked at had much fewer features that I wanted, so I had to look into extending them, and that's when I usually would run into issues.

Eleventy

11ty, a simpler static site generator claims to be the fastest among the node.js/Javascript crop of SSGs. Undoubtedly 11ty works great for several specific use cases, just not for the one I had in mind. By this time in the technology selection I decided that I want to use TailwindCSS for styling, because I started to appreciate the idea that I can create a React/Next.js component and I can fully style it both for light and dark themes.

Then I ran into issues using Tailwind with 11ty. After a few hours of attempting to make it work, I moved on to see how I can get Mermaid to work with 11ty. It turns out that there is no server-side rendering, build-time diagram generation option, so that was out. I didn't want the site to have to make a remote call to render the diagram, I wanted it pre-rendered during the build, so I moved on to the next technology.

Gatsby

Gatsby.js claims to be the best React-based framework, and I have no reason to doubt it. For my use case though, I had to pass, because I couldn't get some things to work with Gatsby either, so after a couple days of investigation I took a pass.

Technology Choices

After about a month of investigation I made the decision and recorded it in my dev journal: "build a static site generator and base it on unified.js."

Having created a lot of projects over the course of about a month or so I concluded that the default Next App options are pretty good:

• TypeScript
• ESLint
• TailwindCSS
• Use the src directory
• App router

To these I added the following:

• unified.js family of parsers and plugins
• prettier for code and TailwindCSS

At some point during my investigation I came across the unified.js project, or more like a family of projects consisting of parsers and plugins. After looking into many different technology choices these stood out. So I based my SSG project on unified. Major thanks to every contributor to unified and its ecosystem of parsers and plugins. Their work is pretty good!

The project uses the following packages:

• unified
• remark-parse
• remark-emoji
• remark-math
• remark-gfm
• remark-rehype
• rehype-katex
• rehype-stringify
• rehype-pretty-code
• rehype-format

The above are parsers or plugins for the processing pipeline and each is acting on a type node in the tree.

Unfortunately, I ran into trouble with the rehype-mermaid plugin. When I included it in the project, I got error messages coming from deep in the call stack. Even after several hours of investigation, reading and debugging into source code I could not figure out what it was. So I implemented a workaround. I wrote two plugins: remark-mermaid2 and rehype-mermaid2 to split up markdown processing for Mermaid code blocks. Then I added a separate project just to convert Mermaid code blocks into SVG. Decidedly not my preferred solution, but it works, and now it renders Mermaid diagrams during the build, and thus the site is generated in its entirety at build time.

To support page bundles I wrote a plugin to move files into the public folder at the appropriate location for runtime. And I also wrote a plugin to implement the [Note] feature. None of them are robust enough for others to use yet. I intend to make them a bit better and share them in the future.

One of the ironies of this project, in light of an earlier article of mine, is that I ran into trouble setting up Jest and StoryBook, so automated testing using frameworks fell by the wayside. These libraries had issues with the version of the packages I have been using and I didn't want to go back to previous versions of the dev stack.

This seems to be a common problem in the Javascript/Node ecosystem: a project moves faster than another and if you want to use the slower evolving project, then you must stay back from the latest versions of the faster project. Unfortunately, when the new features are something that you really want, then you have a choice to make. This probably would’ve not been my choice if this would be production code for a customer, but for my hobby project, this worked.

Conclusions

Working with the Javascript ecosystem, and I consider Node, React, Next.js, and TypeScript to be part of that, is fraught with issues. Version problems pop up out of nowhere, and debugging them is non-trivial. When something doesn't work, there is no indication as to what might be going wrong, never mind the "why" it might be going wrong. Often searching for answers is not much help either, since as these technologies evolve rapidly, there isn't enough knowledge in the dev community about them. And some old answers are unfortunately no longer valid. And due to the newness of these packages even ChatGPT4 was just going in circles, providing no real help.

What is even more difficult than working in one ecosystem is working across two when the lines are not clearly delineated, the tooling is not sufficiently evolved, and knowledge is not readily available. This was the case as I attempted to work with Hugo and Go templating that crossed over to solid Javascript territory.

While I had a good understanding of how a static site generator should work, I never worked on one before, only my teams did. There is a "distance" between a solid theoretical, architectural, and design understanding of a problem and the concrete details required to implement something. Getting to those details is not easy for someone coming into it cold. Once I had a working environment setup, and I was able to see my code run and debug into it, then I started making fast progress, but getting to that point took longer than it should've.

This leads me to my final remark: we as a profession have a serious "startup" problem: it is very difficult for a novice, or novice to a given area, to get started being productive and writing the first line of code. Sometimes even writing a "Hello, World!" is too difficult for a novice. The guidance that's available on the web assumes certain a level of knowledge that a novice rarely has. And universities don't teach that either. We need to do a better job helping someone get started, because that someone is us more often than we care to admit it.