Opinions About Compiler Building

2026-08-12

My random opinions about compiler building in the 21st century based on 30+ years of software consulting. I think of language design and compiler-building as two different things. One should actually avoid building compilers in the 21st century - we have enough compilers already. There’s always room for new languages, though. (In fact, I favour languages that use something better than characters for syntax, but, I digress…)

(Caveat: some of the links below point to experimental code, I write more about this stuff in my substack articles, blog, YouTube (all listed below))

prototype the language and play with it first - use OhmJS to whip together a parser - OhmJS is based on PEG technology which makes it easy to write grammars
use Ohm-editor (part of the OhmJS package) - it’s essentially a REPL for grammar writing that shows parse trees (CSTs), it saves a lot of time up front
when satisfied, you can re-code the grammar using whatever other technology you deem is necessary (e.g. Pratt parsing, whatever) - writing the code twice (or more) is more efficient than starting out dealing with implementation issues right off the bat
the easiest part of building a compiler is writing a parser (and scanner if you use something other than PEG), but, it’s been formalized and therefore discussed more often than the hard parts
you can build an MVI (Minimum Viable Implementation) of a language by transpiling code written in the language to some already existing language (e.g. Python, Javascript, etc.) - I favour using a text-to-text transmogrifier (“transpiler”) like the T2T stuff in Parts Based Programming
Steve Philips invented a new language Voltair by transpiling Voltair code into Go code. I think that he used an LLM to build the transpiler.
Alan Kay is on record saying that new languages should use existing languages as “assembler”
type checking is best expressed as a relational program - use something like Prolog or one of its descendants
use triples only, don’t bother with the OO extensions to Prolog - a triple is - relation(subject,object)
you can code up your own exhaustive search inferencer based on Nils Holm’s Prolog in Scheme code or my automated port of it to Javascript, but why bother? Write it first in Prolog (or …), then when satisfied re-code it in something else
type checking is much harder and more involved than parsing, start by expressing the rules in relational form and getting the rules right before calcifying implementation details
when doing code emission, emit “dumb” code first, then clean it up with a peepholer as described in Fraser/Davidson or Cordy’s OCG (GCC uses RTL, IMO, Cordy’s OCG is even better) - go for simplicity first, make code emission easy, then tighten up later after measurements prove which parts need to be tightened up
Holt et al, invented Data Descriptors - a way to normalize all data into the same form, which then makes things like allocation easy to think about
I favour a UNIX-y pipeline approach - use multiple stages, each written in a language that is better suited to each particular problem, e.g. Ohm for parsing, Prolog for type checking, Cordy’s MISTs for portability, etc.
making a compiler blazing fast and efficient is a different problem from creating a language, Production Engineers should worry about efficiency, but these issues should not impose themselves on the initial design, MVI of a language
first, write an interpreter for the language, and only then tighten up efficiency by building a compiler for the language
a compiler / language is just a really big program that takes a long time to implement
Javascript or Lisp are pretty good IRs (Intermediate Representation), LLVM is overkill and complicated, Python is good but it’s indentation-based syntax is painful to generate (I “solve” this problem by generating code in a meta-python (Python with braces instead of indentation) then use a small (40 lines of JS) program to replace brace-bracketed code with properly indented code)
what we “know” about compiler-building is based on crufty old ideas from the 20th century, we have better tools and machines today and we should be able to do better - my PBP kernel] (“multi-tasking”, “coroutining”) (especially - 0d.rt,- stock.rt,- jit.rt) is written in a meta-language- .rtthat generates the kernel in multiple languages (Python, JS, CL - it can do more, but I ran out of interest)

Appendix - T2T

T2T is one of the tools in the PBP toolkit — once you install PBP, you have access to T2T.

Appendix RWR

RWR is part of T2T. This is the specification for RWR

Appendix - PBP

Appendix - Further About PBP

Towards Parts Based Programming

PBP cookbook playlist

Decision Tree Diagram Transmogrifier

State Machine Diagram Tool

FDD LLM - 5 Whys Tool - code repository

State Machine Tool - code repository

Decision Tree Tool - code repository

PBP all tools (PBP, T2T, das2json)

The Spherical Cows of Programming

The Wrong Spherical Cow - First Principles of Parts Based Programming

The Case For Composable Notations (1 / 5) - The Spherical Cow We Forgot We Were Riding

The Case For Composable Notations (2 / 5) - The Restrictions That Came With The Cow

The Case For Composable Notations (3 / 5) - State Isn’t The Enemy The Case For Composable Notations (4 / 5) - Ease of Expression Is the Whole Point

The Case For Composable Notations (5 / 5) - UNIX Already Showed Us the Way

The Case For Composable Notations (6 / 5) - WIP - Notations in Progress

Newspaper

Opinions About Compiler Building

Opinions About Compiler Building

2026-08-12

Appendix - T2T

Appendix RWR

Appendix - PBP

Appendix - Further About PBP

See Also