Write code that writes code

Wouldn’t you rather write a program than type the same thing 100 times? Me too, let’s talk about how.

I fell in love

Let me tell you about a time I fell in love. With code that writes code. In 2016, I was working with a terrible library in Java. One such issue was that instead of using reflection, there was a large amount of boiler plate. My team had to patch it because it was incomplete and buggy. Every field we wanted to add required 12 lines of code. 12 lines that involved snake_case, camelCase, and GoCase.

Here’s an example for one field total in cents:

@XmlElement(name = "total_in_cents")

private Integer totalInCents;

public Integer getTotalInCents() {
  return totalInCents;
}

public Integer setTotalInCents(final Object totalInCents) {
  this.totalInCents = integerOrNull(totalInCents);
}

Notice this has “total_in_cents”, “totalInCents” and “TotalInCents”. And I had to add ~60 of these, for a total of ~800 lines. Doing this by hand would be grueling non-stop typing. Typing that’s error prone and would inflame my newly developed RSI. Using find and replace would help, but still require a monsterous amount of repetition. 3 find and replaces per sectionx60…180 repetitions of almost the same thing. Repetition…maybe my computer could to do this for me? Luckily, I had just read The Art of Unix Programming (a must read for any fast programmer) which talked about the:

“Rule of Generation: Avoid hand-hacking; write programs to write programs when you can”.

This was a perfect opportunity to try it out. 2 pleasant hours of reading documentation, learning a new tool (m4), and coding later, I had a data file ~60 lines long, a macro script 12 lines long, and 12 lines of boiler plate, for ~84 lines which expanded 10x into ~800 lines of perfect, error free code. I was in love.

How to write code that writes code

Writing code that writes code is easy. Code is text. If we can write a program that outputs text, we can write a program that writes code. Here’s an importantly pithy example in bash that generates a hello world program:

echo echo hello world

This pithy example is important because writing code that writes code sounds advanced and requires sophisticated techniques. It isn't and it doesn’t. Writing code that writes code is easy. For a less pithy example, here’s a 39-line python solution to my problem above:

#!/usr/bin/env python3

import json
import sys

def first_char_upper(s):
    return s[0:1].upper() + s[1:]

def first_char_lower(s):
    return s[0:1].lower() + s[1:]

def snake_case(l):
    return "_".join(l)
# method names in python should be _, but for a blog post, using the convention that the method creates is better
def GoCase(l):
    return "".join([first_char_upper(w) for w in l])

def camelCase(l):
    return first_char_lower(GoCase(l))

def code(e):
    s = ""
    s += "    @XmlElement(name = \"{}\")".format(snake_case(e["name"])) + "\n"
    s += "    private {} {};".format(e["type"], camelCase(e["name"])) + "\n"
    s += "    public {} get{}() {{".format(e["type"], GoCase(e["name"])) + "\n"
    s += "      return {};".format(camelCase(e["name"])) + "\n"
    s += "    }" + "\n"
    s += "" + "\n"
    s += "    public {} set{}(final Object {}) {{".format(e["type"], GoCase(e["name"]), camelCase(e["name"])) + "\n"
    s += "      this.{} = {}OrNull({});".format(camelCase(e["name"]), first_char_lower(e["type"]), camelCase(e["name"])) + "\n"
    s += "    }" + "\n"
    s += "\n"
    return s

# unix filter that takes JSON on stdin and writes Java on stdout
# Built as RLW
# R
data = json.load(sys.stdin)
# L
output = "".join([code(e) for e in data])
# W
print(output)

Notice how straight forward the code is. There’s no tricks, no advanced algorithms, no ASTs. Just simple text generation.

Tools

Python

I'll keep saying it, python is the programming language for a fast coder. It does everything. Including code generation. When solving a generic code generation problem, python is our goto. Like the solution above, I recommend the program’s interface be a unix filter that takes in JSON on stdin and it writes the generated code to stdout. Internally, I recommend the code be structured as an RLW.

Jinja

Jinja’s a w̶e̶b̶ text template engine for python. Although Jinja’s meant for generating web pages, its really a general purpose text template engine. Code is text, so we can use it to generate code too. Here’s a jinja template to generate the presented code.

    @XmlElement(name = "{{ snake }}")
    private {{type}} {{ camel }};
    public {{type}} get{{go}}() {
      return {{camel}};
    }

    public {{type}} setTotalInCents(final Object {{camel}}) {
      this.{{camel}} = {{lower_type}}OrNull({{camel}});
    }

and its called with

jinja2.Template(text).render(
    snake=snake_case(e["name"]),
    camel=camelCase(e["name"]),
    go=GoCase(e["name"]),
    type=e["type"],
    lower_type=first_char_lower(e["type"]),
)

Pretty simple and straightforward.

Sed

sed outputs text, so it can also generate code. For example, this sed script takes in a list of files and creates a bash program which backs them up.

sed 's/.*/cp & &.bak/'

Bash

Our pithy example for code generation was bash to bash. This highlights the two ways to use bash when generating code, as a generator and target. Bash contains an amazing number of powerful programs from file operations to network utilities which are easy to invoke. Sometimes its easier to invoke them by generating the bash rather than trying to do a system call from the host program.

m4

m4 is the classic unix macro expander. A macro expander generates text, so it should be great at generating code.

Don’t use m4.

It’s unpopular in 2020 and has an unusual syntax. This means getting help is slow, and writing m4 is quite slow. The only time you should use m4 is when its already being used like with sendmail. Otherwise, stick to python.

Conclusion

We’ve seen how valuable and easy it is to write code that writes code. It is much more beneficial than manually writing out tedious code again and again.

This post also concludes our fast typing post. Next week, we’ll start a new series, “look ma, no mouse”. We’ll go over all of the tools you need to do everything while keeping your hands on the keyboard. If you don’t want to miss out, just click “Subscribe now” below.