LLM workflows

I’ve been waiting to write about LLMs because I’ve had very little to say about LLM’s that isn’t been obvious. All prompt engineering advice I’ve had and came across online distilled down to:

1. prompt better like you would to a human.

2. Tell the LLM to be good at it’s job (fun quirk of LLMs is that it does the average of what it’s seen, so careful prompting can deal with slop)

and I don’t need to tell you that. Most of my usage has been the obvious usage, and since I had nothing novel so say, I said nothing. However, lately I’ve started to come across some non-obvious ways to use LLMs.

Surgeon style programming

We can reorganize ourselves as a head surgeon from the surgical team in The Mythical Man Month. The book, from 50 years ago discusses that every surgical team needs a human language lawyer, human toolsmith, etc. but getting people to agree to that was hard. Well now, you can go forth, be the surgeon, and ask away for language questions and tools. As the LLMs get better, they’ll do a better job of being the specialists mentioned in the book and you’ll need to be a better head surgeon.

Single File Pair Programming

I’ve been pair programming a lot lately with chatgpt 4o in nvim, here’s what it looks like:

Let say we’re working on a file, call it “program”, and you want some help, so you write in a file, prompt.txt, the following:

hey can you fix all of the foo widgets to be bar widgets?

Then, we run nvim. Pick the lines you want, run the following:

:SELECTION .! ./pair

which is implemented as

#!/usr/bin/env bash
cat ~/code/prompt-lib/just-code.txt ~/code/prompt-lib/top-coder.txt prompt.txt - | gpt -m gpt-4o

where gpt is my wrapper around chatgpt (you can get it here)

sudo add-apt-repository ppa:code-faster/ppa
sudo apt update
sudo apt install gpt-toolkit

and viola, chatgpt will fix your foo widgets for you. Alternatively I could’ve had the prompt been arguments to pair, but personally I like to be able to modify my prompt in a file rather than worry about getting it right inside of the command editor in nvim. I’m sure emacs users will feel smug about this.

What makes this flow interesting is that this is the ideal flow. You ask chatgpt to fix it, and it. does. It may not be computer efficient, you have to pay for the entire section to be rewritten, rather than say a smart script that does a find and replace. But it is human efficient in accordance with “make the machine do the dirty work”.

For now, this flow only works with single file modifications. We’d need either AI with more codebase awareness or significantly cleverer prompting to do anything more powerful.

Sometimes I’ll also throw in an input file as context which might be the input to program, which is an easy tweek with this workflow.

hey can you fix all of the foo widgets to be bar widgets? Below is an example input followed by the program

#!/usr/bin/env bash
cat ~/code/prompt-lib/just-code.txt ~/code/prompt-lib/top-coder.txt prompt.txt input - | gpt -m gpt-4o

Prompt-lib

LLMs are usually quite opinionated in that they like to talk. And we like the rule of silence because we just want code. So I have a lot of repeats in my prompting. “Just output code.” “Give me this particular style of JS” Thankfully since everything’s plaintext, we can store these two ideas each as text files and slap them together with our prompt and code with cat at the end. These are basically library files, but instead of functions, they’re composable promptlets. (personally, I suspect one of the main solutions to the AI slop problem will be an AI that specializes in outputting useful prompts.)

Large Refactors

We’d need either AI with more codebase awareness or significantly cleverer prompting to do anything more powerful.

Let’s do that now.

Sadly I don’t have a good example for this blog post, so bear with my hand waving.

For a find and replace example, we’d use find to get the whole project tree, then feed it to chat gpt and have it generate a sed script for the whole project, then run it in bash.

Something like:

find > tree.txt
echo "write a sed script to fix foo to bar for this tree" > prompt.txt
cat just-code.txt prompt.txt tree.txt | gpt -m gpt-4o > refactor
# inspect just in case
cat refactor
cat refactor | bash

For something more sophisticated we would have to ask chatgpt to generate a more sophisticated program capable of more sophisticated refactors, for example tsmod. At this point we need to do serious work to give chatgpt the context of the files needed to generate the program. Unclear if worth.