Mastering Sed: Capture the Flags

In this tutorial, we’ll explore sed’s useful command-line flags. Without CLI flags, sed’s clunky. Oh you want to perform find and replace on a file? First you need to create a temp file, and don’t forget to clean it up! Tedious. Knowing these sed flags makes our life fast and easy. I know, grinding through facts like what each flag does is tedious. Plus why bother remembering when we can always look at the man page. All valid points. That’s why this tutorial, like all of my tutorials, is interactive. Each flag we go through comes with an example that you can try yourself in your own terminal. Even if all you do is copy and paste the examples and skip the dry reading, it’s worth. Trying an example primes your brain for the next time you see something similar. Your brain will think, "Hey, didn’t I do something like this with some sed flag that one time on CodeFaster? Yeah, I did, let me look it up again" You’ll come back, find the flag, actually read the short explaination to understand the flag just enough to solve your problem quickly and get back to shipping code.

-n

Example

seq 10 | sed -n 1p

Explanation

sed -n suppresses sed’s normal behavior of printing everything. This flag is crucial when using sed to print parts of its input. We saw this in part 1, but it’s so important that it’s worth revisiting.

-i(nplace)

Example

echo find > f.txt && sed -i "s/find/replace/" f.txt && cat f.txt

Explanation

sed -i allows us to edit files in place with sed. Without -i, sed reads from stdin and writes to stdout. With -i, sed reads from a file and then writes to the same file. We need this because sed "s/find/replace/" < f.txt > f.txt does not work. > f.txt will overwrite f.txt to be blank before it’s read.

Technically, -i is unnecessary because we could use temporary files to get the same effect. However managing temporary files tedious and slow, so -i is preferred.

-e(xpression)

Example

echo find | sed -e '1i\first' -e 's/find/second/'

Explanation

sed -e allows us to execute multiple sed commands in a single invocation. Although multiple commands can usually be delimited with a semicolon (for example, 2p;4p), the a and i command cannot be because they treat ; as text to be inserted/appended.

Technically, -e is unnecessary because we could use multiple sed commands composed with |. However, if sed is being used with flags -i or -z, each invocation needs the same flags, which gets tedious. We’d rather just use -e and save ourselves the effort.

-z

Example

find -print0 | sed -z -e 's/\0/\n\n/g' -e 's/$/\n/'

Explanation

sed -z uses the null byte (\0) to separate "lines" instead of the newline byte (\n) which sed normally uses. Data will be separated by null bytes when the data itself could contains newlines. The classic example is filenames containing newlines. Yes Unix lets you do this. Yes it’s the right design decision. No you shouldn’t ever do it. To handle this unusual case find has a -print0 which outputs filenames delimited by null bytes instead of newlines. We can then use sed -z to process this data. Perhaps, even to write a program that changes the filename to something without newlines.

Conclusion

Finally. We got through the flags. Boring, but important work. We’ll have some fun with these flags next week to automate repetitive tasks like code refactoring. Refactoring by hand is slow and error prone, but a program can fix hundeds of files in under a second. If you don’t want to miss this, just click the "Subscribe now" button.