xargs considered harmful

Just Use Sed

Jul 16

Everyone loves xargs. Except me. I hate it.

But, but, everyone loves

$ ls | grep foo | xargs rm

Weird, because:

$ rm $(ls | grep foo)

is a lot shorter. To understand why I hate it, look at the bugs section.

The -L option is incompatible with the -I option, but perhaps should not be.

Huh? Two options of the same program are incompatible. That sounds like bad design. And, well, that’s because it’s bad design.

And from xarg’s bad design, we can learn good design and therefore faster coding.

Let’s keep digging:

-L, from the man page

-L max-lines
Use at most max-lines nonblank input lines per command line. Trailing blanks cause an input line to be logically continued on the next input line. Implies -x.

and for example:

$ seq 10 | xargs -L 3
1 2 3
4 5 6
7 8 9
10

I love xargs -L. It’s really great for grouping lines. No other program does it.

UPDATE 2020-07-18, taharqa and others showed me that paste can be used to do this:
$ seq 10 | paste -d' ' - - -
I now hate xargs -L.

-I, from the man page

-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character. Implies -x and -L 1

and for example:

$ seq 10 | xargs -I X echo xXx
x1x
x2x
x3x
x4x
x5x
x6x
x7x
x8x
x9x
x10x

Which… looks a lot like sed piped into bash:

$ seq 10 | sed 's/.*/echo x&x/' | bash
x1x
x2x
x3x
x4x
x5x
x6x
x7x
x8x
x9x
x10x

Except, it doesn’t always work. Let’s say we try to use xargs in a simple case, just run the input (like bash)

$  echo seq 10 | xargs -IX X
/usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server

Huh..what? Yup, -I is finnicky. Plus it doesn’t play with -L. I hate it, it’s awful.

But why is xargs -I so bad?

Why is it finnicky and why does it lead to a bug?

Mainly because xargs is trying to do 3 things and so does some of them (1,2) poorly:

Generate commands based on data from stdin (ex. xargs -I X rm xXx)
run commands (ex. xargs -I X rm xXx)
group data from stdin (ex. xargs -L)

They all seem fine when you start with ls | grep foo | xargs rm. Generalize how many lines get grouped into the output (-L), and generalize command generation (-I). But these quickly tangle into a mess. Command generation assumes only 1 line per output. grouping assumes STDIN is slapped on the end of a line.

Better design

UPDATE 2020-07-18, taharqa and others showed me that paste can be used to do -L. As such, xargs can be replaced by other tools and therefore shouldn’t exist.

xargs should only have -L. So seq 10 | xargs -L 3 is a great usage of xargs. Grouping lines is useful and unique. It’s simple and won’t merit a bugs section.

What’s strange is that in a better design xargs should also not take a command, only -L. So no more xargs rm. Huh!? The most common use of xargs…is bad design? Yup.

What if you want to delete a bunch of files. You could just use bash:

$ rm $(ls | grep foo)

Or sed/bash if you want a pipeline:

$ ls | grep foo | sed 's/.*/rm &/' | bash

If you need to generate more custom commands from the input, change the sed command to your custom command.

The General Lesson

The lesson we learn is that the unix inventors were right. Again.

Write program that do one thing and do it well.

If you’re tempted to do 3 things like xargs did. Lie. Down. Wait until the feeling passes. Then, get up see if you can already do what you want with existing tools. For xargs, you can. So, just don’t. It’ll be (infinitely) fast to write, you’ll have feature parity with the original monstrosity, and there won’t be a bugs section about how two features don’t play nice.