Useless Use of Cat Award (partmaps.org)

I would still use the former.

Chances are, I'm not going to remember awk's somewhat weird, more unfamiliar syntax -- everyone knows about cat, everyone knows about piping, those two go very well together.

Using cat with pipes typically linearizes $x operation in our minds, it makes it easier to come up with solutions, and makes it easier to change them as modification is needed.

I don't think it's fair to get awk involved. It's a scripting language, not a single-purpose utility.

It's silly when set up like that. But I still prefer the separate invocations -- they become building blocks. It means I can replace the "cat file" with "echo testdata", it reads nice, and for more complicated scripts, it's easy to format across lines:

    zcat /var/log/somdeamon/* \
     | grep "simple event like maybe 'error'"
     | grep "complicated date expression or whatever"
     | some other filter
     | summation?
     | maybe sort and uniq

I can start that with a simple (z)cat of one file (rather than the thousands of lines I might want to work on) -- and it doesn't require much arcane knowledge of each of the tools used, for each step.

I do agree that awk might be a little under-appreciated, but if you really feel that way, maybe you should just use snobol.

To me a "wasted cat" clearly states intention: get/feed data. Then the rest of the steps either filter, sum, or manipulate data.

The final thing that makes the "waste"-argument silly, is that Linux[1] has pretty lightweight processes. I'm not sure what the state of the various BSDs are, but it'd surprise if at least FreeBSD hasn't gotten a lot better on this too. Either way, one would think that any text pipeline would be io-bound, not limited by the number of forks...

[1] http://bulk.fefe.de/scalability/

Your first line of code is much better. It optimizes for clarity, presenting the programmer's intention in a simple logical sequence. It also likely optimizes for programmer time and mental expenditure.

Why not:

  awk '/bar/ { print $1 }' file | sort -u

I find those kind of examples a little unfair as what you're actually doing is comparing the syntax of different shells/languages (the Bourne Shell vs awk). You could write examples in Perl or Python as well - to prove how pointless awk and sort is as well if you really wanted. In fact you could even rewrite sort in awk.

While I disagree with most of the comments on here about how:

    cat $file | grep "pattern"

is easier to read than

    grep "pattern" $file

(in my opinion, if you're a sysadmin then you should know how grep works and thus the optimised solution shouldn't be any harder to read. But at the end of the day, whatever gets the job done).

However I do draw the line at your example because you're now comparing apples to oranges and suggesting that everyone should learn an additional programming language on top of memorising every standard Unix/GNU/BSD command and their flags.

    <$file grep "patter"

Are you sure about that, what if file contains the equal sign?

  $ touch foo=bar
  $ file foo=bar
  foo=bar: empty
  $ awk '/bar/ { print $1 | "sort -u"}' foo=bar
  
  
  
  
  
  
  bar
  
  screw this!
  ^D
  bar
  $ awk '/bar/ { print $1 | "sort -u"}' <foo=bar
  $

I'm going to agree with you here. 99% of the time this UUOC stuff is just premature optimization. If it gets the job done it does not matter if it wastes a few milliseconds.

    cat somefile |while read line
    do
      echo $line|cat -|pipe...
    done

   for f in *;
   do
      cat $f | ...
   done

    exec 5<words.txt
    while read line <&5 ; do
        echo "line read: $line"
    done
    exec <&5-

Exactly. Things like the "useless use of wc -l" fall into a similar category for me. When I want to count the lines of stdout, I use "wc -l". It may not be the simplest, most efficient or fewest keystrokes, but it is extremely easy to remember and fits well into the model of a pipeline of simple single-purpose tools. I tend to use cat at the beginning of pipelines just because it keeps the flow left-to-right, and makes it much clearer what the source of the data is.

There's value in knowing some of the advanced things that the textutils and bash can do together, and sometimes there's value in using those things. I don't dispute that, but I do dispute the need to shame people who use the tools in ways that you personally don't like.

Shame or not, I'm going to keep using cat at the beginning of my pipelines unless I have a really good reason not to. If that makes some people think I'm not in the Unix wizard in-group, I don't really care.

    cat file

    cat file | grep foo

    cat file | grep foo | wc -l

  less file

  less file | grep foo

  less file | grep foo | wc -l

As an alternative, you can always rewrite:

   cat file | grep xyz

To:

    <file grep xyz

   <file grep xyz

    < file grep xyz

This form is also nicely modular, setting up a data source at the left of the pipe in a way that's easy to modify to swap in a different kind of data source:

   cat file | grep xyz
   gzcat file | grep xyz
   xzcat file | grep xyz
   pv file | grep xyz
   curl url | grep xyz

There are many places I get data, and it sometimes changes (e.g. I decide to compress a file and read it back with xzcat instead of cat, leaving the rest of the script unchanged). Unless you give me an alternate syntax that can handle all of them, I'm not interested in the fact that a particular utility has chosen to provide its own idiosyncratic syntax to special-case only the 'cat' data source, with an -f parameter or whatever.

    cat file |

    <file

    <file grep xyz

I agree, and like the fact that you can then read your commandline as a series of steps, going from input to output.

However, in bash (and other shells?) you can write "<file grep xyz" rather than "cat file | grep xyz".

You could argue that people who complain about wasting a cat are ignoring the Unix philosophy of "do one thing and do it well." Moreso with wc -l vs grep -c. If I know that wc -l is how to count lines, I don't want to remember the options for counting lines built into every other command.

>Mentally, I think of most Unix utilities as operating on STDIN and producing STDOUT

Yeah, that is actually very much the point. You are passing the contents of the file in to grep's STDIN. So why not just do exactly that rather than cat it first for no reason? You don't need to remember how to tell commands to read from a file, you just need to learn the basics of how your shell works.

    <file grep xyz

Shorter version in bash: |& less

It redirects both outputs at once.

Be careful when redirecting STDERR to STDOUT if yo uare also planning to redirect STDOUT to a file. The order in which things are done affects the outcome. These are not the same:

  # Redirect STDOUT to file, and STDERR to file
  command >/tmp/command.out 2>&1
  
  # Redirect STDOUT to file, and STDERR to screen (as STDOUT)
  command 2>&1 >/tmp/command.out

The important thing to remember is that 2>&1 is redirecting to what STDOUT is currently pointing to, not redirecting to STDOUT itself.

  command >& /tmp/command.out

Why is it not actually useless?

Or just

    ls -1

(that's a digit one)

I use this for programs which otherwise output to pagers, when I'd rather have the output just in terminal scrollback. (And unlike the -1 option it's generic to all programs with terminal behaviors.)