As any regular Unix user can tell you, there are three primary “streams” in which programs are fed and output data.    STDIN, which by default is the keyboard.   STDOUT, which by default is your terminal/screen, and STDERR, which by default is also your terminal/screen.   In addition to their textual reference, these three also have file descriptor numbers assigned (you’ll find out why in a moment).  Good Unix shell users regularly take full advantage of all of these.   In fact, if you’ve done any Unix or Linux work you’ve likely used them without even knowing.   For example.

cat ~/textfile.txt | more

You’ve certainly used the pipe (‘|’) character before, but do you know what it is actually doing behind-the-scenes?   Effectively, the pipe tells the shell to “stitch” the STDOUT from the ‘cat’ command to STDIN of the ‘more’ command.    In Unix-land this is referred to as creating a pipeline.

Master of redirection

In magician terms, redirection means to draw away the focus of the audience so something sneaky and magical can be occurring elsewhere.   While there’s very little sneaky going on in the Unix shell, when used correctly STDIN and STDOUT redirection can be pretty magical.   For instance:

/usr/bin/somepossiblybrokenprogram.sh  1> /var/log/myapp.log  2> /var/log/myapp-error.log

Stiches the “regular” output of your possiblybrokenprogram to STDIN of a log file, while the errors go to STDIN of a log file aptly named myapp-error.log.    Remember I mentioned the file descriptor numbers?   Now you know what they’re for, and here’s how they’re assigned:

STDIN  - 0
STDOUT - 1
STDERR - 2

Now you should be able to see how we did that bit of “magic” above.   Redirection of output can be darn handy when debugging or extremely useful when running things as scheduled (cron) jobs.   Also note: by default the ’1′ when redirecting STDOUT is unnecessary, since using a bare “>” assumes you mean STDOUT.  We specified it above for clarity’s sake, and you’re welcome to do it as well.  Either way works.

There is a downside to this however–you can only redirect any given file descriptor once per command. …but what if you want to log things to a file and see it on the screen at the same time?  Impossible right?   NAY!

Line up to the ‘tee’

That’s exactly where the ‘tee’ utility comes into play.    Don’t think of ‘tee’ as in a golf tee, but rather like a ‘tee’ in plumbing terms.  In other words, it takes the stream and splits it in two, three, or even more:    STDOUT and STDIN simultaneously.  Even STDOUT and STDIN multiple times.   I know you’re already scratching your head, so how about a few examples.  First the simplest:

/usr/bin/somepossiblybrokenprogram.sh | tee /var/log/myapp.log

Hopefully you see what’s going on here.   Instead of wholly redirecting STDOUT to a single file, we’ve instead utilized the ‘tee’ command to split it to both the screen and a file named /var/log/myapp.log. Pretty handy huh?   But wait, there’s more!

/usr/bin/somepossiblybrokenprogram.sh | tee /var/log/myapp.log /var/log/other.log /var/log/joe.sh.log

You see what it’s doing?  Not only is it displaying to the screen, but it simultaneously wrote the output to three separate log files!

Finally the ‘tee’ command has an option to append to files instead of overwriting them each time it is invoked–which would be a bummer if you were debugging something.   You do that like so:

/usr/bin/somepossiblybrokenprogram.sh | tee -a /var/log/myapp.log /var/log/other.log /var/log/joe.sh.log

There you have it!  The very tiny, but immensely useful ‘tee’ command.

Learning Linux Commands is part of a blog series that highlights useful Linux/Unix commands for our web hosting clients.  Keep our blog RSS feed refreshed for new entries–we’ll be adding many more soon!  

Much to my surprise he asked for help with amazingly complex methods he had tried.

Okay everyone, no need to hit this thumbtack with a sledgehammer!  Let’s say you have a file “textfile.txt” and want to put “<Header>” on the top:

echo -e “<Header>\n$(cat textfile.txt)” > textfile.txt

There you go, nice and simple.