Originally Published: Monday, 3 July 2000 Author: Tom Dominico, Jr.
Published to: learn_articles_firststep/General Page: 1/1 - [Std View]

Shell Skills, Part 2: I/O, Redirection, and Pipes

In this installment of the "Shell Skills" series, you'll receive the background you need before you can start stringing commands together like a pro. Specifically, we're going to talk about terms such as "standard input", "standard output", "redirection", and "piping". Don't be intimidated - we'll give you some examples, and soon you'll be one step closer to mastering the shell.

In this installment of the "Shell Skills" series, you'll receive the background you need before you can start stringing commands together like a pro. Specifically, we're going to talk about terms such as "standard input", "standard output", "redirection", and "piping". Don't be intimidated - we'll give you some examples, and soon you'll be one step closer to mastering the shell.

Often, a command needs someplace to send information, often in the form of text. This place is called "standard output". So, "where" exactly is that? Is it a text file, or the terminal screen, or a printer? Well, it can be any of those places. The key is something called "redirection".

By default, the shell "redirects" any output from a command to a special file that represents your terminal. (In Linux, all devices are represented by special files under the "/dev" directory.) Your terminal is represented by "/dev/ttyn", where n is a unique identifier for that particular terminal. So, by default, all information from a command will be sent to the terminal window. For example, in the case of the "ls" command, this would be a directory listing. There is also a second type of output, known as "standard error". It can be used by programs to separate regular output from error messages, so that they don't get mixed up. For example, you might want to save a log of only the error messages generated by a program. Later in this article, we'll show you how you can redirect standard output and/or standard error to someplace other than the terminal.

A command sometimes needs to receive information from somewhere, in order to do its job. It takes this information from "standard input". If you don't specify otherwise, standard input is read from the terminal. That is, whatever you type at the terminal after entering the command will be taken as input by the command. For example, the "cat" command is usually used to display the contents of a file. However, simply typing the command by itself (without a filename to read), gives you a blank line. If you type a word at the command line and press enter, what you just typed will be displayed to you. The "cat" command has used the text you entered at the terminal as its standard input.

You can choose to redirect both standard input and standard output, so that they come from/go to different places. You do this with the "<" (for redirecting standard input) and ">" (for redirecting standard output) symbols. For example, if I wanted to be able to save a directory listing to a file, I could type the following:

ls -l /home/tom > directory_list.txt

Normally, the shell would have sent standard output to the screen. However, I used the ">" symbol to redirect it to a file. The information "travels" in the direction of the arrow (from the command to the file). However, if the file "directory_list.txt" had already existed, it would have been overwritten. To prevent this, we can append standard output to a file by using the symbol ">>". If the file does not already exist, it is created. For example:

ls -l /home/tom > directory_list.txt ls -l /home/bob >> directory_list.txt

This would create a file called "directory_list.txt", which would contain the contents of the directory "/home/tom". It would then append the contents of the directory "/home/bob" to that file.

As mentioned earlier, there is a second kind of output, called "standard error". A command can send error messages to standard error so that it doesn't get mixed up with standard input. However, by default, the shell sends standard error to the terminal, so you would want to manually redirect it in order to keep it separate. You can do this by using the symbol "2>". The number "2" is simply the way that the shell refers to standard error. Here's an example:

cat nonexistent_file cat: nonexistent_file: No such file or directory

cat nonexistent_file 2> error.txt (Note: This command produces no output, as standard error has been redirected.) cat error.txt cat: nonexistent_file: No such file or directory

You can also redirect standard input, with the "<" symbol. For example, this might come in handy with the "mail" command, which is used to send email from the command line.

mail tomd@linux.com < firststep_is_great.txt

This would send the contents of the "firststep_is_great.txt" file as the body of an email to "tomd@linux.com". If I hadn't specified a filename, it would have taken standard input from the command line.

As you can see, redirection can be a powerful tool. The "pipe" command can be equally powerful. The shell uses a pipe to connect standard output of one command to the standard input of another, all on a single line. For example, the "less" command can display the contents of standard input (usually a file), one page at a time. This is very handy, as the output of some commands can span many pages. Here's an example of how you might use the pipe command ("|"):

ls -l /home/tom/large_directory | less

This would send the contents of the directory to the "less" utility, which would then display them one page at a time. Here's a slightly fancier example:

who | sort | lpr

This takes the output of the "who" command (which lists the users currently on the system), passes it to the "sort" filter (which sorts the listing by username), and then prints it out with the "lpr" command. None of these commands were specifically designed to work with one another, but since they all use standard input and output in the same way, they can do so easily.

As you can see, the use of redirection and pipes can be very powerful. The number of ways in which you can string commands together to perform complex tasks is limited only by your imagination. The command line can be a very powerful tool, when used to its full potential. Hopefully, you've gotten a small taste of its power. Now that you have the proper background, we'll cover some essential commands in the next "Shell Skills" article. Until then, feel free to experiment with the different ways in which you can string commands together to perform useful tasks. Have fun!