Ubuntu Server 18.04 – Redirecting output

How to Install Intellij IDEA on Windows 10

Before we go any further, we should have a discussion about redirecting output. One of the many great things about Linux is that it’s made up of many utilities, each doing one thing and doing that one thing well. These smaller commands and utilities become even more efficient because we can take the output of one command and redirect it into the input of another command. Take this command as an example:

cat /var/log/syslog | grep apache2

Here, we’re using the cat command to print the contents of the system log stored at /var/log/syslog. Using the pipe symbol, we’re redirecting the output of that command and feeding it into grep, which is looking for the string apache2. When used together, we’re able to fetch lines from the system log that reference apache2. This is useful if we’re troubleshooting some sort of issue with the apache2 service, and we’d like to look at the log file but ignore any lines that aren’t relevant to our investigation. This is something you’ll probably find yourself doing a lot.

Working with redirecting output is probably best explained with simple text examples. These are admittedly contrived, but understanding how to work with text is important:

echo "this is a test" >> ~/myfile.txt

Here, we’re simply using the echo command, which normally outputs to the Terminal, but instead telling it to direct its output to a file named myfile.txt, stored in your home directory. If we run this command over and over, it will continue to append this string to the file. With a single greater-than symbol, the output from echo will actually wipe out the contents of myfile.txt:

echo "this is a test" > ~/myfile.txt

This distinction is important, because you may want to add a new line to the end of a file, and not have the existing contents deleted. Therefore, it’s best to get in the habit of using two greater-than symbols by default. There will be, of course, situations in which you do want to wipe out a file, but its best to make two greater-than symbols the one you commit to muscle memory.

Effectively, we’re working with a File Descriptor called Standard Output (also referred to as stdout) with commands that produce output. File descriptors are numbers that refer to open files, and these numbers are integers. Standard output is a special file descriptor, with an integer of 1. Anytime text is printed to the Terminal, we call that standard output. Therefore, when the ls command is run, it’s printing the contents of the current working directory to standard output. Another important file descriptor is Standard Error (stderr), which constitutes output that is considered to be produced in error. Standard error is designated by a file descriptor of 2. Why is this important? Sometimes when working with the shell, we would like it to do something different with error output than with normal output. Perhaps we want to run a command, and have all error output go to a file, but any non-error output to not be captured. Sometimes, even vice versa. The best way to illustrate this is with an example. Let’s run this command (make sure to NOT use sudo):

find /etc -name *apache*

With this hypothetical example, we’re telling the shell to look in the /etc directory and return a list of files and/or directories that contain apache in the name.

The problem is that as a normal user (we didn’t use sudo) we don’t have privileges to access every file and directory in /etc. Among the output of this variation of the find command, we might see lines of output similar to this:

find: '/etc/lvm/backup': Permission denied 
find: '/etc/lvm/archive': Permission denied 
find: '/etc/vpnc': Permission denied 
find: '/etc/ssl/private': Permission denied 
find: '/etc/libvirt/secrets': Permission denied 
find: '/etc/polkit-1/localauthority': Permission denied 

With really large directories, we can have so many errors that we won’t even be able to see the results that we actually care about. Obviously, you could just prefix the find command with sudo, but we don’t want to use sudo unless we really need to. For our purposes, there aren’t going to be any configuration files for Apache that we don’t at least have read access to, so we don’t need to use sudo here, but trimming the output would be helpful. We can do this instead:

find /etc -name *apache* 2>/dev/null

We’ve redirected output with a greater-than symbol before, but here we’re doing it a bit differently. We’re directing output from the find command with 2> instead of >. Since 2 is the identifier of stderr, we’re telling the shell to direct any errors to /dev/null. /dev/null is a special device that is similar to a black hole: anything that enters it is never seen or heard from again. Basically, anything copied or moved to /dev/null is eliminated and wiped from existence. Any output we direct there isn’t printed, but deleted. In fact, don’t do this, but you can move a file there to get rid of it too (it’s still easier to use the rm command though).

Going further, we can also direct specific output to a file as well. Here’s an example:

find etc -name *apache* 1> ~/myfile.txt 

Here, we’re copying stdout (not stderr) to a file. The ability to copy stdout or stderr to specific places will increasingly become useful as you become more proficient with the shell.

Comments are closed.