>, >>, <, and |

Shell pipes allow you to conveniently redirect command-line output.

See also

Examples

Redirect stdout to file

Redirect stdout to a file and stderr to the same place

Redirect stdout to another program’s stdin

Use a file as a program’s stdin

Redirect to an output file > and >>

The > character will redirect a program’s output to a file instead of printing it to the terminal.

$ ls
file.txt music.mp3
$
$ ls > output.txt
$ cat output.txt
file.txt
music.mp3
output.txt

The > character will overwrite the output file.

The >> character acts the similarly to >. However, >> will append to the output file instead of overwriting it.

This makes it easy to append lines to a file

$ echo 'more' > output.txt
$ cat output.txt
more
$ echo 'and more' > output.txt
$ cat output.txt
and more
$ echo 'beautiful append' >> output.txt
$ cat output.txt
and more
beautiful append

Output redirection characters will not add a trailing newline to the output file.

Output redirection characters will create the output file if it does not exist. However, they will not create missing directory structures.

stdout and stderr

Some command (like curl) will output to both stdout and stderr. By default, the > and >> markers use 1=stdout.

A file redirection pipe can choose between these streams by choosing the file descriptor number to redirect.

The standard file descriptor numbers are 1=stdout and 2=stderr.

Redirecting the complete output of curl to a file could look as follows:

$ curl example.com 1>example.html 2>example-status.log
$ cat example.html
<html>
...
</html>
$ cat example-status.log
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   169  100   169    0     0   2159      0 --:--:-- --:--:-- --:--:--  2139

You may want both streams to go to the same file. If you rewrite the command above to naievely use the same file for both, you will likely notice that the % Total headers lines do not appear. In short, this is due to the file being opened two times by the seperate references in the pipe operator.

However, it is still possible to redirect both streams to the same file without losing output.

$ curl example.com 1>example-full.html 2>&1
$ cat example.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   169  100   169    0     0    459      0 --:--:-- --:--:-- --:--:--   460
<html>
...
<html>

The 2>&1 piece redirects output from stderr to stdin. Since stdin is already redirected to the output file, stderr will use the already-opened file.

Order matters - this technique depends on the stdout stream already being redirected to the output file before the stderr stream is redirected to the same place. Putting the 2>&1 before the 1>example-full.txt will result in stderr still getting printed to the terminal.

Note: In these examples, I explicitly specified stdout using 1>file.txt. Since the redirectors use stdout by default, the 1 could be omitted. i.e. >file.txt would work just the same.

Viewing Output

You can view changes to an in-progress output file in real-time using tail with follow

tail -f output.txt

or, to include all lines in the file

tail -n +0 -f output.txt

Redirect output to the next program’s stdin |

Some programs (like grep) accept stdin as an input. The | operator pipes a program’s stdout to another program’s stdin.

$ ls
beefslab.html  file.txt  music.mp3  output.txt
$ ls | grep .txt
file.txt
output.txt

The pipe operator only accepts the stdout stream of he source program. Unlike > and >>, you cannot override this by adding a simple file descriptor number.

However, you can take advantage of 2>&1 redirection

$ curl beefslab.com 2>&1 | grep er
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
<head><title>301 Moved Permanently</title></head>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.18.0</center>

To pipe exclusively stderr, you can first pipe the original stdout to /dev/null

bash$ curl beefslab.com 1>/dev/null 2>&1 | grep er
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

SHELL NOTE: For me, this technique works with bash@5.1.16 but not zsh@5.9

See also

Use a file as a program’s stdin <

Sometimes it is useful to use a file as stdin. This can be done using <.

The following command will execute each SQL query in queries.sql in the PostgreSQL database by using queries.sql as the input stream rather than the terminal.

$ psql postgres postgres < queries.sql
DROP TABLE
DROP TABLE
ALTER TABLE
...

This is better than something like cat queries.sql | psql postgres postgres since it does not include the overhead of cat executing and reading the file into a stream for psql.

Other References