Pipes and Redirects

The shell's true power isn't any single command — it's the ability to compose commands. Pipes and redirects let you connect programs together so the output of one becomes the input of the next. This is the design philosophy of Unix: small tools that each do one thing well, combined into pipelines that do complex things simply.

Standard streams

Every program in Unix has three standard streams:

• stdin (0) — input, usually from the keyboard
• stdout (1) — normal output, usually printed to the terminal
• stderr (2) — error messages, also printed to the terminal by default

Pipes and redirects manipulate these streams.

The pipe: |

The pipe | takes the stdout of the left command and sends it as stdin to the right command:

ls -la | less           # page through a long ls output
ls | sort               # list files in alphabetical order
ls | wc -l              # count the number of files

Chain as many pipes as you need:

ls -la | grep ".sh" | sort | head -5

This: lists all files in detail → keeps only lines containing .sh → sorts them → shows the first 5. No intermediate files, no temporary variables — just data flowing left to right.

Output redirection: > and >>

The > redirect operator sends stdout to a file, creating or overwriting it:

echo "Hello" > output.txt          # creates output.txt with "Hello"
ls -la > filelist.txt              # saves ls output to a file

>> appends to a file instead of overwriting:

echo "First line" > log.txt
echo "Second line" >> log.txt
echo "Third line" >> log.txt
cat log.txt                        # shows all three lines

Input redirection: the < operator

< redirects a file into a command's stdin:

sort < names.txt          # sort reads from the file rather than the keyboard
wc -l < report.txt        # count lines in report.txt

In practice, many commands accept filenames directly (sort names.txt), so < is used less often than | and >. But it's essential to understand because it completes the picture of how streams work.

Redirecting errors: 2>

Error messages go to stderr, not stdout. That means they're not captured by > or | by default:

ls /nonexistent > output.txt       # error still appears on screen
ls /nonexistent 2> errors.txt      # error goes to errors.txt
ls /nonexistent 2>/dev/null        # discard the error entirely
ls /nonexistent > out.txt 2>&1     # merge stderr into stdout, both go to out.txt

/dev/null is a special file that discards everything written to it. When you want to silence errors, redirect to /dev/null.

Essential pipeline tools

These three commands become central to almost every pipeline:

grep — filter lines matching a pattern:

ls -la | grep ".txt"               # show only .txt files
cat /etc/hosts | grep "localhost"  # find localhost entries
grep -r "TODO" ./src/              # search for TODO recursively in a directory
grep -i "error" app.log            # case-insensitive match

wc — count words, lines, or characters:

wc -l file.txt                     # count lines
ls | wc -l                         # count files in current directory
cat essay.txt | wc -w              # count words

sort — sort lines of text:

sort names.txt                     # alphabetical
sort -r names.txt                  # reverse alphabetical
sort -n numbers.txt                # numeric sort
ls -la | sort -k5 -n               # sort ls output by file size (column 5)

A real pipeline

Here's a pipeline that finds the most common words in a file:

cat essay.txt | tr ' ' '\n' | sort | uniq -c | sort -rn | head -10

Breaking it down:

1. cat essay.txt — read the file
2. tr ' ' '\n' — replace spaces with newlines (one word per line)
3. sort — sort alphabetically (required before uniq)
4. uniq -c — count consecutive duplicates
5. sort -rn — sort by count, descending
6. head -10 — show only the top 10

Each command does one small job. The pipe connects them into something genuinely useful.

Do it yourself

Try these pipelines in your terminal:

# Count how many files are in /usr/bin
ls /usr/bin | wc -l

# Find all .sh files in the current directory tree
find . -name "*.sh" 2>/dev/null

# See the most recently modified files
ls -lt | head -10

# Create a file, write to it, then search it
echo "apple" > fruits.txt
echo "banana" >> fruits.txt
echo "cherry" >> fruits.txt
echo "apricot" >> fruits.txt
grep "^a" fruits.txt | sort

Where to go next

You've completed the Shell basics module. You can navigate the filesystem, manage files, use variables and the environment, write scripts, and compose commands into pipelines. The lab is next — put it all together with hands-on practice challenges.