AdvancedShell mastery

Heredocs and Herestrings

Use <<EOF heredocs, <<-EOF for indented scripts, <<< herestrings, and process substitution with <(cmd) and >(cmd) to pass data without temp files.

BashAdvanced10 min read

Recommended first

Parameter Expansion

By the end of this lesson you will be able to:

Write heredocs with <<EOF and <<-EOF for indented use
Pass multi-line strings to commands without a temporary file
Use <<< to feed a string directly to a command's stdin
Use <(cmd) process substitution to treat command output as a file
Use >(cmd) process substitution to treat a command's stdin as a file

Passing multi-line text to a command — a config block, a SQL query, a template — is a common scripting need. The naive approach creates a temporary file, writes to it, passes the path, and then cleans it up. Heredocs, herestrings, and process substitution eliminate that pattern entirely, keeping the data inline with the script and the plumbing invisible.

Heredocs

A heredoc feeds a block of text as stdin to a command. The text ends when the delimiter appears alone on a line:

cat <<EOF
Line one
Line two with $HOME expanded
EOF

# Send a heredoc to a file
cat > /tmp/config.yaml <<EOF
host: localhost
port: 5432
database: myapp
EOF

By default, variables are expanded inside the heredoc. To suppress expansion (useful for code templates or SQL with dollar signs), quote the delimiter:

cat <<'EOF'
This $variable is not expanded.
Neither is this $(command substitution).
EOF

Indented heredocs

A <<- heredoc strips leading tabs (not spaces) from the body, so you can indent the content to match the surrounding code:

configure_app() {
	cat > /etc/app/config.yaml <<-EOF
		host: localhost
		port: ${APP_PORT:-8080}
		debug: false
	EOF
}

<<- strips tabs, not spaces. If your editor converts tabs to spaces (common in Python-focused or space-normalising setups), the indentation will not be stripped and the heredoc will include leading spaces. Either configure your editor to keep tabs for shell scripts, or use <<EOF with no indentation on the body lines.

Herestrings

A herestring feeds a single string directly to a command's stdin — cleaner than echo "..." | cmd because it avoids a subshell:

# Read a string with read (no subshell — variable is available after)
read -r first rest <<< "Alice Bob Charlie"
echo "$first"   # Alice
echo "$rest"    # Bob Charlie

# Pass a variable to a command expecting stdin
grep "pattern" <<< "$content"

# Quick arithmetic via bc
result=$(bc <<< "scale=2; 22/7")
echo "$result"   # 3.14

The subshell advantage is important: echo "Alice Bob" | read first rest would run read in a subshell, and the variables would be lost after the pipe. Herestrings avoid this.

Process substitution with input

<(cmd) runs cmd in a subshell and makes its output available as if it were a file. Commands that require a filename (rather than stdin) can use it:

# diff two command outputs without temp files
diff <(sort file1.txt) <(sort file2.txt)

# comm requires two sorted files
comm <(sort list_a.txt) <(sort list_b.txt)

# while read without the pipeline-subshell problem
while IFS= read -r line; do
  echo "Got: $line"
done < <(find /var/log -name "*.log" -newer /tmp/marker)

The while ... done < <(cmd) pattern is the idiomatic fix for the pipeline subshell problem seen in the loops lesson: variables set inside the loop are visible after it ends because the while runs in the current shell.

Process substitution with output

>(cmd) is the mirror: it provides a writable "file" that pipes data into cmd. Useful for tee-style logging:

# Log to both stdout and a file simultaneously
make build 2>&1 | tee >(grep -c "error" > /tmp/error_count.txt)

# Split a stream for two different processors
sort file.txt > >(head -5 > /tmp/top5.txt) > >(tail -5 > /tmp/bottom5.txt)

Process substitution is Bash-specific (and available in zsh and ksh). It is not POSIX sh. If your scripts must run under /bin/sh, stick to pipes and temporary files. In /usr/bin/env bash scripts, process substitution is fully available.

Check your understanding

Do it yourself

# Heredoc to a file
cat > /tmp/hello.txt <<'EOF'
Hello from a heredoc.
$HOME is not expanded here.
EOF
cat /tmp/hello.txt

# Herestring with read (no subshell)
read -r a b c <<< "one two three"
echo "a=$a b=$b c=$c"

# diff two sorted lists
diff <(echo -e "apple\nbanana\ncherry" | sort) \
     <(echo -e "banana\ncherry\ndate" | sort)

# while read from process substitution
count=0
while IFS= read -r line; do
  count=$((count + 1))
done < <(seq 1 5)
echo "Counted $count lines"   # 5 — visible because no pipe subshell

Where to go next

Heredocs and process substitution make data flow seamless. The next lesson — Traps and signals — builds on trap EXIT from the intermediate tier to handle the full range of Unix signals, clean up robustly, and manage long-running background processes.

Finished reading? Mark it complete to track your progress.

Parameter Expansion

Master all the ${...} forms — default values, error on unset, string length, prefix/suffix stripping, substitution, and case conversion.

Traps and Signals

Register cleanup code with trap EXIT/INT/TERM, understand Unix signals, use kill and wait for process management, and handle subshell signal propagation.