Heredocs and Process Substitution in Bash
Two bash syntaxes remove most of the reasons to write temporary files.
Heredocs
Passing multi-line text to a command’s standard input:
cat <<EOF
Hello $USER
Today is $(date +%A)
EOF
Variables and command substitutions are expanded. Quote the delimiter to stop that:
cat <<'EOF'
This $VARIABLE stays literal
So does $(this)
EOF
Quote the delimiter whenever you are writing a config file. Config files are full of dollar signs that are not shell variables, and an unquoted heredoc silently replaces them with nothing.
sudo tee /etc/nginx/conf.d/app.conf > /dev/null <<'EOF'
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header X-Real-IP $remote_addr;
}
EOF
Without the quotes, $remote_addr becomes empty and the config is broken in a way that is easy to miss.
The sudo tee pattern is covered in our tee guide, and it exists because a plain redirect happens as your user before sudo runs.
Indentation
if true; then
cat <<-'EOF'
This is indented in the script
but not in the output
EOF
fi
The - strips leading tabs. Not spaces. An editor set to expand tabs to spaces breaks this silently, which is why indented heredocs so often do not work as expected.
Herestrings
For a single line:
grep pattern <<< "$variable"
jq '.name' <<< "$json"
Shorter than echo "$var" | grep pattern and avoids a process.
Process substitution
Presents a command’s output where a filename is expected:
diff <(sort file1) <(sort file2)
The shell runs each command and substitutes a path like /dev/fd/63. diff opens it as a normal file and never knows the difference.
A pipe cannot do this, because a pipe only feeds standard input and diff needs two file arguments.
Where it earns its place
# Compare command output from two machines
diff <(ssh host1 'rpm -qa | sort') <(ssh host2 'rpm -qa | sort')
# Set comparison without temporary files
comm -23 <(sort a.txt) <(sort b.txt)
# Feed a config to a command that wants a file
mytool --config <(envsubst < template.conf)
# Read into a loop without a subshell
while read -r line; do
count=$((count + 1))
done < <(some-command)
That last one matters more than it looks. This does not work:
some-command | while read -r line; do
count=$((count + 1))
done
echo "$count" # empty, the loop ran in a subshell
The pipe puts the loop in a subshell, so variables set inside it are lost when it exits. Process substitution with < <(...) keeps the loop in the current shell and the variable survives. This is one of the most confusing behaviours in bash and process substitution is the standard fix.
Note the space between < and <(. Without it the syntax is wrong.
Output substitution
It works in the other direction too:
command | tee >(gzip > out.gz) >(wc -l > count.txt) > /dev/null
Sends the same stream to several consumers, each of which thinks it is writing to a file.
Our comm and diff guides cover the comparison tools this pairs with.
Portability
Heredocs are POSIX and work everywhere.
Process substitution is not. It is a bash, zsh, and ksh feature. A script with #!/bin/sh will fail on it, and on Debian and Ubuntu /bin/sh is dash, which does not support it.
Use #!/usr/bin/env bash if you rely on it, which our bash error handling guide recommends anyway.
Frequently Asked Questions
What is a heredoc?
A way to pass multi-line text to a command’s standard input directly in the script, ending at a delimiter you choose. It replaces the pattern of writing a temporary file, using it, and deleting it.
Why should I quote the heredoc delimiter?
Quoting it stops the shell expanding variables, command substitutions, and backticks inside the body. That is what you want when writing a config file that legitimately contains dollar signs, and forgetting it silently mangles the output.
What does the dash in the heredoc operator do?
It strips leading tab characters from each line, so the body can be indented to match the surrounding code. It strips tabs only, not spaces, which is why indented heredocs often fail in editors configured to expand tabs.
What is process substitution?
Syntax that runs a command and presents its output as a filename, so tools expecting a file can read from a command. The shell creates a file descriptor path such as /dev/fd/63 and passes that as the argument.
Why can I not use a pipe instead of process substitution?
A pipe connects standard input, so it only works for one input and only when the command reads stdin. Process substitution provides a path, which means you can supply several command outputs to a program that takes multiple file arguments.
Is process substitution portable?
No. It is a bash, zsh, and ksh feature and is not in POSIX sh. Scripts with a sh shebang will fail on it, so use a bash shebang or fall back to temporary files.