CTLA: Shell Scripting and Command-Line Tools

This domain checks whether you can build a script that behaves correctly under real input - spaces in filenames, failing commands, empty variables - and whether you can pick the right text-processing tool rather than hand-rolling loops. Expect questions about quoting, exit status propagation, redirection order, and safe use of find/xargs/awk/sed.

Quoting and expansion

Single quotes are fully literal; double quotes still expand $var, $(...), and backticks, but prevent word splitting.
"$@" passes each argument as its own word; unquoted $@ re-splits on IFS; "$*" joins arguments with the first IFS character.
Unquoted variable expansion undergoes word splitting and globbing - the single most common scripting bug.

Exit status and control flow

$? is 0 for success. In a pipeline, the status is that of the last command unless set -o pipefail is set.
set -euo pipefail is the defensive default: exit on error, error on unset variables, propagate pipeline failures.
trap 'cleanup' EXIT runs regardless of how the script ends.
[ ] is POSIX test and splits words; [[ ]] is a bash keyword with pattern matching and safe empty-variable handling.

Redirection order matters

cmd > out 2>&1 sends both streams to the file. cmd 2>&1 > out sends stderr to the original stdout (the terminal) and only stdout to the file, because redirections are evaluated left to right.

Text toolkit

awk -F, '{print $3}' for field extraction, sed 's/old/new/g' for substitution, grep -E for extended regex, find ... -print0 | xargs -0 to survive arbitrary filenames.

Worked example

#!/usr/bin/env bash
set -euo pipefail
for f in $(find /data -name '*.csv'); do
  awk -F, '{print $3}' "$f" > "$f.col"
done

Two defects. First, command substitution re-splits on whitespace, so a path like /data/q1 sales.csv becomes two iterations. Second, a failure inside the for list does not abort under set -e the way most people assume. Rewrite:

find /data -name '*.csv' -print0 |
  while IFS= read -r -d '' f; do awk -F, '{print $3}' "$f" > "$f.col"; done

-print0 plus read -d '' preserves exact names; IFS= read -r prevents trimming and backslash mangling.

Traps

[ $var = x ] errors when var is empty - quote it or use [[ ]].
Assuming set -e catches failures in if, &&, or command substitution.
Using xargs without -0 on filenames containing spaces.
Forgetting that variables set in a while loop fed by a pipe live in a subshell.