DevOps Engineering: Scripting, Automation, and Tooling

This domain tests whether your automation fails loudly, safely and repeatably: shell semantics under set -e, exit codes in pipelines, retry behaviour against APIs, and portable tooling choices that hold up in CI.

Core rules

• Bash header: set -euo pipefail plus IFS=$'\n\t'. -e exits on an unhandled failure, -u errors on unset variables, and pipefail makes a pipeline fail if any stage fails - without it, curl … | grep returns grep's status and hides the fetch failure.
• Quote every expansion ("$var", "${arr[@]}"); prefer [[ ]] over [ ]; use $(...) over backticks; never parse ls - use globs or find -print0 | xargs -0.
• Clean up with traps: trap 'rm -f "$tmp"' EXIT. Handle signals. Make scripts idempotent so a rerun is harmless.
• Exit codes are the CI contract: 0 is success, non-zero failure; distinguish retryable from fatal outcomes.
• Python: subprocess.run([...], check=True) without shell=True; pinned dependencies; argparse for parameters; log to stderr; explicit timeouts on every network call.
• Retries: only transient errors (429, 5xx, timeouts), exponential backoff with jitter, capped attempts, honour Retry-After.
• Lint automation with shellcheck, shfmt and pre-commit hooks so defects surface before CI.

Worked example

Script line: set -eu; curl -fsS "$URL" | grep -q '"status":"ok"' && echo healthy. Reasoning: pipefail is missing, so when curl fails, grep reads empty input and exits 1 - but because the pipeline sits on the left of &&, set -e does not abort the script. Execution continues and later steps act on stale state. -f correctly makes curl fail silently on HTTP 4xx/5xx, but only pipefail propagates that failure out of the pipeline. Fix: add -o pipefail, replace the && chain with an explicit if block, and add --max-time and --retry so the step cannot hang the runner indefinitely.

Traps

• Assuming set -e applies inside if conditions, &&/|| chains and some command substitutions - it does not.
• Retrying every error; retrying a 400 or a permission failure just multiplies load.
• Unpinned latest container tags or action versions in automation, so behaviour changes without a commit.
• Writing state-mutating scripts without a lock, so two concurrent CI jobs interleave.