The Cron Job That Only Works When a Human Watches

cron does not run a login shell. It executes the command with SHELL=/bin/sh, PATH=/usr/bin:/bin, HOME set to the crontab owner's home, no controlling terminal, and the working directory set to that home. Everything the interactive shell picked up from /etc/profile, ~/.bashrc, and the venv activation is simply absent.

The durable fix is to make the script self-sufficient, not to teach the crontab about your shell:

#!/usr/bin/env bash
set -euo pipefail
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export LC_ALL=C
/opt/reports/venv/bin/python3 /opt/reports/sync.py --in /srv/inbox --out /srv/out

Calling the interpreter by absolute path removes the need to activate anything. If values genuinely belong to the schedule, put them at the top of the crontab (PATH=, SHELL=/bin/bash, [email protected]), never only in a profile file.

The empty-CSV symptom is a second, separate bug, not a PATH bug - look at locale and client environment: LC_ALL/LANG can change sort order or decimal separators used in filtering, and database clients read PGPASSFILE, PGHOST, and HOME, which cron sets to /root. Inspect the destination path for typos and confirm the script writes to the same place in both modes.

Reproduce cron's environment to prove the fix:

env -i /bin/sh -c '/usr/local/bin/sync_reports.sh'

Add set -x with timestamps while debugging, then remove it.

Mutual exclusion belongs in the script so it holds no matter who launches it:

#!/usr/bin/env bash
set -euo pipefail
exec 9>/var/lock/sync_reports.lock
flock -n 9 || { echo "$(date -Is) previous run still active, skipping"; exit 0; }

An flock -n wrapper in the crontab line is equivalent and simpler, but easy to lose when someone copies the command elsewhere.

Variant choice:

• flock -n (fail fast, skip this tick). Right answer here: a 20-40 minute job on a 15-minute schedule will never catch up, so waiting queues runs and worsens the backlog. Log the skip and alert if skips exceed a threshold - that signal means the schedule is wrong, not the code.
• flock -w 600 (wait). Only sensible if the backlog genuinely drains; otherwise it just delays the collision.

Operational caveats to mention:

• Keep the lock file on local disk. flock over NFS relies on lockd and degrades to best-effort; a stale lock or a hung peer can wedge the schedule.
• A lock does not repair the data already duplicated. Add an idempotent write on the database side (unique key plus upsert, or delete-and-insert keyed on report date) as defence in depth - correct code plus a lock beats a lock alone.
• The 20-40 minute runtime versus a 15-minute schedule is itself the finding: either move to */30 or make the job incremental.

The % problem: in a crontab, every unescaped % is converted to a newline before the shell sees it, and everything after the first one is fed to the command as standard input. That is why ... --tag $(date +%Y-%m-%d) truncates at %Y and feeds -m-%d) to the process. Escape as \%, or better, keep format strings inside the script where cron's parsing rules do not apply - the second option is what I would push for, because the escape is invisible in review and gets lost on the next copy-paste.

Missed runs: cron cannot do this. A @reboot entry fires on every boot, including boots where nothing was missed, which duplicates work. Use a systemd timer over the existing unit:

# sync-reports.timer
[Timer]
OnCalendar=*:0/15
Persistent=true
AccuracySec=1min
RandomizedDelaySec=30
Unit=sync-reports.service

Persistent=true records the last trigger time on disk and fires one catch-up run if the timer was inactive when it should have run - it triggers once, not once per missed interval, which is what you want. AccuracySec and RandomizedDelaySec stop every host waking at exactly :00 and hammering the database.

Migration notes: move the redirections and environment into the service unit (StandardOutput=append:/var/log/sync.log), keep the flock guard inside the script, and systemctl disable --now the old crontab entry in the same change so the timer does not run alongside it.