Report job misses its window when the VM is powered off
A reporting VM runs Ubuntu 22.04 and must write /srv/reports/daily.csv before 06:00 Europe/Berlin. The job is defined in /etc/cron.d/report as 0 1 * * * report /opt/report/run.sh. The hypervisor powers the VM off from Friday 23:00 to Monday 05:00; the VM is also suspended during a Wednesday maintenance window from 00:30 to 03:00. On Monday the CSV is missing; on Thursday it contains stale data because run.sh calculates the report date with date -d yesterday when it finally runs. timedatectl reports System clock synchronized: no and NTP service: inactive; the hardware clock has drifted 11 minutes fast over the previous month. /var/log/syslog has no cron entries for the outage periods, and logrotate has already rotated the older cron logs. The team wants a job that catches up once after boot, never runs twice for the same date, and completes before the 06:00 deadline.
1. Explain why cron produced no run and no error for the missed Monday and Wednesday windows, and what evidence you would collect before changing anything.
Read the worked answer
cron is an event scheduler, not a catch-up scheduler. When the VM is powered off or suspended, there is no cron daemon to notice 01:00, so no job starts and no error is generated. A syntax error or a failing command would log something; a missed wall-clock instant does not. The gap in /var/log/syslog is expected, not evidence of deletion. Collect journalctl -u cron --since '7 days ago', journalctl --list-boots, last -x, uptime -s, and the report script's own log. Confirm the VM was down across the window and that the cron file is still present and not ignored because of wrong ownership or mode in /etc/cron.d. Use systemctl status cron and journalctl -u cron to distinguish daemon inactivity from a parse error. Also inspect /etc/cron.d/report with stat and check for a missing newline at the end of the file, which can make cron ignore the last line. Preserve this evidence before migrating the job.
2. Write systemd service and timer units that replace the cron job. The timer must run at 01:00 Berlin time, catch up after a missed run, avoid duplicate same-day execution, and log failures to the journal. Give activation commands.
Read the worked answer
Create /etc/systemd/system/report.service:
[Unit]
Description=Daily report
After=network-online.target time-sync.target
Wants=network-online.target
[Service]
Type=oneshot
User=report
Group=report
ExecStart=/usr/bin/flock -n /run/report.lock /opt/report/run.sh
Nice=10
Create /etc/systemd/system/report.timer:
[Unit]
Description=Daily report timer
[Timer]
OnCalendar=*-*-* 01:00:00 Europe/Berlin
Persistent=true
AccuracySec=1min
RandomizedDelaySec=0
Unit=report.service
[Install]
WantedBy=timers.target
Activate and verify:
sudo systemctl daemon-reload
sudo systemctl enable --now report.timer
systemctl list-timers --all report.timer
sudo systemctl start report.service
journalctl -u report.service -n 50
Reasoning: Persistent=true stores the last trigger in /var/lib/systemd/timers and runs once after boot if the scheduled time was missed. It does not replay every missed day, so the script must generate all missing dates or the latest date idempotently. flock plus an output-file check prevents a manual run and the timer run from duplicating work. AccuracySec=1min keeps the run inside the deadline without spreading it unpredictably. After=time-sync.target is useful only if systemd-time-wait-sync.service is enabled; otherwise verify the clock separately.
3. The clock drift means the 01:00 trigger can fire at the wrong wall-clock time. Which time sync service would you enable and how would you verify it is working, given outbound UDP 123 is allowed only to 10.20.0.10 and 10.20.0.11? Why not just run ntpdate from cron?
Read the worked answer
Use chrony. Install chrony, set:
server 10.20.0.10 iburst
server 10.20.0.11 iburst
makestep 1.0 3
in /etc/chrony/chrony.conf, then systemctl enable --now chrony and disable systemd-timesyncd if it is active, since two NTP clients conflict. iburst gets fast initial samples; makestep 1.0 3 steps the clock only for the first three updates if offset exceeds one second, correcting the 11-minute drift instead of slewing for hours. Verify with chronyc sources -v (both servers reachable, one selected), chronyc tracking (offset, stratum, leap normal), and timedatectl (synchronized yes). If using systemd-timesyncd instead, set NTP=10.20.0.10 10.20.0.11 in /etc/systemd/timesyncd.conf, timedatectl set-ntp true, and check timedatectl timesync-status. Do not run ntpdate from cron: it is deprecated, steps the clock abruptly, does not discipline drift continuously, and can race with a running NTP daemon. Also confirm UDP 123 is permitted only to those two addresses and that the firewall allows replies.