Linux Basics 21: Scheduled Tasks — The Traps Of crontab And systemd Timers

Published 2022-04-29 20:30 Updated 2022-04-29 22:10 1164 words 6 min read ... Page views Handwritten Every word here was typed by me.

Linux Basics 25: Wrapping Up — Self-Audit Checklists And Where To Go DeeperLinux Basics 24: Boot And Rescue — UEFI, GRUB, initramfs And Emergency ModeLinux Basics 23: Containers And Kernel Boundaries — Namespaces, Cgroups And What docker exec DoesLinux Basics 22: Logs And A Debugging Method — Six Steps From Symptom To Root CauseLinux Basics 21: Scheduled Tasks — The Traps Of crontab And systemd TimersLinux Basics 20: Shell Scripting, Part Two — trap, Logging, Idempotence And LockingLinux Basics 19: Shell Scripting, Part One — Quoting, Tests And Exit CodesLinux Basics 18: Performance And Tracing — Where Is The Machine Actually StuckLinux Basics 17: Firewalls — How iptables, nftables, firewalld And ufw RelateLinux Basics 16: SELinux — When Every Permission Is Correct And The Service Is Still DeniedLinux Basics 15: Advanced Permissions — sudoers Wildcards, ACLs And CapabilitiesLinux Basics 14: SSH And Remote Operations — Keys, Permissions And Three Kinds Of TunnelLinux Basics 13: Networking Fundamentals — Troubleshoot From The Bottom UpLinux Basics 12: Disks And Filesystems — From Block Device To Mount PointLinux Basics 11: systemd — units, systemctl And journalctl Done ProperlyLinux Basics 10: Package Management — What apt Actually Does, And The Four Common ErrorsLinux Basics 09: Users And Groups — passwd, shadow And sudoersLinux Basics 08: Environment Variables And Shell Startup Files — Why ssh Cannot Find Your PATHLinux Basics 07: Processes And Signals — The Real Difference Between TERM And KILLLinux Basics 06: Pipelines And Redirection — Three Streams, One Ordering TrapLinux Basics 05: Text Processing — Turning grep/sort/uniq Into Your Analysis ToolkitLinux Basics 04: Permissions And Ownership — rwx Is Not The Same On Files And DirectoriesLinux Basics 03: The Real Semantics Of ls, cp, mv, rm — And The * That The Shell ExpandsLinux Basics 02: Why / Looks Like This, and What An Inode Really StoresLinux Basics 01: Terminals, Shells, and How Commands Are Actually Found
The five cron fields and the OR relationship between day-of-month and day-of-week, cron’s minimal-environment traps (PATH, working directory, % escaping, mailed output, missing variables), crontab versus cron.d, anacron for machines that are often off, and systemd timers with OnCalendar and Persistent.

What This Chapter Solves

The same script that runs perfectly when you type it in a terminal fails silently the moment you put it in crontab — no output, no error, nobody even knows it never ran. Sometimes it also gets skipped entirely, because the machine happened to be powered off at that moment.

This chapter nails down cron's environment traps (the root cause of most "the scheduled job does not work" reports), then looks at its modern replacement, systemd timers, and how they close each of those gaps.

1. The Five Fields

*  *  *  *  *  command
│  │  │  │  └─ day of week (0-7, both 0 and 7 are Sunday; MON..SUN also work)
│  │  │  └──── month       (1-12)
│  │  └─────── day of month(1-31)
│  └────────── hour        (0-23)
└───────────── minute      (0-59)
ExpressionMeaning
*/15 * * * *every 15 minutes
0 3 * * *daily at 03
0 9 * * 1-509
on weekdays
0 0 1 * *00
on the first of each month
@rebootonce at every boot

Bottom line: when both day-of-month and day-of-week are set, cron treats them as OR — 0 0 1 * 5 does not mean "the 1st if it is a Friday", it means "the 1st or every Friday", so both fire. If you want AND, test for it inside the command.

2. The Environment Traps (The Point Of This Chapter)

When cron runs your job, its environment is a completely different one from the shell you log into:

PATH=/usr/local/bin:/usr/bin:/bin
MAILTO=
SHELL=/bin/bash
PATH=/usr/bin:/bin     ← minimal; /usr/local/bin is often not in it
HOME=/root             ← the working directory is your home, not where you cd'd
everything else empty  ← no alias, no .bashrc, no nvm
SymptomCause
command not foundthe directory is not in PATH (see chapter 08 on environments)
relative paths miss filesthe working directory is $HOME, not the script's directory
the command is cut short% is special in crontab and must be written \%
nobody sees the errorcron mails output to MAILTO; set MAILTO= to silence it
$VAR is emptyvariables are not inherited; declare them at the top or source them in the script

The first rule of writing a crontab: use absolute paths, and declare every needed variable at the top. The standard way to debug "why doesn't it run" is to paste the command into a terminal and run it under env -i /bin/bash — reproducing that minimal environment usually makes the problem obvious at once.

3. Managing Cron And Reading Its Logs

crontab -l                     # this user's scheduled jobs
crontab -e                     # edit them
crontab -r                     # delete them all, no confirmation — handle with care
ls /etc/cron.d/                # system level: same format plus a user field
cat /etc/cron.allow            # whitelist; if present, only listed users may use cron
sudo journalctl -u cron -n 50  # cron logs on Debian/Ubuntu

Files in /etc/cron.d/ use the same syntax as a crontab, only with an extra user column, which makes them easy to ship in a package:

30 2 * * * root /usr/local/bin/backup.sh /srv/data

4. anacron: When The Machine Is Often Off

cron assumes the machine is always on — any job whose slot falls during downtime is simply lost. anacron remembers "the last day it ran" and catches up on a daily or weekly basis after boot:

1  5   cron.daily    run-parts /etc/cron.daily
7  10  cron.weekly   run-parts /etc/cron.weekly
FieldMeaning
period (days)run at least once every N days
delay (minutes)wait this long after boot before running, to spread the load

5. systemd Timers: Modern, And Able To Catch Up

[Unit]
Description=Run backup daily

[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
RandomizedDelaySec=10m

[Install]
WantedBy=timers.target
[Unit]
Description=Nightly backup

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh /srv/data
sudo systemctl enable --now backup.timer
systemctl list-timers --all            # when everything fires next
systemctl status backup.timer
sudo journalctl -u backup.service -n 30

Bottom line: three things beat cron here — Persistent=true makes a job missed during downtime run after the next boot (exactly anacron's job), RandomizedDelaySec spreads jobs so a fleet does not hit one server at the same second, and logs land in journald automatically.

Note: I first wrote OnCalendar as 03:00 without seconds; systemd parses it, but the catch-up behaviour is clearer with the full *-*-* 03:00:00. systemctl list-timers is the single most useful command — it tells you exactly when the job runs next.

6. logrotate

Left alone, logs will fill the disk. logrotate splits, compresses and retains N copies, then asks the service to reopen its file:

/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    postrotate
        systemctl reload myapp >/dev/null 2>&1 || true
    endscript
}

It is usually driven by /etc/cron.daily/logrotate or a systemd timer. The postrotate block tells the service to reopen its log file, otherwise it keeps writing to the old inode that has been renamed away.

7. Hands-On Checklist

crontab -l                                               # 1. existing jobs
printf '#!/bin/sh\ndate >> ~/cron.log\n' | tee ~/bin/t.sh >/dev/null; chmod +x ~/bin/t.sh   # 2. make a script
env -i /bin/bash -c '~/bin/t.sh'                         # 3. reproduce cron’s minimal environment
systemd-analyze calendar '*-*-* 03:00:00'                # 4. validate an OnCalendar expression
systemctl list-timers --all | head                       # 5. every timer on the system
sudo journalctl -u cron -n 20 2>/dev/null || sudo journalctl -u crond -n 20   # 6. cron logs
sudo logrotate -d /etc/logrotate.d/myapp                 # 7. dry run, changes nothing

Summary

  • In the five cron fields, day-of-month and day-of-week together mean OR; both 0 and 7 are Sunday.
  • Environment traps are cron's number one problem: minimal PATH, $HOME as working directory, variables not inherited, % needing an escape, output sent by mail.
  • crontab -r wipes jobs without asking — be careful; system jobs go in /etc/cron.d/ with the extra user field.
  • For machines that are often off, use anacron or a systemd timer with Persistent=true to catch up.
  • A systemd timer adds catch-up, randomised jitter and journald logging; systemctl list-timers shows the next run.
  • Keep logs pruned with logrotate rather than waiting for a full disk.

Next Up

Logs and a debugging method: how journald and /var/log divide the work, a six-step path from symptom to root cause, and three real cases — a full disk, a taken port, and a permission denial.

Written by Handwritten

If you enjoyed this, leave a comment~

Loading...