How to Monitor Cron Jobs and Scheduled Tasks

Cron jobs are the hidden workhorses of most web applications — sending emails, processing payments, generating reports, syncing data, running backups. And they fail silently.

When a web page breaks, users see an error. When a cron job stops running, nothing visible happens. Your backups quietly stop being made. Your nightly invoice emails stop going out. Your data sync falls 3 days behind. You discover this when something catastrophic happens — and by then the damage is done.

Cron job monitoring (also called heartbeat monitoring for scheduled tasks) solves this by detecting when a job stops running and alerting you immediately.

Why Cron Jobs Fail Silently

Unlike web requests that return errors and trigger application logging, a failed cron job typically produces:

• No HTTP error
• No visible change to users
• Only a log entry (if you're checking logs)
• Possibly nothing at all (if the process silently exits)

Common reasons cron jobs stop:

• The cron process itself crashes
• A code change breaks the job's logic
• A database or external service the job depends on becomes unavailable
• The job takes longer than expected and the next invocation starts before the previous one finishes
• Disk space runs out, preventing the job from completing
• Permission changes break the script's access to required resources
• Deployment overwrites the job configuration

How Heartbeat Monitoring Works

Heartbeat monitoring flips the detection model:

1. You configure an expected check-in interval (e.g., every 5 minutes)
2. Your cron job sends a "ping" to a monitoring URL at the end of each successful run
3. The monitoring service tracks these pings
4. If no ping arrives within the expected window (e.g., 7 minutes), an alert fires

This is fundamentally different from HTTP uptime monitoring — instead of the monitoring service calling your application, your application calls the monitoring service.

# Example: Add this to the end of your cron job script
curl -s "https://your-heartbeat-url/ping/your-monitor-token" > /dev/null

If the job fails before reaching this line, or never runs at all, no ping is sent, and you get alerted.

Setting Up Cron Job Monitoring

Step 1: Create a Heartbeat Monitor

In Domain Monitor, create a heartbeat monitor and configure:

• Expected interval: How often the job should run (e.g., every 5 minutes, every hour, every day)
• Grace period: How long to wait after the expected time before alerting (e.g., 2 minutes for a 5-minute job, to allow for slight execution delays)
• Alert channels: Who gets notified when the job misses its schedule

Step 2: Add the Ping to Your Job

For a shell script:

#!/bin/bash
# Your cron job logic here
/usr/bin/python3 /path/to/your/script.py

# Ping heartbeat monitor on success
if [ $? -eq 0 ]; then
    curl -s "https://monitoring-url/ping/TOKEN" > /dev/null
fi

For a Python script:

import requests

def run_job():
    # Your job logic
    process_invoices()
    sync_data()

if __name__ == "__main__":
    run_job()
    # Signal successful completion
    requests.get("https://monitoring-url/ping/TOKEN", timeout=5)

For Node.js:

const https = require('https');

async function runJob() {
    // Your job logic
    await processQueue();
    
    // Signal success
    https.get('https://monitoring-url/ping/TOKEN');
}

runJob().catch(console.error);

Step 3: Configure Your Crontab

# Example: run every 5 minutes
*/5 * * * * /path/to/your/job.sh

# Example: run daily at 2am
0 2 * * * /path/to/daily-report.py

The monitoring interval should match your cron schedule, with a grace period accounting for execution time.

What to Monitor with Heartbeats

High priority — monitor immediately:

• Backup jobs (critical for disaster recovery)
• Payment processing jobs (billing depends on this)
• SSL certificate renewal (prevents outages from expired certs)
• User notification emails (customers expect to receive them)
• Data synchronisation jobs (business decisions depend on current data)

Medium priority:

• Report generation
• Analytics processing
• Cache warming
• Log rotation

Lower priority (still worth monitoring):

• Cleanup jobs
• Non-critical data imports
• Development environment refreshes

Detecting Slow Jobs

In addition to detecting failed jobs, heartbeat monitoring can detect when jobs run too slowly. If a job that normally completes in 30 seconds starts taking 5 minutes, this indicates a problem (database performance, external API slowness) even if the job ultimately succeeds.

Configure the grace period to match expected execution time. A job that usually takes 1 minute should have a grace period of 3-4 minutes — giving it reasonable time to complete while still alerting on significant delays.

Integrating with Your Deployment Process

After every deployment, verify your cron jobs are still running correctly. A common deployment mistake is overwriting crontab entries, changing file paths, or breaking environment variable access.

Add a post-deployment check to your runbook: "Verify last heartbeat received from critical jobs within the past [schedule interval]."

Cron Monitoring vs. Process Monitoring

Heartbeat monitoring detects when a job doesn't run. Process monitoring detects when a process crashes.

For long-running background workers (as opposed to scheduled cron jobs), combine:

• Process management (systemd, supervisor, PM2) to automatically restart crashed workers
• External HTTP monitoring on a health endpoint the worker exposes
• Heartbeat monitoring for the work the worker performs

The heartbeat monitoring guide covers this broader context.

Set up heartbeat monitoring for your cron jobs at Domain Monitor — know the moment a scheduled task stops running.

Related Articles

• "Monitoring Background Jobs and Scheduled Tasks"
• Custom HTTP monitor statuses (for our Pro Plan users)
• "How to Monitor AWS EC2 Instance Uptime"