Logging and Monitoring

Running rsync without logging is like running backups with your eyes closed — you won't know if something failed until you need to restore. Proper logging provides audit trails, performance insight, and failure detection for every rsync operation.

Real-Time Progress

Per-File Progress

# Show progress bar for each file being transferred
rsync -avP /var/www/html/ user@backup:/backups/www/

The -P flag combines --progress (show transfer speed per file) and --partial (keep partially transferred files for resume).

Output example:

index.php
 4,096 100% 3.91MB/s 0:00:00 (xfr#1, to-chk=1234/1500)
styles.css
 12,345 67% 11.73MB/s 0:00:01

Human-Readable Sizes

rsync -avhP /var/www/html/ user@backup:/backups/www/

The -h flag converts bytes to KB/MB/GB for easier reading.

Transfer Statistics

--stats Flag

Get a summary report after the transfer:

rsync -avz --stats /var/www/html/ user@backup:/backups/www/

Output:

Number of files: 45,832 (reg: 38,291, dir: 7,541)
Number of created files: 23
Number of deleted files: 0
Number of regular files transferred: 156
Total file size: 12.47G bytes
Total transferred file size: 34.21M bytes
Literal data: 33.89M bytes
Matched data: 0 bytes
File list size: 1.23M
Total bytes sent: 35.21M
Total bytes received: 12.3K

sent 35.21M bytes received 12.3K bytes 4.70M bytes/sec
total size is 12.47G speedup is 354.12

Key metrics to monitor:

Metric	What It Tells You
Number of regular files transferred	How many files actually changed
Total file size vs Transferred size	Delta transfer efficiency
Speedup	How much data rsync avoided sending (higher = better)
Number of deleted files	When using --delete — verify expected deletions
Total bytes sent	Actual network usage

Log to File

Basic File Logging

rsync -av --log-file=/var/log/rsync/backup.log \
 /var/www/html/ user@backup:/backups/www/

Dated Log Files

Give each run its own log file:

rsync -av --log-file="/var/log/rsync/backup_$(date +%F_%H%M).log" \
 --stats \
 /var/www/html/ user@backup:/backups/www/

Combine Stats + Log

rsync -avz --stats --log-file=/var/log/rsync/backup.log \
 /var/www/html/ user@backup:/backups/www/

Itemized Changes (--itemize-changes)

Get detailed information about what changed for each file:

rsync -av --itemize-changes /var/www/html/ /backup/www/

Output format:

>f.st...... index.php # File transferred, size and time changed
>f..t...... config.yaml # File transferred, time changed only
cd+++++++++ new-directory/ # New directory created
*deleting old-file.txt # File deleted from destination

Decoding the Flags

>f.st......
│││││
│││││
│││└┴── Other attributes (permissions, owner, group, etc.)
││└──── Size changed
│└───── Timestamp changed
└────── f=file, d=directory, L=symlink

Symbol	Meaning
>f	File being sent to destination
cd	Directory being created
.	Attribute unchanged
s	Size differs
t	Timestamp differs
p	Permissions differ
o	Owner differs
g	Group differs

Logging in Automated Scripts

Production Backup Script with Logging

#!/bin/bash
# backup-with-logging.sh
set -e

TIMESTAMP=$(date +%F_%H%M)
LOG_DIR="/var/log/rsync"
LOG_FILE="$LOG_DIR/backup_$TIMESTAMP.log"
ALERT_EMAIL="admin@example.com"

mkdir -p "$LOG_DIR"

echo "[$TIMESTAMP] Starting backup..." >> "$LOG_FILE"

# Run rsync with stats and logging
if rsync -avz --stats \
 --exclude='cache/' \
 --exclude='*.log' \
 --log-file="$LOG_FILE" \
 /var/www/html/ user@backup:/backups/www/ \
 >> "$LOG_FILE" 2>&1; then
 
 echo "[$TIMESTAMP] Backup completed successfully" >> "$LOG_FILE"
else
 EXIT_CODE=$?
 echo "[$TIMESTAMP] Backup FAILED (exit code: $EXIT_CODE)" >> "$LOG_FILE"
 
 # Send alert on failure
 echo "Rsync backup failed. Check $LOG_FILE" | \
 mail -s " Backup Failed on $(hostname)" "$ALERT_EMAIL"
fi

Cron with Logging

# Daily backup at 2 AM with logging
0 2 * * * /usr/local/bin/backup-with-logging.sh

# Weekly full backup with separate log
0 3 * * 0 rsync -avz --stats --log-file="/var/log/rsync/weekly_$(date +\%F).log" \
 /var/www/ user@backup:/backups/weekly/

Log Rotation

Prevent logs from filling up disk space:

/etc/logrotate.d/rsync

/var/log/rsync/*.log {
 daily
 rotate 30
 compress
 delaycompress
 missingok
 notifempty
 create 644 root root
}

Or use a simple cleanup in your backup script:

# Delete rsync logs older than 30 days
find /var/log/rsync/ -name "*.log" -mtime +30 -delete

Analyzing Logs

Check for Errors

# Find errors in logs
grep -i "error\|failed\|denied" /var/log/rsync/backup.log

# Check what was deleted
grep "deleting" /var/log/rsync/backup.log

# Count files transferred
grep "^>f" /var/log/rsync/backup.log | wc -l

Monitor in Real Time

# Watch log as backup runs
tail -f /var/log/rsync/backup.log

# Watch for errors only
tail -f /var/log/rsync/backup.log | grep -i "error\|failed"

Common Pitfalls

Pitfall	Consequence	Prevention
No logging on cron jobs	Failures go unnoticed for days/weeks	Always use --log-file in automated runs
Logs in /var/www/	Publicly accessible, potential data leak	Store logs in /var/log/rsync/
No log rotation	Logs fill the disk	Use logrotate or find -mtime cleanup
Not checking exit codes	Script reports success even on failure	Check $? and alert on non-zero exit
Only using --progress (no persistent log)	No record after terminal closes	Use --log-file for persistent records
Not monitoring --stats output	Inefficient transfers go unnoticed	Review speedup ratio periodically

Quick Reference

# Real-time progress
rsync -avhP /src/ user@dest:/dest/

# Transfer statistics
rsync -avz --stats /src/ user@dest:/dest/

# Log to file
rsync -av --log-file=/var/log/rsync/backup.log /src/ user@dest:/dest/

# Itemized changes (detailed diff)
rsync -av --itemize-changes /src/ /dest/

# Combination: stats + log + progress
rsync -avzhP --stats --log-file=/var/log/rsync/backup.log /src/ user@dest:/dest/

# Analyze logs
grep -i "error\|failed" /var/log/rsync/backup.log
grep "deleting" /var/log/rsync/backup.log

What's Next

• Dry Run and Testing — Validate commands before execution
• Cron Automation — Schedule monitored backups
• Troubleshooting — Fix common rsync errors