Compression and Bandwidth
Two factors dominate rsync transfer performance: compression (shrinking data before it goes over the wire) and bandwidth control (preventing rsync from saturating your network). Getting these right can mean the difference between a 10-minute backup and a 2-hour one.
Compression (-z)
How It Works
When you enable compression with -z, rsync compresses file data before sending it over the network, then decompresses on the receiving end. This trades CPU time for reduced network traffic.
# With compression (data compressed in transit)
rsync -avz /var/www/html/ user@remote:/backup/www/
# Without compression (raw data sent)
rsync -av /var/www/html/ user@remote:/backup/www/
When to Use Compression
| Scenario | Use -z? | Why |
|---|---|---|
| Remote transfer over internet | Yes | Reduces data sent over slow links |
| Transfer over LAN (1 Gbps+) | No | Network is faster than compression overhead |
| Text-heavy files (code, SQL, configs, logs) | Yes | Text compresses 5–10x |
Already-compressed files (.jpg, .mp4, .gz, .zip) | No | Can't compress further; wastes CPU |
| Mixed content (code + images) | Maybe | Compression helps some files, not others |
Compression Levels
Fine-tune the CPU vs bandwidth trade-off:
# Low compression (fast, less CPU, less effective)
rsync -avz --compress-level=1 /var/www/ user@remote:/backup/
# Default compression (level 6 — good balance)
rsync -avz /var/www/ user@remote:/backup/
# Maximum compression (slow, most CPU, smallest transfer)
rsync -avz --compress-level=9 /var/www/ user@remote:/backup/
| Level | CPU Usage | Compression Ratio | Best For |
|---|---|---|---|
| 1 | Low | Minimal | Fast networks where you want slight shrinkage |
| 6 (default) | Medium | Good | Most remote transfers |
| 9 | High | Maximum | Very slow connections where bandwidth is precious |
Skip Compression for Specific File Types
Rsync automatically skips compression for files that are already compressed. You can customize this list:
# Rsync's default skip list includes: gz, zip, jpg, mp4, etc.
# Add custom skip patterns:
rsync -avz --skip-compress=gz/jpg/mp4/png/zip/rar/pdf \
/var/www/ user@remote:/backup/
Bandwidth Limiting (--bwlimit)
Why Limit Bandwidth?
Running rsync at full speed on a production server can:
- Slow down your website for visitors
- Saturate shared hosting bandwidth
- Trigger rate limiting from cloud providers
- Overwhelm VPS network interfaces
How to Limit
# Limit to 5 MB/s (5000 KB/s)
rsync -avz --bwlimit=5000 /var/www/ user@remote:/backup/
# Limit to 1 MB/s (conservative for production hours)
rsync -avz --bwlimit=1000 /var/www/ user@remote:/backup/
# Limit to 10 MB/s (generous for off-hours)
rsync -avz --bwlimit=10000 /var/www/ user@remote:/backup/
Bandwidth Recommendations
| Scenario | Recommended Limit | Reasoning |
|---|---|---|
| Production VPS during business hours | 1–5 MB/s | Leave bandwidth for visitors |
| Production VPS off-hours | 10–50 MB/s | Site has minimal traffic |
| Dedicated server | 20–100 MB/s | More capacity available |
| LAN transfer | No limit | Network isn't shared |
| Metered connection | 1–2 MB/s | Avoid overage charges |
Scheduling Strategy
Run bandwidth-limited backups during business hours and unlimited backups at night:
# Cron: 2 AM unlimited backup
0 2 * * * rsync -avz /var/www/ backup-server:/backups/www/
# Cron: 2 PM throttled sync for critical changes
0 14 * * * rsync -avz --bwlimit=2000 /var/www/ backup-server:/backups/www/
Delta Transfer vs Whole File
Rsync uses delta transfer by default for remote syncs — sending only the changed parts of files. For local syncs, it uses whole file mode.
| Mode | How It Works | Best For |
|---|---|---|
| Delta transfer (default for remote) | Compares checksums, transfers only changed blocks | Remote/WAN transfers |
Whole file (--whole-file) | Copies full files regardless of changes | Fast LAN, or when checksum computation is slower than copying |
# Force whole-file mode (faster on fast LANs)
rsync -av --whole-file /var/www/ /nas/backup/www/
# Force delta transfer (rare — it's default for remote)
rsync -av --no-whole-file /var/www/ /nas/backup/www/
Monitoring Transfer Performance
Progress Display
# Per-file progress bar
rsync -avzP /var/www/ user@remote:/backup/
# Human-readable file sizes
rsync -avzhP /var/www/ user@remote:/backup/
Post-Transfer Statistics
rsync -avz --stats /var/www/ user@remote:/backup/
Example output:
Number of files: 45,832
Total file size: 12.4 GB
Total transferred file size: 340 MB
Total bytes sent: 285 MB
Transfer speed: 15.2 MB/s
Speedup: 36.47
Key metrics to watch:
- Speedup — how much data rsync avoided sending (higher = more efficient)
- Total transferred file size vs Total file size — delta transfer savings
Logging for Automated Jobs
rsync -avz --stats --log-file=/var/log/rsync-backup.log \
/var/www/ user@backup:/backups/www/
Optimizing for Different Network Conditions
Slow Internet (< 10 Mbps)
rsync -avz --compress-level=9 --bwlimit=500 -P \
/var/www/ user@remote:/backup/
- Maximum compression to minimize data
- Low bandwidth limit to prevent saturation
- Progress + resume for large transfers
Fast Internet (100+ Mbps)
rsync -avz --compress-level=1 -P \
/var/www/ user@remote:/backup/
- Light compression (CPU is the bottleneck, not network)
- No bandwidth limit needed
Fast LAN (1 Gbps+)
rsync -av --whole-file /var/www/ /nas/backup/
- No compression (network is faster than CPU)
- Whole-file mode (reading + writing is faster than checksumming)
Unstable Connection
rsync -avz -P --partial-dir=.rsync-partial --timeout=60 \
/var/www/ user@remote:/backup/
-Penables resume--partial-dirstores partial files in a hidden directory--timeoutprevents hanging on dropped connections
Common Pitfalls
| Pitfall | Impact | Prevention |
|---|---|---|
Using -z on LAN transfers | Wastes CPU, slows transfer | Skip compression for fast networks |
Using -z on already-compressed files | No benefit + CPU overhead | Rsync auto-skips most, but check |
No --bwlimit on production servers | Website slows down during backup | Always limit during business hours |
--bwlimit too aggressive | Backup never finishes before next run | Ensure backup completes in its window |
Not using -P for large transfers | Can't resume after interruption | Always use -P for GB+ transfers |
Ignoring --stats output | No visibility into transfer efficiency | Enable stats for monitoring |
Quick Reference
# Standard remote backup (with compression)
rsync -avzP /var/www/ user@remote:/backup/
# Bandwidth-limited production backup
rsync -avz --bwlimit=5000 /var/www/ user@remote:/backup/
# Maximum compression (slow connection)
rsync -avz --compress-level=9 -P /var/www/ user@remote:/backup/
# Fast LAN transfer (no compression)
rsync -av --whole-file /var/www/ /nas/backup/
# View transfer statistics
rsync -avz --stats /var/www/ user@remote:/backup/
# Log to file for automated jobs
rsync -avz --log-file=/var/log/rsync.log /var/www/ remote:/backup/
What's Next
- Parallel and Incremental Sync — Scale rsync for larger datasets
- Options and Flags — Complete flag reference
- Cron Automation — Schedule optimized backups