🧾 Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

• Logs are the heartbeat of your servers 💓 — but if left unmanaged, they'll crash your systems.
• In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

---

📑 Table of Contents

• 1️⃣ Log Cleanup Scripts
• 2️⃣ Apache & Middleware Examples
• 3️⃣ Docker & K8s Cleanup
• 4️⃣ logrotate Configuration
• 5️⃣ How to Run & Test logrotate
• 🏁 Summary

---

1️⃣ Log Cleanup Scripts

#!/bin/bash
# Delete logs older than 7 days for Tomcat & Jenkins
find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
echo "🧹 Tomcat & Jenkins logs older than 7 days removed!"

💬 Output:

🧹 Tomcat & Jenkins logs older than 7 days removed!

---

2️⃣ Apache & Middleware Examples

#!/bin/bash
ARCHIVE="/opt/log_archive/apache"
mkdir -p $ARCHIVE
find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
echo "📦 Apache logs compressed and archived in $ARCHIVE"

💬 Output:

📦 Apache logs compressed and archived in /opt/log_archive/apache

---

3️⃣ Docker & Kubernetes Log Cleanup

#!/bin/bash
echo "🐳 Cleaning Docker & K8s logs..."
docker system prune -af
find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
echo "✅ Docker and Kubernetes logs cleaned!"

💬 Output:

✅ Docker and Kubernetes logs cleaned!

---

4️⃣ logrotate Configuration

# /etc/logrotate.d/devops-services
/opt/tomcat/logs/*.log
/var/log/apache2/*.log
/var/lib/jenkins/logs/*.log
/var/log/docker/*.log {
    weekly
    rotate 6
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2 2>/dev/null || true
        systemctl restart tomcat 2>/dev/null || true
    endscript
}

💬 Result:

🔁 Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

---

5️⃣ How to Run & Test logrotate

Once your config is ready, you can run logrotate manually or let cron handle it.

🧠 Step 1 – Check Syntax of Your Config

sudo logrotate -d /etc/logrotate.d/devops-services

💬 Output:

Reading config file /etc/logrotate.d/devops-services
Handling /opt/tomcat/logs/*.log
Handling /var/log/apache2/*.log
**dry-run mode (no rotation performed)**

⚙️ Step 2 – Run logrotate Manually

sudo logrotate -f /etc/logrotate.d/devops-services

💬 Output:

Rotating logs for Tomcat, Apache, Jenkins...
Compression complete.
Rotation successful ✅

⏰ Step 3 – Schedule via Cron (Daily or Weekly)

# /etc/cron.daily/logrotate
/usr/sbin/logrotate /etc/logrotate.conf

💡 By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.

---

🏁 Summary

• ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
• ✅ Created unified logrotate configuration.
• ✅ Learned to manually test (-d) and force-run (-f) logrotate.
• ✅ Automated rotation through cron for 24×7 log hygiene.

← Previous: Part 10 – Middleware Health Monitoring Next: Part 12 – Backup & Restore Automation →