💾 Shell Scripting for Beginners – Part 12: Backup & Restore Automation Project (DevOps & Middleware Edition)

• Here we'll automate backup and restore tasks for real middleware & DevOps systems like Jenkins, Tomcat, WebSphere, MySQL, Docker, and Apache.
• Each example includes a working restore script and troubleshooting tips. ⚙️

---

📑 Table of Contents

• 1️⃣ Jenkins Backup & Restore
• 2️⃣ Tomcat Backup & Restore
• 3️⃣ WebSphere Backup & Restore
• 4️⃣ MySQL Database Backup & Restore
• 5️⃣ Docker Containers Backup & Restore
• 6️⃣ Apache Webserver Backup & Restore
• 7️⃣ Automating Backups
• 🧠 Pro Tips & Troubleshooting
• 🏁 Summary

---

1️⃣ Jenkins Backup & Restore

#!/bin/bash
JENKINS_HOME="/var/lib/jenkins"
BACKUP_DIR="/opt/backups/jenkins"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/jenkins_$(date +%F_%H-%M).tar.gz -C $JENKINS_HOME .
echo "✅ Jenkins backup created in $BACKUP_DIR"

♻️ Restore Jenkins

#!/bin/bash
BACKUP_FILE="/opt/backups/jenkins/jenkins_2025-11-02_02-10.tar.gz"
service jenkins stop
tar -xzf "$BACKUP_FILE" -C /var/lib/jenkins
chown -R jenkins:jenkins /var/lib/jenkins
service jenkins start
echo "♻️ Jenkins restored successfully."

---

2️⃣ Tomcat Backup & Restore

#!/bin/bash
TOMCAT_HOME="/opt/tomcat"
BACKUP_DIR="/opt/backups/tomcat"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/tomcat_$(date +%F).tar.gz $TOMCAT_HOME/webapps $TOMCAT_HOME/conf
echo "📦 Tomcat backup completed."

♻️ Restore Tomcat

#!/bin/bash
BACKUP_FILE="/opt/backups/tomcat/tomcat_2025-11-02.tar.gz"
service tomcat stop
tar -xzf "$BACKUP_FILE" -C /
service tomcat start
echo "♻️ Tomcat restored and restarted."

---

3️⃣ WebSphere Backup & Restore

#!/bin/bash
WAS_PROFILE="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"
BACKUP_DIR="/opt/backups/websphere"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/was_$(date +%F_%H-%M).tar.gz -C $WAS_PROFILE .
echo "📁 WebSphere configuration backup done."

♻️ Restore WebSphere

#!/bin/bash
BACKUP_FILE="/opt/backups/websphere/was_2025-11-02_03-00.tar.gz"
service was stop
tar -xzf "$BACKUP_FILE" -C /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
service was start
echo "♻️ WebSphere restored successfully."

---

4️⃣ MySQL Database Backup & Restore

#!/bin/bash
DB="middlewaredb"
USER="root"
PASS="Secret123"
BACKUP_DIR="/opt/backups/mysql"
mkdir -p $BACKUP_DIR
mysqldump -u $USER -p$PASS $DB > $BACKUP_DIR/${DB}_$(date +%F).sql
gzip $BACKUP_DIR/${DB}_$(date +%F).sql
echo "🗄️ MySQL backup completed."

♻️ Restore MySQL

#!/bin/bash
BACKUP_FILE="/opt/backups/mysql/middlewaredb_2025-11-02.sql.gz"
gunzip "$BACKUP_FILE"
mysql -u root -pSecret123 middlewaredb < /opt/backups/mysql/middlewaredb_2025-11-02.sql
echo "✅ Database restored successfully."

---

5️⃣ Docker Containers Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/docker"
mkdir -p $BACKUP_DIR
for cid in $(docker ps -q); do
  cname=$(docker inspect --format='{{.Name}}' $cid | cut -d'/' -f2)
  docker export $cid > $BACKUP_DIR/${cname}_$(date +%F).tar
done
echo "🐳 Docker containers exported successfully."

♻️ Restore Docker Container

#!/bin/bash
BACKUP_FILE="/opt/backups/docker/myapp_2025-11-02.tar"
docker import "$BACKUP_FILE" myapp_restored:latest
docker run -d --name myapp_restored myapp_restored:latest
echo "♻️ Docker container restored and running."

---

6️⃣ Apache Webserver Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/apache"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/apache_$(date +%F).tar.gz /etc/apache2 /var/www/html /etc/ssl
echo "🌐 Apache configuration, website, and SSL backed up."

♻️ Restore Apache

#!/bin/bash
BACKUP_FILE="/opt/backups/apache/apache_2025-11-02.tar.gz"
tar -xzf "$BACKUP_FILE" -C /
service apache2 restart
echo "✅ Apache restored and running."

---

7️⃣ Automating Backups

Schedule automatic backups using cron so your systems stay protected 24×7.

# Run daily at 2 AM
0 2 * * * /opt/scripts/backup_all.sh >> /var/log/backup_all.log 2>&1

# Run on every reboot
@reboot /opt/scripts/backup_all.sh >> /var/log/backup_boot.log 2>&1

💬 Output:

[CRON] Scheduled backup started…
✅ Jenkins, Tomcat, WebSphere, MySQL, Docker, Apache backed up successfully.

---

🧠 Pro Tips & Troubleshooting

✅ General Best Practices

• Use a dedicated /opt/backups partition or NFS mount to avoid filling system drives.
• Compress large backups with gzip or zstd [Developed by Meta] for better space efficiency.
• Store daily, weekly, and monthly copies separately (retention policy).
• Automate restores in a test environment weekly to ensure recovery works.

⚙️ Troubleshooting Common Issues

• ❌ Permission Denied: Run backups as sudo or fix ownership using chown.
• 📦 Disk Full: Use df -h to check space before backup. Add cleanup logic for old files.
• 🧾 Log Rotation: Add logrotate entry to keep backup logs small and readable.

---

🏁 Summary

• ✅ Full backup & restore automation for all major middleware and DevOps tools.
• 🧩 Scripts are modular, reusable, and easy to integrate in CI/CD pipelines.
• ⚙️ Cron ensures zero manual intervention for daily protection.

← Previous: Part 11 – Log Management & Rotation Next: Part 13 – Subshells & Process Control →

🧾 Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

• Logs are the heartbeat of your servers 💓 — but if left unmanaged, they'll crash your systems.
• In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

---

📑 Table of Contents

• 1️⃣ Log Cleanup Scripts
• 2️⃣ Apache & Middleware Examples
• 3️⃣ Docker & K8s Cleanup
• 4️⃣ logrotate Configuration
• 5️⃣ How to Run & Test logrotate
• 🏁 Summary

---

1️⃣ Log Cleanup Scripts

#!/bin/bash
# Delete logs older than 7 days for Tomcat & Jenkins
find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
echo "🧹 Tomcat & Jenkins logs older than 7 days removed!"

💬 Output:

🧹 Tomcat & Jenkins logs older than 7 days removed!

---

2️⃣ Apache & Middleware Examples

#!/bin/bash
ARCHIVE="/opt/log_archive/apache"
mkdir -p $ARCHIVE
find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
echo "📦 Apache logs compressed and archived in $ARCHIVE"

💬 Output:

📦 Apache logs compressed and archived in /opt/log_archive/apache

---

3️⃣ Docker & Kubernetes Log Cleanup

#!/bin/bash
echo "🐳 Cleaning Docker & K8s logs..."
docker system prune -af
find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
echo "✅ Docker and Kubernetes logs cleaned!"

💬 Output:

✅ Docker and Kubernetes logs cleaned!

---

4️⃣ logrotate Configuration

# /etc/logrotate.d/devops-services
/opt/tomcat/logs/*.log
/var/log/apache2/*.log
/var/lib/jenkins/logs/*.log
/var/log/docker/*.log {
    weekly
    rotate 6
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2 2>/dev/null || true
        systemctl restart tomcat 2>/dev/null || true
    endscript
}

💬 Result:

🔁 Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

---

5️⃣ How to Run & Test logrotate

Once your config is ready, you can run logrotate manually or let cron handle it.

🧠 Step 1 – Check Syntax of Your Config

sudo logrotate -d /etc/logrotate.d/devops-services

💬 Output:

Reading config file /etc/logrotate.d/devops-services
Handling /opt/tomcat/logs/*.log
Handling /var/log/apache2/*.log
**dry-run mode (no rotation performed)**

⚙️ Step 2 – Run logrotate Manually

sudo logrotate -f /etc/logrotate.d/devops-services

💬 Output:

Rotating logs for Tomcat, Apache, Jenkins...
Compression complete.
Rotation successful ✅

⏰ Step 3 – Schedule via Cron (Daily or Weekly)

# /etc/cron.daily/logrotate
/usr/sbin/logrotate /etc/logrotate.conf

💡 By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.

---

🏁 Summary

• ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
• ✅ Created unified logrotate configuration.
• ✅ Learned to manually test (-d) and force-run (-f) logrotate.
• ✅ Automated rotation through cron for 24×7 log hygiene.

← Previous: Part 10 – Middleware Health Monitoring Next: Part 12 – Backup & Restore Automation →

🎯Shell Scripting for Beginners – Part 10: Middleware Health Monitoring Project (Mixed Example).

• This is our final, beginner-friendly project 🎯 — where we'll combine everything you've learned: variables, if-else, loops, functions, logging, and cron automation.
• Let's create a simple Middleware Health Monitoring script that checks Tomcat, Jenkins, Docker, MySQL, and SSL expiry.

---

📑 Table of Contents

• 1️⃣ Overview
• 2️⃣ Full Simple Script
• 3️⃣ Example Output
• 4️⃣ Automate the Script

---

1️⃣ Overview

This script uses:

• Variables – store names, logs, and domains.
• Functions – reusable checks for each service.
• If-Else – to decide if a service needs restart.
• For loop – to check multiple Docker containers.
• While loop – to retry connection.
• Logs – write output to a file.

---

2️⃣ Full Middleware Health Monitoring Script

#!/bin/bash
# MiddlewareBox - Simple Health Monitor

LOG="/var/log/middleware_health.log"
domain="middlewarebox.com"
echo "===== Middleware Health Check =====" >> $LOG
date >> $LOG

# --- Function to check Tomcat ---
check_tomcat() {
  echo "Checking Tomcat..."
  if pgrep -f "org.apache.catalina.startup.Bootstrap" > /dev/null; then
     echo "✅ Tomcat is running" | tee -a $LOG
  else
     echo "🚨 Tomcat is down, restarting..." | tee -a $LOG
     /opt/tomcat/bin/startup.sh
     echo "✅ Tomcat restarted" | tee -a $LOG
  fi
}

# --- Function to check Jenkins ---
check_jenkins() {
  echo "Checking Jenkins..."
  if systemctl is-active --quiet jenkins; then
     echo "✅ Jenkins is active" | tee -a $LOG
  else
     echo "🚨 Jenkins not running, restarting..." | tee -a $LOG
     systemctl restart jenkins
     echo "✅ Jenkins restarted" | tee -a $LOG
  fi
}

# --- Function to check MySQL ---
check_db() {
  echo "Checking MySQL..."
  if mysql -u root -pSecret -e "show databases;" > /dev/null 2>&1; then
     echo "✅ MySQL connection successful" | tee -a $LOG
  else
     echo "🚨 MySQL connection failed" | tee -a $LOG
     echo "Retrying in 5 seconds..."
     sleep 5
     mysql -u root -pSecret -e "show databases;" && echo "✅ MySQL reconnected" | tee -a $LOG
  fi
}

# --- Check Docker Containers using FOR Loop ---
check_docker() {
  echo "Checking Docker containers..."
  for c in nginx webapp db
  do
    status=$(docker inspect -f '{{.State.Status}}' $c 2>/dev/null)
    if [ "$status" != "running" ]; then
      echo "🚨 Container $c is $status, restarting..." | tee -a $LOG
      docker start $c
    else
      echo "✅ Container $c is running" | tee -a $LOG
    fi
  done
}

# --- Check SSL certificate expiry ---
check_ssl() {
  echo "Checking SSL expiry..."
  expiry=$(echo | openssl s_client -servername $domain -connect $domain:443 2>/dev/null | openssl x509 -noout -dates | grep notAfter | cut -d= -f2)
  days_left=$(( ($(date -d "$expiry" +%s) - $(date +%s)) / 86400 ))
  if [ $days_left -le 15 ]; then
    echo "⚠️ SSL expires in $days_left days!" | tee -a $LOG
  else
    echo "✅ SSL valid for $days_left days" | tee -a $LOG
  fi
}

# --- MAIN EXECUTION FLOW ---
check_tomcat
check_jenkins
check_db
check_docker
check_ssl

echo "✅ Health check completed successfully!" | tee -a $LOG
echo "========================================" >> $LOG

💬 Example Output:

✅ Tomcat is running
✅ Jenkins is active
✅ MySQL connection successful
✅ Container nginx is running
✅ SSL valid for 70 days
✅ Health check completed successfully!

---

3️⃣ Automate the Script

# Run every 30 minutes
*/30 * * * * /opt/scripts/middleware_health.sh >> /var/log/health_cron.log 2>&1

# Run automatically on system reboot
@reboot /opt/scripts/middleware_health.sh >> /var/log/boot_health.log 2>&1

💬 Output (Cron log):

[BOOT] Health check executed
✅ All services healthy

---

🏁 Summary

• ✅ Combined all major shell scripting concepts.
• ✅ Checks Tomcat, Jenkins, Docker, MySQL, and SSL in one file.
• ✅ Uses if, for, functions, and basic while retry logic.
• ✅ Can run manually or via cron / reboot automation.

← Previous: Part 9 – Scheduling & Automation Next: Part 11 – Log Management & Rotation →

🕒 Shell Scripting for Beginners – Part 9: Scheduling & Automation

• In DevOps and Middleware environments, automation isn't complete until it's scheduled.
• Here, we master cron, at, @reboot, and Jenkins integration — running scripts automatically for backups, restarts, monitoring, and cleanups.

---

📑 Table of Contents

• 1️⃣ Automating with Cron
• 2️⃣ One-Time Tasks with At
• 3️⃣ Auto-Execute on System Reboot
• 4️⃣ Jenkins Scheduler Integration
• 5️⃣ Real Middleware Automation Examples
• 6️⃣ Tips & Troubleshooting
• 🏁 Summary

---

⏰ 1️⃣ Automating with Cron

• cron executes scripts periodically.
• Perfect for recurring jobs like backups, cleanup, and health checks.

Example 1: Daily Jenkins Backup at 3 AM

0 3 * * * /opt/scripts/jenkins_backup.sh >> /var/log/jenkins_backup.log 2>&1

💬 Output (log):

[03:00] Jenkins backup started
[03:03] ✅ Jenkins backup completed

Example 2: Weekly Tomcat Log Cleanup

0 2 * * 0 /opt/scripts/cleanup_tomcat_logs.sh >> /var/log/tomcat_cleanup.log 2>&1

💬 Output:

🧹 Deleted old logs from /opt/tomcat/logs
✅ Weekly cleanup done

---

🕐 2️⃣ One-Time Tasks with At

at executes a job once — ideal for maintenance windows, patches, or urgent restarts.

Example 3: Restart WebSphere at 11:30 PM

echo "/opt/IBM/WebSphere/AppServer/bin/stopServer.sh server1 && sleep 15 && /opt/IBM/WebSphere/AppServer/bin/startServer.sh server1" | at 23:30

💬 Output:

job 17 at Sat Nov 1 23:30:00 2025

Example 4: Run Backup Once in 2 Hours

echo "/opt/scripts/backup_db.sh" | at now + 2 hours

💬 Output:

job 18 at Sat Nov 1 05:30:00 2025

---

🔄 3️⃣ Auto-Execute on System Reboot

The @reboot keyword in crontab ensures your scripts run every time Linux boots. Perfect for restarting services, mounting drives, or initializing environments.

---

Example 5: Auto-Start Tomcat After Reboot

@reboot /opt/tomcat/bin/startup.sh >> /var/log/tomcat_reboot.log 2>&1

💬 Output (after reboot):

✅ Tomcat started automatically at boot.

Example 6: Health Check Script at Boot

@reboot /opt/scripts/health_check.sh >> /var/log/health_boot.log 2>&1

💬 Output (log):

[BOOT] Checking WebSphere, Jenkins, NGINX...
✅ All services active.

💡 To verify reboot jobs, check grep CRON /var/log/syslog after restart.

---

⚙️ 4️⃣ Jenkins Scheduler Integration

In CI/CD, Jenkins provides cron-like scheduling for build pipelines.

Example 7: Daily Jenkins Build at 2 AM

pipeline {
  triggers { cron('H 2 * * *') }
  stages {
    stage('Nightly Build') {
      steps {
        sh '/opt/scripts/build_app.sh'
      }
    }
  }
}

💬 Output:

✅ Jenkins triggered build_app.sh
Build completed successfully.

---

🧰 5️⃣ Real Middleware & DevOps Automation Examples

Example 8: NGINX Health Monitor Every 10 Minutes

*/10 * * * * /opt/scripts/nginx_health.sh >> /var/log/nginx_monitor.log 2>&1

💬 Output:

[02:40] 🌐 NGINX is active
[02:50] 🚨 NGINX down – restarted automatically.

Example 9: Database Backup Every 6 Hours

0 */6 * * * mysqldump -u root -pSecret middlewaredb > /backup/db_$(date +%F_%H).sql

💬 Output:

✅ Backup completed: db_2025-11-01_06.sql

Example 10: Restart Jenkins After Crash Detection

*/15 * * * * pgrep jenkins >/dev/null || systemctl restart jenkins

💬 Output:

[03:15] 🚨 Jenkins was down, restarted successfully.

Example 11: Auto-Remount Missing Volume

@reboot mount | grep /mnt/backup >/dev/null || mount /dev/sdb1 /mnt/backup

💬 Output:

✅ /mnt/backup auto-mounted on system startup.

---

🧠 6️⃣ Tips & Troubleshooting

• 📄 Check logs → grep CRON /var/log/syslog
• 🔒 Ensure permissions → chmod +x /opt/scripts/*.sh
• ⚙️ Test reboot jobs manually → sudo run-parts /etc/cron.d/
• 📧 Use MAILTO in crontab for notifications
• 🚨 For Jenkins jobs, check /var/lib/jenkins/jobs/*/builds

---

🏁 Summary

• Cron → Schedules recurring jobs
• At → Runs one-time tasks
• @reboot → Executes scripts on startup
• Jenkins → Manages enterprise-level schedules
• 💡 Combine them to create self-healing, auto-starting Middleware environments.

← Previous: Part 8 – Debugging & Logging Next: Part 10 – Middleware Health Monitoring →