💾 Shell Scripting for Beginners – Part 12: Backup & Restore Automation Project (DevOps & Middleware Edition)

• Here we'll automate backup and restore tasks for real middleware & DevOps systems like Jenkins, Tomcat, WebSphere, MySQL, Docker, and Apache.
• Each example includes a working restore script and troubleshooting tips. ⚙️

---

📑 Table of Contents

• 1️⃣ Jenkins Backup & Restore
• 2️⃣ Tomcat Backup & Restore
• 3️⃣ WebSphere Backup & Restore
• 4️⃣ MySQL Database Backup & Restore
• 5️⃣ Docker Containers Backup & Restore
• 6️⃣ Apache Webserver Backup & Restore
• 7️⃣ Automating Backups
• 🧠 Pro Tips & Troubleshooting
• 🏁 Summary

---

1️⃣ Jenkins Backup & Restore

#!/bin/bash
JENKINS_HOME="/var/lib/jenkins"
BACKUP_DIR="/opt/backups/jenkins"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/jenkins_$(date +%F_%H-%M).tar.gz -C $JENKINS_HOME .
echo "✅ Jenkins backup created in $BACKUP_DIR"

♻️ Restore Jenkins

#!/bin/bash
BACKUP_FILE="/opt/backups/jenkins/jenkins_2025-11-02_02-10.tar.gz"
service jenkins stop
tar -xzf "$BACKUP_FILE" -C /var/lib/jenkins
chown -R jenkins:jenkins /var/lib/jenkins
service jenkins start
echo "♻️ Jenkins restored successfully."

---

2️⃣ Tomcat Backup & Restore

#!/bin/bash
TOMCAT_HOME="/opt/tomcat"
BACKUP_DIR="/opt/backups/tomcat"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/tomcat_$(date +%F).tar.gz $TOMCAT_HOME/webapps $TOMCAT_HOME/conf
echo "📦 Tomcat backup completed."

♻️ Restore Tomcat

#!/bin/bash
BACKUP_FILE="/opt/backups/tomcat/tomcat_2025-11-02.tar.gz"
service tomcat stop
tar -xzf "$BACKUP_FILE" -C /
service tomcat start
echo "♻️ Tomcat restored and restarted."

---

3️⃣ WebSphere Backup & Restore

#!/bin/bash
WAS_PROFILE="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"
BACKUP_DIR="/opt/backups/websphere"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/was_$(date +%F_%H-%M).tar.gz -C $WAS_PROFILE .
echo "📁 WebSphere configuration backup done."

♻️ Restore WebSphere

#!/bin/bash
BACKUP_FILE="/opt/backups/websphere/was_2025-11-02_03-00.tar.gz"
service was stop
tar -xzf "$BACKUP_FILE" -C /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
service was start
echo "♻️ WebSphere restored successfully."

---

4️⃣ MySQL Database Backup & Restore

#!/bin/bash
DB="middlewaredb"
USER="root"
PASS="Secret123"
BACKUP_DIR="/opt/backups/mysql"
mkdir -p $BACKUP_DIR
mysqldump -u $USER -p$PASS $DB > $BACKUP_DIR/${DB}_$(date +%F).sql
gzip $BACKUP_DIR/${DB}_$(date +%F).sql
echo "🗄️ MySQL backup completed."

♻️ Restore MySQL

#!/bin/bash
BACKUP_FILE="/opt/backups/mysql/middlewaredb_2025-11-02.sql.gz"
gunzip "$BACKUP_FILE"
mysql -u root -pSecret123 middlewaredb < /opt/backups/mysql/middlewaredb_2025-11-02.sql
echo "✅ Database restored successfully."

---

5️⃣ Docker Containers Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/docker"
mkdir -p $BACKUP_DIR
for cid in $(docker ps -q); do
  cname=$(docker inspect --format='{{.Name}}' $cid | cut -d'/' -f2)
  docker export $cid > $BACKUP_DIR/${cname}_$(date +%F).tar
done
echo "🐳 Docker containers exported successfully."

♻️ Restore Docker Container

#!/bin/bash
BACKUP_FILE="/opt/backups/docker/myapp_2025-11-02.tar"
docker import "$BACKUP_FILE" myapp_restored:latest
docker run -d --name myapp_restored myapp_restored:latest
echo "♻️ Docker container restored and running."

---

6️⃣ Apache Webserver Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/apache"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/apache_$(date +%F).tar.gz /etc/apache2 /var/www/html /etc/ssl
echo "🌐 Apache configuration, website, and SSL backed up."

♻️ Restore Apache

#!/bin/bash
BACKUP_FILE="/opt/backups/apache/apache_2025-11-02.tar.gz"
tar -xzf "$BACKUP_FILE" -C /
service apache2 restart
echo "✅ Apache restored and running."

---

7️⃣ Automating Backups

Schedule automatic backups using cron so your systems stay protected 24×7.

# Run daily at 2 AM
0 2 * * * /opt/scripts/backup_all.sh >> /var/log/backup_all.log 2>&1

# Run on every reboot
@reboot /opt/scripts/backup_all.sh >> /var/log/backup_boot.log 2>&1

💬 Output:

[CRON] Scheduled backup started…
✅ Jenkins, Tomcat, WebSphere, MySQL, Docker, Apache backed up successfully.

---

🧠 Pro Tips & Troubleshooting

✅ General Best Practices

• Use a dedicated /opt/backups partition or NFS mount to avoid filling system drives.
• Compress large backups with gzip or zstd [Developed by Meta] for better space efficiency.
• Store daily, weekly, and monthly copies separately (retention policy).
• Automate restores in a test environment weekly to ensure recovery works.

⚙️ Troubleshooting Common Issues

• ❌ Permission Denied: Run backups as sudo or fix ownership using chown.
• 📦 Disk Full: Use df -h to check space before backup. Add cleanup logic for old files.
• 🧾 Log Rotation: Add logrotate entry to keep backup logs small and readable.

---

🏁 Summary

• ✅ Full backup & restore automation for all major middleware and DevOps tools.
• 🧩 Scripts are modular, reusable, and easy to integrate in CI/CD pipelines.
• ⚙️ Cron ensures zero manual intervention for daily protection.

← Previous: Part 11 – Log Management & Rotation Next: Part 13 – Subshells & Process Control →

🎯Shell Scripting for Beginners – Part 10: Middleware Health Monitoring Project (Mixed Example).

• This is our final, beginner-friendly project 🎯 — where we'll combine everything you've learned: variables, if-else, loops, functions, logging, and cron automation.
• Let's create a simple Middleware Health Monitoring script that checks Tomcat, Jenkins, Docker, MySQL, and SSL expiry.

---

📑 Table of Contents

• 1️⃣ Overview
• 2️⃣ Full Simple Script
• 3️⃣ Example Output
• 4️⃣ Automate the Script

---

1️⃣ Overview

This script uses:

• Variables – store names, logs, and domains.
• Functions – reusable checks for each service.
• If-Else – to decide if a service needs restart.
• For loop – to check multiple Docker containers.
• While loop – to retry connection.
• Logs – write output to a file.

---

2️⃣ Full Middleware Health Monitoring Script

#!/bin/bash
# MiddlewareBox - Simple Health Monitor

LOG="/var/log/middleware_health.log"
domain="middlewarebox.com"
echo "===== Middleware Health Check =====" >> $LOG
date >> $LOG

# --- Function to check Tomcat ---
check_tomcat() {
  echo "Checking Tomcat..."
  if pgrep -f "org.apache.catalina.startup.Bootstrap" > /dev/null; then
     echo "✅ Tomcat is running" | tee -a $LOG
  else
     echo "🚨 Tomcat is down, restarting..." | tee -a $LOG
     /opt/tomcat/bin/startup.sh
     echo "✅ Tomcat restarted" | tee -a $LOG
  fi
}

# --- Function to check Jenkins ---
check_jenkins() {
  echo "Checking Jenkins..."
  if systemctl is-active --quiet jenkins; then
     echo "✅ Jenkins is active" | tee -a $LOG
  else
     echo "🚨 Jenkins not running, restarting..." | tee -a $LOG
     systemctl restart jenkins
     echo "✅ Jenkins restarted" | tee -a $LOG
  fi
}

# --- Function to check MySQL ---
check_db() {
  echo "Checking MySQL..."
  if mysql -u root -pSecret -e "show databases;" > /dev/null 2>&1; then
     echo "✅ MySQL connection successful" | tee -a $LOG
  else
     echo "🚨 MySQL connection failed" | tee -a $LOG
     echo "Retrying in 5 seconds..."
     sleep 5
     mysql -u root -pSecret -e "show databases;" && echo "✅ MySQL reconnected" | tee -a $LOG
  fi
}

# --- Check Docker Containers using FOR Loop ---
check_docker() {
  echo "Checking Docker containers..."
  for c in nginx webapp db
  do
    status=$(docker inspect -f '{{.State.Status}}' $c 2>/dev/null)
    if [ "$status" != "running" ]; then
      echo "🚨 Container $c is $status, restarting..." | tee -a $LOG
      docker start $c
    else
      echo "✅ Container $c is running" | tee -a $LOG
    fi
  done
}

# --- Check SSL certificate expiry ---
check_ssl() {
  echo "Checking SSL expiry..."
  expiry=$(echo | openssl s_client -servername $domain -connect $domain:443 2>/dev/null | openssl x509 -noout -dates | grep notAfter | cut -d= -f2)
  days_left=$(( ($(date -d "$expiry" +%s) - $(date +%s)) / 86400 ))
  if [ $days_left -le 15 ]; then
    echo "⚠️ SSL expires in $days_left days!" | tee -a $LOG
  else
    echo "✅ SSL valid for $days_left days" | tee -a $LOG
  fi
}

# --- MAIN EXECUTION FLOW ---
check_tomcat
check_jenkins
check_db
check_docker
check_ssl

echo "✅ Health check completed successfully!" | tee -a $LOG
echo "========================================" >> $LOG

💬 Example Output:

✅ Tomcat is running
✅ Jenkins is active
✅ MySQL connection successful
✅ Container nginx is running
✅ SSL valid for 70 days
✅ Health check completed successfully!

---

3️⃣ Automate the Script

# Run every 30 minutes
*/30 * * * * /opt/scripts/middleware_health.sh >> /var/log/health_cron.log 2>&1

# Run automatically on system reboot
@reboot /opt/scripts/middleware_health.sh >> /var/log/boot_health.log 2>&1

💬 Output (Cron log):

[BOOT] Health check executed
✅ All services healthy

---

🏁 Summary

• ✅ Combined all major shell scripting concepts.
• ✅ Checks Tomcat, Jenkins, Docker, MySQL, and SSL in one file.
• ✅ Uses if, for, functions, and basic while retry logic.
• ✅ Can run manually or via cron / reboot automation.

← Previous: Part 9 – Scheduling & Automation Next: Part 11 – Log Management & Rotation →

🕒 Shell Scripting for Beginners – Part 9: Scheduling & Automation

• In DevOps and Middleware environments, automation isn't complete until it's scheduled.
• Here, we master cron, at, @reboot, and Jenkins integration — running scripts automatically for backups, restarts, monitoring, and cleanups.

---

📑 Table of Contents

• 1️⃣ Automating with Cron
• 2️⃣ One-Time Tasks with At
• 3️⃣ Auto-Execute on System Reboot
• 4️⃣ Jenkins Scheduler Integration
• 5️⃣ Real Middleware Automation Examples
• 6️⃣ Tips & Troubleshooting
• 🏁 Summary

---

⏰ 1️⃣ Automating with Cron

• cron executes scripts periodically.
• Perfect for recurring jobs like backups, cleanup, and health checks.

Example 1: Daily Jenkins Backup at 3 AM

0 3 * * * /opt/scripts/jenkins_backup.sh >> /var/log/jenkins_backup.log 2>&1

💬 Output (log):

[03:00] Jenkins backup started
[03:03] ✅ Jenkins backup completed

Example 2: Weekly Tomcat Log Cleanup

0 2 * * 0 /opt/scripts/cleanup_tomcat_logs.sh >> /var/log/tomcat_cleanup.log 2>&1

💬 Output:

🧹 Deleted old logs from /opt/tomcat/logs
✅ Weekly cleanup done

---

🕐 2️⃣ One-Time Tasks with At

at executes a job once — ideal for maintenance windows, patches, or urgent restarts.

Example 3: Restart WebSphere at 11:30 PM

echo "/opt/IBM/WebSphere/AppServer/bin/stopServer.sh server1 && sleep 15 && /opt/IBM/WebSphere/AppServer/bin/startServer.sh server1" | at 23:30

💬 Output:

job 17 at Sat Nov 1 23:30:00 2025

Example 4: Run Backup Once in 2 Hours

echo "/opt/scripts/backup_db.sh" | at now + 2 hours

💬 Output:

job 18 at Sat Nov 1 05:30:00 2025

---

🔄 3️⃣ Auto-Execute on System Reboot

The @reboot keyword in crontab ensures your scripts run every time Linux boots. Perfect for restarting services, mounting drives, or initializing environments.

---

Example 5: Auto-Start Tomcat After Reboot

@reboot /opt/tomcat/bin/startup.sh >> /var/log/tomcat_reboot.log 2>&1

💬 Output (after reboot):

✅ Tomcat started automatically at boot.

Example 6: Health Check Script at Boot

@reboot /opt/scripts/health_check.sh >> /var/log/health_boot.log 2>&1

💬 Output (log):

[BOOT] Checking WebSphere, Jenkins, NGINX...
✅ All services active.

💡 To verify reboot jobs, check grep CRON /var/log/syslog after restart.

---

⚙️ 4️⃣ Jenkins Scheduler Integration

In CI/CD, Jenkins provides cron-like scheduling for build pipelines.

Example 7: Daily Jenkins Build at 2 AM

pipeline {
  triggers { cron('H 2 * * *') }
  stages {
    stage('Nightly Build') {
      steps {
        sh '/opt/scripts/build_app.sh'
      }
    }
  }
}

💬 Output:

✅ Jenkins triggered build_app.sh
Build completed successfully.

---

🧰 5️⃣ Real Middleware & DevOps Automation Examples

Example 8: NGINX Health Monitor Every 10 Minutes

*/10 * * * * /opt/scripts/nginx_health.sh >> /var/log/nginx_monitor.log 2>&1

💬 Output:

[02:40] 🌐 NGINX is active
[02:50] 🚨 NGINX down – restarted automatically.

Example 9: Database Backup Every 6 Hours

0 */6 * * * mysqldump -u root -pSecret middlewaredb > /backup/db_$(date +%F_%H).sql

💬 Output:

✅ Backup completed: db_2025-11-01_06.sql

Example 10: Restart Jenkins After Crash Detection

*/15 * * * * pgrep jenkins >/dev/null || systemctl restart jenkins

💬 Output:

[03:15] 🚨 Jenkins was down, restarted successfully.

Example 11: Auto-Remount Missing Volume

@reboot mount | grep /mnt/backup >/dev/null || mount /dev/sdb1 /mnt/backup

💬 Output:

✅ /mnt/backup auto-mounted on system startup.

---

🧠 6️⃣ Tips & Troubleshooting

• 📄 Check logs → grep CRON /var/log/syslog
• 🔒 Ensure permissions → chmod +x /opt/scripts/*.sh
• ⚙️ Test reboot jobs manually → sudo run-parts /etc/cron.d/
• 📧 Use MAILTO in crontab for notifications
• 🚨 For Jenkins jobs, check /var/lib/jenkins/jobs/*/builds

---

🏁 Summary

• Cron → Schedules recurring jobs
• At → Runs one-time tasks
• @reboot → Executes scripts on startup
• Jenkins → Manages enterprise-level schedules
• 💡 Combine them to create self-healing, auto-starting Middleware environments.

← Previous: Part 8 – Debugging & Logging Next: Part 10 – Middleware Health Monitoring →

📦 Shell Scripting for Beginners – Part 6: Arrays & Arguments

Arrays and arguments give your scripts flexibility — perfect for automating Middleware & DevOps tasks like restarting Tomcat across servers, cleaning WebSphere logs, or passing dynamic inputs to Jenkins jobs.

---

📑 Table of Contents

• Arrays
• Command-line Arguments
• Combining Arrays & Arguments
• Error Handling & Debugging
• Summary

---

1️⃣ Arrays in Shell

🎯 Example 1 – Basic Array

#!/bin/bash
servers=("dev" "stage" "prod")
echo "First server: ${servers[0]}"
echo "All servers: ${servers[@]}"

💬 Output:

First server: dev
All servers: dev stage prod

---

🔁 Example 2 – Loop through Array

#!/bin/bash
apps=("Tomcat" "WebSphere" "JBoss")
for app in "${apps[@]}"; do
  echo "Managing $app service..."
done

💬 Output:

Managing Tomcat service...
Managing WebSphere service...
Managing JBoss service...

---

🧰 Example 3 – Restart Tomcat on Multiple Servers

#!/bin/bash
servers=("app1" "app2" "app3")
for s in "${servers[@]}"; do
  echo "Restarting Tomcat on $s..."
  ssh $s "systemctl restart tomcat" || echo "⚠️ Restart failed on $s"
done
echo "✅ Restart loop completed."

💬 Output:

Restarting Tomcat on app1...
Restarting Tomcat on app2...
⚠️ Restart failed on app3
✅ Restart loop completed.

🧹 Example 4 – Cleanup WebSphere Logs

#!/bin/bash
logs=(SystemOut.log SystemErr.log activity.log)
for file in "${logs[@]}"; do
  echo "Deleting $file..."
  rm -f /opt/IBM/WebSphere/AppServer/logs/$file || echo "⚠️ Could not remove $file"
done
echo "🧹 Old WebSphere logs removed."

💬 Output:

Deleting SystemOut.log...
Deleting SystemErr.log...
Deleting activity.log...
🧹 Old WebSphere logs removed.

---

2️⃣ Command-line Arguments

📦 Example 5 – Access Arguments

#!/bin/bash
echo "Environment: $1"
echo "Version: $2"

💬 Output:

$ ./deploy.sh prod v2.0
Environment: prod
Version: v2.0

🚀 Example 6 – Trigger Jenkins Job

#!/bin/bash
job_name=$1
if [ -z "$job_name" ]; then
  echo "❌ Usage: $0 "
  exit 1
fi
curl -s -X POST "http://localhost:8080/job/$job_name/build?token=deploy123" || echo "⚠️ Failed to trigger job."

💬 Output:

$ ./jenkins_trigger.sh BuildApp
✅ Triggered Jenkins job BuildApp

🧠 Example 7 – Argument Validation

#!/bin/bash
if [ $# -lt 2 ]; then
  echo "Usage: $0  "
  exit 1
fi
echo "Deploying $1 version $2..."

💬 Output:

$ ./deploy.sh myapp v1.5
Deploying myapp version v1.5...

---

3️⃣ Combining Arrays & Arguments

🌍 Example 8 – Check NGINX on Multiple Hosts

#!/bin/bash
hosts=("$@")
for h in "${hosts[@]}"; do
  echo "Checking NGINX on $h..."
  ssh $h "systemctl is-active nginx" >/dev/null 2>&1 && echo "✅ $h OK" || echo "❌ $h DOWN"
done

💬 Output:

$ ./check_nginx.sh host1 host2 host3
✅ host1 OK
✅ host2 OK
❌ host3 DOWN

💾 Example 9 – Query Multiple Databases

#!/bin/bash
dbs=("salesdb" "hrdb" "logdb")
for db in "${dbs[@]}"; do
  echo "Checking $db connection..."
  mysql -h dbserver -u admin -psecret -e "use $db;" >/dev/null 2>&1 && echo "✅ $db reachable" || echo "⚠️ $db not reachable"
done

💬 Output:

Checking salesdb connection... ✅ salesdb reachable
Checking hrdb connection... ⚠️ hrdb not reachable
Checking logdb connection... ✅ logdb reachable

🧩 Example 10 – CI/CD Pipeline Trigger

#!/bin/bash
branches=("main" "dev" "hotfix")
for b in "${branches[@]}"; do
  echo "Triggering pipeline for $b..."
  curl -s -X POST "https://ci.middlewarebox.com/job/build-$b/build" || echo "⚠️ Failed for $b"
done

💬 Output:

Triggering pipeline for main... ✅ Done
Triggering pipeline for dev... ✅ Done
Triggering pipeline for hotfix... ⚠️ Failed for hotfix

---

4️⃣ Error Handling & Debugging

• Use set -x to trace executed commands.
• Check argument count with $#.
• Use "$@" to safely pass all args to loops.
• Handle SSH/curl/mysql failures with || and echo messages.
• Log outputs → script.sh >> /var/log/script.log 2>&1

---

🏁 Summary

• Arrays – store lists of servers, apps, or files.
• Arguments – pass values into scripts for dynamic execution.
• Combine both for multi-server automation.
• Always include error handling to avoid silent failures.

← Previous: Part 5 – Functions in Shell Next: Part 7 – File Handling in Shell →