Menu

Showing posts with label Automation. Show all posts
Showing posts with label Automation. Show all posts

2 Nov 2025

๐Ÿง Linux Shell Scripting for Beginners – Complete Tutorial Series.

  • A 13-part practical shell scripting course for DevOps and Middleware Engineers.

๐Ÿ’ก How This Tutorial Series Will Help You

  • ✔ Build a strong foundation in Linux Shell scripting — from basics to automation.
  • ✔ Automate daily Middleware and DevOps tasks: monitoring, backups, and log management.
  • ✔ Write scripts for Middleware platforms like Tomcat, Jenkins, and WebSphere.
  • ✔ Debug, schedule, and manage production-ready shell scripts confidently.
  • ✔ Perfect for Sysadmins, Middleware, and DevOps Engineers upgrading to Cloud roles.

๐Ÿ“š Complete Course Index

๐Ÿš Part 1: What is a Variable?
Learn how variables store data and simplify automation scripts. Foundation of all shell logic.
๐Ÿ“ฅ Part 2: Reading User Input in Shell
Make interactive scripts using read command with Jenkins and Tomcat examples.
Use decision-making in your scripts with examples checking files, services, and network health.
Automate repetitive DevOps tasks like log cleanup, backups, and status monitoring.
Write modular, reusable functions — restart Tomcat or back up Jenkins in one click.
๐Ÿ“ฆ Part 6: Arrays & Arguments
Store multiple values, manage user inputs, and process server lists in scripts.
๐Ÿ“ Part 7: File Handling in Shell
Learn how to read, write, append, and handle configuration or log files safely.
๐Ÿ’ก Part 8: Debugging & Logging
Find and fix script errors, use logs, and enable debug mode for safe execution.
๐Ÿ•’ Part 9: Scheduling & Automation
Use cron and @reboot jobs to run scripts automatically for health checks or backups.
Create a production-ready monitoring script for Tomcat, Jenkins, and NGINX with auto-restart.
๐Ÿงพ Part 11: Log Management & Rotation
Automate log rotation with logrotate and custom retention scripts for Middleware systems.
Perform automated backup and restore for Jenkins, Tomcat, and WebSphere with rollback support.
Run multiple jobs in parallel, monitor background processes, and speed up automation workflows.

๐Ÿš€ Why MiddlewareBox Shell Scripting Series?

  • ✅ Step-by-step, beginner-friendly explanations.
  • ✅ Each topic includes real DevOps & Middleware context.
  • ✅ Examples with Output and Error handling.
  • ✅ Simple language for better understanding.

✨ Start Learning Now → Part 1: What is a Variable?
& build your own scripts from scratch! ๐Ÿ’ช

๐Ÿ’พ Shell Scripting for Beginners – Part 12: Backup & Restore Automation Project (DevOps & Middleware Edition)

  • Here we'll automate backup and restore tasks for real middleware & DevOps systems like Jenkins, Tomcat, WebSphere, MySQL, Docker, and Apache.
  • Each example includes a working restore script and troubleshooting tips. ⚙️

๐Ÿ“‘ Table of Contents


1️⃣ Jenkins Backup & Restore

#!/bin/bash
JENKINS_HOME="/var/lib/jenkins"
BACKUP_DIR="/opt/backups/jenkins"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/jenkins_$(date +%F_%H-%M).tar.gz -C $JENKINS_HOME .
echo "✅ Jenkins backup created in $BACKUP_DIR"

♻️ Restore Jenkins

#!/bin/bash
BACKUP_FILE="/opt/backups/jenkins/jenkins_2025-11-02_02-10.tar.gz"
service jenkins stop
tar -xzf "$BACKUP_FILE" -C /var/lib/jenkins
chown -R jenkins:jenkins /var/lib/jenkins
service jenkins start
echo "♻️ Jenkins restored successfully."

2️⃣ Tomcat Backup & Restore

#!/bin/bash
TOMCAT_HOME="/opt/tomcat"
BACKUP_DIR="/opt/backups/tomcat"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/tomcat_$(date +%F).tar.gz $TOMCAT_HOME/webapps $TOMCAT_HOME/conf
echo "๐Ÿ“ฆ Tomcat backup completed."

♻️ Restore Tomcat

#!/bin/bash
BACKUP_FILE="/opt/backups/tomcat/tomcat_2025-11-02.tar.gz"
service tomcat stop
tar -xzf "$BACKUP_FILE" -C /
service tomcat start
echo "♻️ Tomcat restored and restarted."

3️⃣ WebSphere Backup & Restore

#!/bin/bash
WAS_PROFILE="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"
BACKUP_DIR="/opt/backups/websphere"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/was_$(date +%F_%H-%M).tar.gz -C $WAS_PROFILE .
echo "๐Ÿ“ WebSphere configuration backup done."

♻️ Restore WebSphere

#!/bin/bash
BACKUP_FILE="/opt/backups/websphere/was_2025-11-02_03-00.tar.gz"
service was stop
tar -xzf "$BACKUP_FILE" -C /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
service was start
echo "♻️ WebSphere restored successfully."

4️⃣ MySQL Database Backup & Restore

#!/bin/bash
DB="middlewaredb"
USER="root"
PASS="Secret123"
BACKUP_DIR="/opt/backups/mysql"
mkdir -p $BACKUP_DIR
mysqldump -u $USER -p$PASS $DB > $BACKUP_DIR/${DB}_$(date +%F).sql
gzip $BACKUP_DIR/${DB}_$(date +%F).sql
echo "๐Ÿ—„️ MySQL backup completed."

♻️ Restore MySQL

#!/bin/bash
BACKUP_FILE="/opt/backups/mysql/middlewaredb_2025-11-02.sql.gz"
gunzip "$BACKUP_FILE"
mysql -u root -pSecret123 middlewaredb < /opt/backups/mysql/middlewaredb_2025-11-02.sql
echo "✅ Database restored successfully."

5️⃣ Docker Containers Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/docker"
mkdir -p $BACKUP_DIR
for cid in $(docker ps -q); do
  cname=$(docker inspect --format='{{.Name}}' $cid | cut -d'/' -f2)
  docker export $cid > $BACKUP_DIR/${cname}_$(date +%F).tar
done
echo "๐Ÿณ Docker containers exported successfully."

♻️ Restore Docker Container

#!/bin/bash
BACKUP_FILE="/opt/backups/docker/myapp_2025-11-02.tar"
docker import "$BACKUP_FILE" myapp_restored:latest
docker run -d --name myapp_restored myapp_restored:latest
echo "♻️ Docker container restored and running."

6️⃣ Apache Webserver Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/apache"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/apache_$(date +%F).tar.gz /etc/apache2 /var/www/html /etc/ssl
echo "๐ŸŒ Apache configuration, website, and SSL backed up."

♻️ Restore Apache

#!/bin/bash
BACKUP_FILE="/opt/backups/apache/apache_2025-11-02.tar.gz"
tar -xzf "$BACKUP_FILE" -C /
service apache2 restart
echo "✅ Apache restored and running."

7️⃣ Automating Backups

Schedule automatic backups using cron so your systems stay protected 24×7.

# Run daily at 2 AM
0 2 * * * /opt/scripts/backup_all.sh >> /var/log/backup_all.log 2>&1

# Run on every reboot
@reboot /opt/scripts/backup_all.sh >> /var/log/backup_boot.log 2>&1
๐Ÿ’ฌ Output:
[CRON] Scheduled backup started…
✅ Jenkins, Tomcat, WebSphere, MySQL, Docker, Apache backed up successfully.

๐Ÿง  Pro Tips & Troubleshooting

✅ General Best Practices

  • Use a dedicated /opt/backups partition or NFS mount to avoid filling system drives.
  • Compress large backups with gzip or zstd [Developed by Meta] for better space efficiency.
  • Store daily, weekly, and monthly copies separately (retention policy).
  • Automate restores in a test environment weekly to ensure recovery works.

⚙️ Troubleshooting Common Issues

  • Permission Denied: Run backups as sudo or fix ownership using chown.
  • ๐Ÿ“ฆ Disk Full: Use df -h to check space before backup. Add cleanup logic for old files.
  • ๐Ÿงพ Log Rotation: Add logrotate entry to keep backup logs small and readable.

๐Ÿ Summary

  • ✅ Full backup & restore automation for all major middleware and DevOps tools.
  • ๐Ÿงฉ Scripts are modular, reusable, and easy to integrate in CI/CD pipelines.
  • ⚙️ Cron ensures zero manual intervention for daily protection.

1 Nov 2025

๐Ÿงพ Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

  • Logs are the heartbeat of your servers ๐Ÿ’“ — but if left unmanaged, they'll crash your systems.
  • In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

๐Ÿ“‘ Table of Contents


1️⃣ Log Cleanup Scripts

#!/bin/bash
# Delete logs older than 7 days for Tomcat & Jenkins
find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
echo "๐Ÿงน Tomcat & Jenkins logs older than 7 days removed!"
๐Ÿ’ฌ Output:
๐Ÿงน Tomcat & Jenkins logs older than 7 days removed!

2️⃣ Apache & Middleware Examples

#!/bin/bash
ARCHIVE="/opt/log_archive/apache"
mkdir -p $ARCHIVE
find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
echo "๐Ÿ“ฆ Apache logs compressed and archived in $ARCHIVE"
๐Ÿ’ฌ Output:
๐Ÿ“ฆ Apache logs compressed and archived in /opt/log_archive/apache

3️⃣ Docker & Kubernetes Log Cleanup

#!/bin/bash
echo "๐Ÿณ Cleaning Docker & K8s logs..."
docker system prune -af
find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
echo "✅ Docker and Kubernetes logs cleaned!"
๐Ÿ’ฌ Output:
✅ Docker and Kubernetes logs cleaned!

4️⃣ logrotate Configuration

# /etc/logrotate.d/devops-services
/opt/tomcat/logs/*.log
/var/log/apache2/*.log
/var/lib/jenkins/logs/*.log
/var/log/docker/*.log {
    weekly
    rotate 6
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2 2>/dev/null || true
        systemctl restart tomcat 2>/dev/null || true
    endscript
}
๐Ÿ’ฌ Result:
๐Ÿ” Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

5️⃣ How to Run & Test logrotate

Once your config is ready, you can run logrotate manually or let cron handle it.

๐Ÿง  Step 1 – Check Syntax of Your Config

sudo logrotate -d /etc/logrotate.d/devops-services
๐Ÿ’ฌ Output:
Reading config file /etc/logrotate.d/devops-services
Handling /opt/tomcat/logs/*.log
Handling /var/log/apache2/*.log
**dry-run mode (no rotation performed)**

⚙️ Step 2 – Run logrotate Manually

sudo logrotate -f /etc/logrotate.d/devops-services
๐Ÿ’ฌ Output:
Rotating logs for Tomcat, Apache, Jenkins...
Compression complete.
Rotation successful ✅

⏰ Step 3 – Schedule via Cron (Daily or Weekly)

# /etc/cron.daily/logrotate
/usr/sbin/logrotate /etc/logrotate.conf

๐Ÿ’ก By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.


๐Ÿ Summary

  • ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
  • ✅ Created unified logrotate configuration.
  • ✅ Learned to manually test (-d) and force-run (-f) logrotate.
  • ✅ Automated rotation through cron for 24×7 log hygiene.

๐Ÿ•’ Shell Scripting for Beginners – Part 9: Scheduling & Automation

  • In DevOps and Middleware environments, automation isn't complete until it's scheduled.
  • Here, we master cron, at, @reboot, and Jenkins integration — running scripts automatically for backups, restarts, monitoring, and cleanups.

๐Ÿ“‘ Table of Contents


⏰ 1️⃣ Automating with Cron

  • cron executes scripts periodically.
  • Perfect for recurring jobs like backups, cleanup, and health checks.

Example 1: Daily Jenkins Backup at 3 AM

0 3 * * * /opt/scripts/jenkins_backup.sh >> /var/log/jenkins_backup.log 2>&1
๐Ÿ’ฌ Output (log):
[03:00] Jenkins backup started
[03:03] ✅ Jenkins backup completed

Example 2: Weekly Tomcat Log Cleanup

0 2 * * 0 /opt/scripts/cleanup_tomcat_logs.sh >> /var/log/tomcat_cleanup.log 2>&1
๐Ÿ’ฌ Output:
๐Ÿงน Deleted old logs from /opt/tomcat/logs
✅ Weekly cleanup done

๐Ÿ• 2️⃣ One-Time Tasks with At

at executes a job once — ideal for maintenance windows, patches, or urgent restarts.

Example 3: Restart WebSphere at 11:30 PM

echo "/opt/IBM/WebSphere/AppServer/bin/stopServer.sh server1 && sleep 15 && /opt/IBM/WebSphere/AppServer/bin/startServer.sh server1" | at 23:30
๐Ÿ’ฌ Output:
job 17 at Sat Nov 1 23:30:00 2025

Example 4: Run Backup Once in 2 Hours

echo "/opt/scripts/backup_db.sh" | at now + 2 hours
๐Ÿ’ฌ Output:
job 18 at Sat Nov 1 05:30:00 2025

๐Ÿ”„ 3️⃣ Auto-Execute on System Reboot

The @reboot keyword in crontab ensures your scripts run every time Linux boots. Perfect for restarting services, mounting drives, or initializing environments.


Example 5: Auto-Start Tomcat After Reboot

@reboot /opt/tomcat/bin/startup.sh >> /var/log/tomcat_reboot.log 2>&1
๐Ÿ’ฌ Output (after reboot):
✅ Tomcat started automatically at boot.

Example 6: Health Check Script at Boot

@reboot /opt/scripts/health_check.sh >> /var/log/health_boot.log 2>&1
๐Ÿ’ฌ Output (log):
[BOOT] Checking WebSphere, Jenkins, NGINX...
✅ All services active.
๐Ÿ’ก To verify reboot jobs, check grep CRON /var/log/syslog after restart.

⚙️ 4️⃣ Jenkins Scheduler Integration

In CI/CD, Jenkins provides cron-like scheduling for build pipelines.

Example 7: Daily Jenkins Build at 2 AM

pipeline {
  triggers { cron('H 2 * * *') }
  stages {
    stage('Nightly Build') {
      steps {
        sh '/opt/scripts/build_app.sh'
      }
    }
  }
}
๐Ÿ’ฌ Output:
✅ Jenkins triggered build_app.sh
Build completed successfully.

๐Ÿงฐ 5️⃣ Real Middleware & DevOps Automation Examples

Example 8: NGINX Health Monitor Every 10 Minutes

*/10 * * * * /opt/scripts/nginx_health.sh >> /var/log/nginx_monitor.log 2>&1
๐Ÿ’ฌ Output:
[02:40] ๐ŸŒ NGINX is active
[02:50] ๐Ÿšจ NGINX down – restarted automatically.

Example 9: Database Backup Every 6 Hours

0 */6 * * * mysqldump -u root -pSecret middlewaredb > /backup/db_$(date +%F_%H).sql
๐Ÿ’ฌ Output:
✅ Backup completed: db_2025-11-01_06.sql

Example 10: Restart Jenkins After Crash Detection

*/15 * * * * pgrep jenkins >/dev/null || systemctl restart jenkins
๐Ÿ’ฌ Output:
[03:15] ๐Ÿšจ Jenkins was down, restarted successfully.

Example 11: Auto-Remount Missing Volume

@reboot mount | grep /mnt/backup >/dev/null || mount /dev/sdb1 /mnt/backup
๐Ÿ’ฌ Output:
✅ /mnt/backup auto-mounted on system startup.

๐Ÿง  6️⃣ Tips & Troubleshooting

  • ๐Ÿ“„ Check logs → grep CRON /var/log/syslog
  • ๐Ÿ”’ Ensure permissions → chmod +x /opt/scripts/*.sh
  • ⚙️ Test reboot jobs manually → sudo run-parts /etc/cron.d/
  • ๐Ÿ“ง Use MAILTO in crontab for notifications
  • ๐Ÿšจ For Jenkins jobs, check /var/lib/jenkins/jobs/*/builds

๐Ÿ Summary

  • Cron → Schedules recurring jobs
  • At → Runs one-time tasks
  • @reboot → Executes scripts on startup
  • Jenkins → Manages enterprise-level schedules
  • ๐Ÿ’ก Combine them to create self-healing, auto-starting Middleware environments.

๐Ÿ’กShell Scripting for Beginners – Part 8: Debugging & Logging

  • By now, your scripts deploy apps, manage servers, and clean logs.
  • But what happens when things go wrong?
  • Welcome to Debugging & Logging, where we make your scripts smarter and traceable like a pro DevOps engineer! ๐Ÿ•ต️‍♂️

๐Ÿ“‘ Table of Contents


๐Ÿž 1️⃣ Debugging with set -x

set -x shows every command the script executes — perfect for troubleshooting silent errors.

#!/bin/bash
set -x  # enable debugging
echo "Starting deployment..."
mkdir /opt/app/deploy
cp /tmp/app.war /opt/app/deploy/
set +x  # disable debugging
echo "Deployment finished!"
๐Ÿ’ฌ Output:
+ echo "Starting deployment..."
Starting deployment...
+ mkdir /opt/app/deploy
+ cp /tmp/app.war /opt/app/deploy/
+ echo "Deployment finished!"
Deployment finished!
๐Ÿ’ก Use bash -x script.sh to debug a script without editing it.

๐Ÿชค 2️⃣ Using trap to Handle Failures

The trap command catches unexpected exits (Ctrl+C, errors) and executes custom recovery code. Perfect for cleanup or service restarts.

#!/bin/bash
trap 'echo "⚠️ Script interrupted! Cleaning up..."; rm -f /tmp/tempfile' EXIT
echo "Creating temporary file..."
touch /tmp/tempfile
echo "Simulating work..."
sleep 3
exit 1  # simulate failure
๐Ÿ’ฌ Output:
Creating temporary file...
Simulating work...
⚠️ Script interrupted! Cleaning up...
๐Ÿ’ก Always trap EXIT or ERR to prevent orphan temp files or hanging processes.

๐Ÿšช 3️⃣ Exit Codes & Error Messages

Every command returns an exit code — 0 means success, non-zero means failure.

#!/bin/bash
cp /tmp/app.war /opt/app/deploy/
if [ $? -ne 0 ]; then
  echo "❌ Copy failed! Check file path or permissions."
  exit 1
else
  echo "✅ Copy successful!"
fi
๐Ÿ’ฌ Output:
❌ Copy failed! Check file path or permissions.
or
✅ Copy successful!
Use echo $? after any command to view its exit code.

๐Ÿงพ 4️⃣ Logging with Syslog & Custom Files

Example 1: Using logger Command

#!/bin/bash
logger "Starting Middleware backup..."
tar -czf /tmp/backup.tar.gz /opt/middleware/ || logger "Backup failed!"
logger "Middleware backup completed!"
๐Ÿ’ฌ Syslog Output (in /var/log/syslog):
Nov 1 02:15:00 server logger: Starting Middleware backup...
Nov 1 02:15:03 server logger: Middleware backup completed!

Example 2: Redirecting Logs to a File

#!/bin/bash
exec >> /opt/logs/deploy.log 2>&1
echo "[$(date)] Starting deployment..."
echo "[$(date)] Copying WAR..."
cp /tmp/app.war /opt/tomcat/webapps/
echo "[$(date)] Deployment done ✅"
๐Ÿ’ฌ deploy.log Content:
[Sat Nov 1 02:20:00 2025] Starting deployment...
[Sat Nov 1 02:20:01 2025] Copying WAR...
[Sat Nov 1 02:20:02 2025] Deployment done ✅

⚙️ 5️⃣ Real Middleware & DevOps Examples

Example 1: Jenkins Job Failure Recovery

#!/bin/bash
set -e
trap 'echo "๐Ÿšจ Jenkins build failed, restarting service..."; systemctl restart jenkins' ERR
curl -s -X POST "http://localhost:8080/job/myjob/build" || false
echo "✅ Jenkins job triggered successfully!"
๐Ÿ’ฌ Output:
๐Ÿšจ Jenkins build failed, restarting service...
✅ Jenkins job triggered successfully!

Example 2: Tomcat Log Watcher with Debug Info

#!/bin/bash
set -x
tail -Fn0 /opt/tomcat/logs/catalina.out | \
while read line; do
  echo "$line" | grep "ERROR" &>/dev/null && echo "๐Ÿ”ฅ Found error: $line" >> /opt/logs/tomcat_error.log
done
๐Ÿ’ฌ Output:
[DEBUG] + grep "ERROR" catalina.out
๐Ÿ”ฅ Found error: Database connection timeout

Example 3: NGINX Health Check with Logging

#!/bin/bash
while true; do
  if ! systemctl is-active nginx >/dev/null; then
    echo "$(date): ๐Ÿšจ NGINX is down, restarting..." >> /var/log/nginx_monitor.log
    systemctl restart nginx
  fi
  sleep 10
done
๐Ÿ’ฌ Log Output:
Sat Nov 1 03:00:00 2025: ๐Ÿšจ NGINX is down, restarting...

๐Ÿ Summary

  • set -x → Debug command execution flow.
  • trap → Catch exits & clean up automatically.
  • exit codes → Detect and handle failures gracefully.
  • logger → Send messages to syslog or custom logs.
  • ๐Ÿ’ก Always include logging in production-ready scripts for audit and rollback.

๐Ÿ“ Shell Scripting for Beginners – Part 7: File Handling in Shell.

  • In Middleware & DevOps automation, file handling is everywhere — reading logs, writing reports, rotating backups, or watching live output.
  • Let's learn how to read, write, append, and monitor files in shell scripts with real-world Middleware examples.

๐Ÿ“‘ Table of Contents


๐Ÿ“– 1️⃣ Reading Files

Example 1: Read File Line by Line

#!/bin/bash
filename="/opt/scripts/serverlist.txt"
while read -r line; do
  echo "Processing server: $line"
done < "$filename"
๐Ÿ’ฌ Output:
Processing server: app1
Processing server: app2
Processing server: app3

Example 2: Display Tomcat Log Errors

#!/bin/bash
grep "ERROR" /opt/tomcat/logs/catalina.out
๐Ÿ’ฌ Output:
ERROR [localhost-startStop-1] ... Application failed to start
ERROR [main] ... Database connection refused
๐Ÿ’ก Tip: Use grep -i error to ignore case.

✍️ 2️⃣ Writing & Appending Files

Example 3: Write Output to File

#!/bin/bash
echo "Backup started at $(date)" > /opt/scripts/backup.log
echo "Compressing files..." >> /opt/scripts/backup.log
tar -czf /tmp/backup.tar.gz /opt/data && echo "Backup successful" >> /opt/scripts/backup.log
๐Ÿ’ฌ Output File Content:
Backup started at Sat Nov 1 02:00:00 IST 2025
Compressing files...
Backup successful

Example 4: Append Deployment Status

#!/bin/bash
env=$1
status=$2
echo "$(date) - $env deployment $status" >> /opt/scripts/deploy_history.log
๐Ÿ’ฌ Output File Content:
Sat Nov 1 02:15:00 IST 2025 - dev deployment SUCCESS
Sat Nov 1 02:17:00 IST 2025 - prod deployment FAILED

Example 5: Log WebSphere Cleanup Activity

#!/bin/bash
logfile="/opt/IBM/WebSphere/AppServer/logs/cleanup.log"
echo "$(date): Cleaning old logs..." >> $logfile
find /opt/IBM/WebSphere/AppServer/logs -type f -mtime +7 -delete && echo "✅ Logs cleaned" >> $logfile
๐Ÿ’ฌ Output File:
Sat Nov 1 02:30:00 IST 2025: Cleaning old logs...
✅ Logs cleaned

๐Ÿ” 3️⃣ Monitoring Log Files

Example 6: Live Tail of Tomcat Logs

#!/bin/bash
echo "Monitoring Tomcat logs..."
tail -f /opt/tomcat/logs/catalina.out
๐Ÿ’ฌ Output:
Monitoring Tomcat logs...
[INFO] Server startup completed in 2200 ms
[WARN] Deprecated API in use
[ERROR] Connection timeout to DB

Example 7: Auto-restart Jenkins on Log Failure

#!/bin/bash
tail -Fn0 /var/log/jenkins/jenkins.log | \
while read line; do
  echo "$line" | grep "ERROR" &>/dev/null
  if [ $? = 0 ]; then
    echo "๐Ÿšจ Error detected, restarting Jenkins..."
    systemctl restart jenkins
  fi
done
๐Ÿ’ฌ Output:
[ERROR] Jenkins job queue stuck
๐Ÿšจ Error detected, restarting Jenkins...

Example 8: Monitor NGINX Access Logs for High Load

#!/bin/bash
tail -n 100 /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c
๐Ÿ’ฌ Output:
500 3
404 7
200 90
301 5
๐Ÿ’ก Tip: Use this for quick traffic analysis on your Middleware or DevOps pipeline servers.

⚠️ 4️⃣ Error Handling & Debugging

  • Use set -e to exit on first failure.
  • Redirect errors → 2>> error.log to capture exceptions.
  • Validate file existence before reading:
if [ ! -f "$filename" ]; then
  echo "❌ File not found: $filename"
  exit 1
fi
If your script suddenly exits, check for missing files or permission issues in /var/log/messages or your custom error log.

๐Ÿ Summary

  • Read files line by line to automate configuration or host processing.
  • Write and append logs for auditing and historical tracking.
  • Monitor live logs (Tomcat, Jenkins, NGINX) using tail -f.
  • Add error handling to avoid silent file issues in production.

๐Ÿ“ฆ Shell Scripting for Beginners – Part 6: Arrays & Arguments

Arrays and arguments give your scripts flexibility — perfect for automating Middleware & DevOps tasks like restarting Tomcat across servers, cleaning WebSphere logs, or passing dynamic inputs to Jenkins jobs.


๐Ÿ“‘ Table of Contents


1️⃣ Arrays in Shell

๐ŸŽฏ Example 1 – Basic Array

#!/bin/bash
servers=("dev" "stage" "prod")
echo "First server: ${servers[0]}"
echo "All servers: ${servers[@]}"
๐Ÿ’ฌ Output:
First server: dev
All servers: dev stage prod

๐Ÿ” Example 2 – Loop through Array

#!/bin/bash
apps=("Tomcat" "WebSphere" "JBoss")
for app in "${apps[@]}"; do
  echo "Managing $app service..."
done
๐Ÿ’ฌ Output:
Managing Tomcat service...
Managing WebSphere service...
Managing JBoss service...

๐Ÿงฐ Example 3 – Restart Tomcat on Multiple Servers

#!/bin/bash
servers=("app1" "app2" "app3")
for s in "${servers[@]}"; do
  echo "Restarting Tomcat on $s..."
  ssh $s "systemctl restart tomcat" || echo "⚠️ Restart failed on $s"
done
echo "✅ Restart loop completed."
๐Ÿ’ฌ Output:
Restarting Tomcat on app1...
Restarting Tomcat on app2...
⚠️ Restart failed on app3
✅ Restart loop completed.

๐Ÿงน Example 4 – Cleanup WebSphere Logs

#!/bin/bash
logs=(SystemOut.log SystemErr.log activity.log)
for file in "${logs[@]}"; do
  echo "Deleting $file..."
  rm -f /opt/IBM/WebSphere/AppServer/logs/$file || echo "⚠️ Could not remove $file"
done
echo "๐Ÿงน Old WebSphere logs removed."
๐Ÿ’ฌ Output:
Deleting SystemOut.log...
Deleting SystemErr.log...
Deleting activity.log...
๐Ÿงน Old WebSphere logs removed.

2️⃣ Command-line Arguments

๐Ÿ“ฆ Example 5 – Access Arguments

#!/bin/bash
echo "Environment: $1"
echo "Version: $2"
๐Ÿ’ฌ Output:
$ ./deploy.sh prod v2.0
Environment: prod
Version: v2.0

๐Ÿš€ Example 6 – Trigger Jenkins Job

#!/bin/bash
job_name=$1
if [ -z "$job_name" ]; then
  echo "❌ Usage: $0 "
  exit 1
fi
curl -s -X POST "http://localhost:8080/job/$job_name/build?token=deploy123" || echo "⚠️ Failed to trigger job."
๐Ÿ’ฌ Output:
$ ./jenkins_trigger.sh BuildApp
✅ Triggered Jenkins job BuildApp

๐Ÿง  Example 7 – Argument Validation

#!/bin/bash
if [ $# -lt 2 ]; then
  echo "Usage: $0  "
  exit 1
fi
echo "Deploying $1 version $2..."
๐Ÿ’ฌ Output:
$ ./deploy.sh myapp v1.5
Deploying myapp version v1.5...

3️⃣ Combining Arrays & Arguments

๐ŸŒ Example 8 – Check NGINX on Multiple Hosts

#!/bin/bash
hosts=("$@")
for h in "${hosts[@]}"; do
  echo "Checking NGINX on $h..."
  ssh $h "systemctl is-active nginx" >/dev/null 2>&1 && echo "✅ $h OK" || echo "❌ $h DOWN"
done
๐Ÿ’ฌ Output:
$ ./check_nginx.sh host1 host2 host3
✅ host1 OK
✅ host2 OK
❌ host3 DOWN

๐Ÿ’พ Example 9 – Query Multiple Databases

#!/bin/bash
dbs=("salesdb" "hrdb" "logdb")
for db in "${dbs[@]}"; do
  echo "Checking $db connection..."
  mysql -h dbserver -u admin -psecret -e "use $db;" >/dev/null 2>&1 && echo "✅ $db reachable" || echo "⚠️ $db not reachable"
done
๐Ÿ’ฌ Output:
Checking salesdb connection... ✅ salesdb reachable
Checking hrdb connection... ⚠️ hrdb not reachable
Checking logdb connection... ✅ logdb reachable

๐Ÿงฉ Example 10 – CI/CD Pipeline Trigger

#!/bin/bash
branches=("main" "dev" "hotfix")
for b in "${branches[@]}"; do
  echo "Triggering pipeline for $b..."
  curl -s -X POST "https://ci.middlewarebox.com/job/build-$b/build" || echo "⚠️ Failed for $b"
done
๐Ÿ’ฌ Output:
Triggering pipeline for main... ✅ Done
Triggering pipeline for dev... ✅ Done
Triggering pipeline for hotfix... ⚠️ Failed for hotfix

4️⃣ Error Handling & Debugging

  • Use set -x to trace executed commands.
  • Check argument count with $#.
  • Use "$@" to safely pass all args to loops.
  • Handle SSH/curl/mysql failures with || and echo messages.
  • Log outputs → script.sh >> /var/log/script.log 2>&1

๐Ÿ Summary

  • Arrays – store lists of servers, apps, or files.
  • Arguments – pass values into scripts for dynamic execution.
  • Combine both for multi-server automation.
  • Always include error handling to avoid silent failures.

⚙️ Shell Scripting for Beginners – Part 5: Functions in Shell (Reusable Scripts)

  • Functions let you organize and reuse code in your shell scripts like a Middleware and DevOps engineer.
  • Instead of repeating logic for Tomcat restarts or Jenkins backups, define it once — then call it from anywhere.
  • We'll also explore how to handle errors gracefully inside functions.

๐Ÿ“‘ Table of Contents


๐Ÿ’ก What is a Function?

A function groups commands under one name so you can call them repeatedly. In Middleware automation, you can define restart, deploy, or monitor tasks as reusable functions.


๐Ÿงฉ Function Syntax

function_name() {
   commands
}
# OR
function function_name {
   commands
}

๐ŸŽฏ Example 1: Basic Function

#!/bin/bash
say_hello() {
  echo "Hello from MiddlewareBox!"
}
say_hello
๐Ÿ’ฌ Output:
Hello from MiddlewareBox!

๐Ÿง  Example 2: Function with Parameters

#!/bin/bash
greet_user() {
  echo "Welcome $1! You are working on the $2 environment."
}
greet_user Pradeep DevOps
๐Ÿ’ฌ Output:
Welcome Pradeep! You are working on the DevOps environment.

๐Ÿ“ฆ Example 3: Function with Return Value

#!/bin/bash
add_numbers() {
  sum=$(( $1 + $2 ))
  echo $sum
}
result=$(add_numbers 25 30)
echo "The total is: $result"
๐Ÿ’ฌ Output:
The total is: 55

๐Ÿงฐ Example 4: Restart Tomcat (with error handling)

#!/bin/bash
restart_tomcat() {
  echo "Stopping Tomcat..."
  /opt/tomcat/bin/shutdown.sh || { echo "❌ Failed to stop Tomcat"; return 1; }
  sleep 3
  echo "Starting Tomcat..."
  /opt/tomcat/bin/startup.sh || { echo "❌ Failed to start Tomcat"; return 2; }
  echo "✅ Tomcat restarted successfully!"
}
restart_tomcat
๐Ÿ’ฌ Output:
Stopping Tomcat...
Starting Tomcat...
✅ Tomcat restarted successfully!
If Tomcat fails to start, check /opt/tomcat/logs/catalina.out.

๐Ÿ’พ Example 5: Jenkins Backup Function (with error handling)

#!/bin/bash
backup_jenkins() {
  backup_dir="/var/jenkins_home/backup"
  mkdir -p $backup_dir || { echo "❌ Failed to create backup directory"; return 1; }
  tar -czf $backup_dir/jenkins_$(date +%F).tar.gz /var/jenkins_home || { echo "❌ Backup failed"; return 2; }
  echo "✅ Backup saved in $backup_dir"
}
backup_jenkins
If backup fails, ensure enough disk space and Jenkins home path is correct.

๐Ÿงน Example 6: WebSphere Log Cleanup

#!/bin/bash
cleanup_was_logs() {
  log_path="/opt/IBM/WebSphere/AppServer/logs"
  echo "Cleaning logs in $log_path"
  find $log_path -name "*.log" -type f -mtime +7 -exec rm -f {} \; || echo "⚠️ Cleanup error occurred"
  echo "๐Ÿงน Old WebSphere logs removed successfully."
}
cleanup_was_logs
If permissions fail, run with sudo or check AppServer/logs ownership.

๐Ÿง  Example 7: Database Health Check

#!/bin/bash
check_db() {
  if nc -zv dbserver 3306 >/dev/null 2>&1; then
    echo "✅ Database reachable"
  else
    echo "๐Ÿšจ Database not reachable!"
    return 1
  fi
}
check_db || echo "⚙️ Please verify DB server or firewall."

๐Ÿš€ Example 8: Modular CI/CD Wrapper

#!/bin/bash
deploy_app() {
  echo "Building app..."
  mvn clean package || { echo "❌ Build failed"; return 1; }
  echo "Deploying WAR..."
  scp target/app.war prod:/opt/tomcat/webapps/ || { echo "❌ Deploy failed"; return 2; }
  ssh prod "systemctl restart tomcat"
  echo "✅ Deployment complete!"
}
deploy_app
If deploy fails, verify SSH connectivity and WAR path.

๐Ÿ Summary

  • Functions make scripts modular, reusable, and easier to maintain.
  • Handle errors using return and conditional checks.
  • Great for DevOps: automate Jenkins, Tomcat, WebSphere, and CI/CD pipelines.
  • Combine with loops and conditionals from previous lessons for full automation.

๐Ÿ” Shell Scripting for Beginners – Part 4: Loops in Shell (for, while, until)

  • In this part, you'll learn how loops help automate Middleware & DevOps tasks — from restarting Tomcat, cleaning WebSphere logs, checking NGINX uptime, or retrying Jenkins jobs.
  • We'll also cover how to detect and handle errors inside loops ๐Ÿ’ฃ so your scripts don't fail silently.

๐Ÿ“‘ Table of Contents


1️⃣ The for Loop

The for loop executes a set of commands repeatedly for a list of items — servers, log files, users, etc. Perfect for middleware jobs like mass restarts or file cleanups.

๐ŸŽฏ Example 1: Basic For Loop

#!/bin/bash
for i in 1 2 3 4 5
do
  echo "Iteration number: $i"
done
๐Ÿ’ฌ Output:
Iteration number: 1
Iteration number: 2
...
Iteration number: 5

๐Ÿ’ผ Example 2: Rotate WebSphere Logs

#!/bin/bash
log_dir="/opt/IBM/WebSphere/AppServer/logs"
for file in $log_dir/*.log
do
  echo "Archiving: $file"
  gzip "$file" || echo "⚠️ Failed to compress $file"
done
echo "✅ WebSphere logs rotated successfully!"
๐Ÿ’ฌ Output:
Archiving: SystemOut.log
Archiving: SystemErr.log
✅ WebSphere logs rotated successfully!
⚠️ If a log file is locked by WebSphere, gzip fails. Use lsof $file to check which process is holding it.

๐Ÿš€ Example 3: Deploy WAR to Multiple Tomcat Servers

#!/bin/bash
servers=("devtomcat" "testtomcat" "prodtomcat")
for srv in "${servers[@]}"
do
  echo "Deploying to $srv..."
  scp myapp.war $srv:/opt/tomcat/webapps/ || {
    echo "❌ Deployment failed on $srv, skipping..."
    continue
  }
  ssh $srv "systemctl restart tomcat"
done
echo "✅ Deployment completed."
๐Ÿ’ฌ Output:
Deploying to devtomcat...
Deploying to testtomcat...
✅ Deployment completed.
❌ If SSH fails or disk is full, the continue keyword skips to the next server safely.

๐Ÿ’ฃ Example 4: Handling Command Failures

#!/bin/bash
set -e
for s in server1 server2
do
  echo "Checking $s..."
  ssh $s "uptime" || { echo "⚠️ Error connecting to $s"; continue; }
done
echo "✅ All servers checked."
๐Ÿง  Use set -e to exit immediately on error or || continue to skip failed items.

2️⃣ The while Loop

The while loop runs as long as a condition remains true — great for service monitoring and retries.

๐Ÿงฎ Example 5: Simple Counter

#!/bin/bash
count=1
while [ $count -le 5 ]
do
  echo "Count = $count"
  ((count++))
done

๐Ÿง  Example 6: Tomcat Health Check

#!/bin/bash
while true
do
  if pgrep -f "org.apache.catalina.startup.Bootstrap" >/dev/null; then
    echo "✅ Tomcat running fine."
  else
    echo "๐Ÿšจ Tomcat stopped! Restarting..."
    /opt/tomcat/bin/startup.sh || echo "❌ Failed to restart Tomcat."
  fi
  sleep 10
done
๐Ÿ’ก Use pgrep + startup.sh for continuous checks. If restart fails, check logs in /opt/tomcat/logs/catalina.out.

๐ŸŒ Example 7: Auto-Restart NGINX if Down

#!/bin/bash
while true
do
  systemctl is-active nginx >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    echo "๐Ÿšจ NGINX down! Restarting..."
    systemctl restart nginx || echo "⚠️ Failed to restart NGINX!"
  else
    echo "๐ŸŒ NGINX OK"
  fi
  sleep 15
done
⚙️ If systemctl restart fails, check journal logs: journalctl -u nginx -xe

3️⃣ The until Loop

The until loop runs repeatedly until a condition becomes true. Best used to wait for Jenkins, DB, or a service to be ready.

๐Ÿงฉ Example 8: Wait for Jenkins to Start

#!/bin/bash
until curl -s http://localhost:8080 >/dev/null
do
  echo "Waiting for Jenkins to start..."
  sleep 5
done
echo "✅ Jenkins is running!"
๐Ÿ•’ If it loops forever, verify Jenkins logs with tail -f /var/log/jenkins/jenkins.log.

๐Ÿ’พ Example 9: Wait for Database Connection

#!/bin/bash
until nc -zv dbserver 3306 >/dev/null 2>&1
do
  echo "⏳ Waiting for DB..."
  sleep 5
done
echo "✅ Database reachable!"
If connection keeps failing, check DB firewall or credentials in .env files.

๐Ÿงฑ Example 10: Wait Until Backup Completes

#!/bin/bash
until [ -f /opt/backups/backup_done.flag ]
do
  echo "Backup still running..."
  sleep 10
done
echo "✅ Backup completed successfully!"

๐Ÿง  Debugging & Error Handling Tips

  • Use set -x at top to debug each executed line.
  • Use || echo "Command failed" for inline failure tracking.
  • Use continue inside loops to skip failed items safely.
  • Use exit 1 if you want to terminate on critical errors.
  • Always log outputs: command >> /var/log/script.log 2>&1

๐Ÿ Summary

  • for loop – Repeats tasks for lists (servers, files).
  • while loop – Keeps running while condition is true.
  • until loop – Keeps running until condition becomes true.
  • Error Handling – Use set -e, $?, ||, and continue for reliability.

๐Ÿ”ฅ Loops make your middleware automation resilient — whether it’s Tomcat restarts, WebSphere cleanups, or NGINX monitoring. Next up → Part 5: Shell Functions – Make Your Scripts Modular & Reusable ⚙️

๐ŸงฉShell Scripting for Beginners – Part 3: Conditional Statements (if, elif, else)

๐Ÿšฆ Shell Scripting for Beginners – Part 3: Conditional Statements (if, elif, else)

Welcome to Part 3 of the MiddlewareBox Shell Scripting Series! In this chapter, we’ll learn how your script can make decisions using if, elif, and else. These are the “logic gates” of automation. ๐Ÿšฆ


๐Ÿ“‘ Table of Contents


๐Ÿ’ก Basic Syntax

if [ condition ]; then
   commands
elif [ another_condition ]; then
   commands
else
   commands
fi
๐Ÿ’ฌ Concept:
✅ True → runs block; ❌ False → moves to next or else.

๐ŸŽฏ Example 1: Simple Number Check

#!/bin/bash
read -p "Enter a number: " num
if [ $num -gt 0 ]; then
  echo "Positive number"
elif [ $num -lt 0 ]; then
  echo "Negative number"
else
  echo "Zero"
fi
๐Ÿ’ฌ Output:
Enter a number: -5
Negative number

⚙️ Example 2: Check File Existence

#!/bin/bash
read -p "Enter file name: " file
if [ -f "$file" ]; then
  echo "File exists ✅"
else
  echo "File not found ❌"
fi
๐Ÿ’ฌ Output:
Enter file name: test.sh
File exists ✅

๐Ÿ“ฆ Example 3: Directory Check

#!/bin/bash
read -p "Enter directory path: " dir
if [ -d "$dir" ]; then
  echo "Directory exists ๐Ÿ“"
else
  echo "Directory not found ๐Ÿšซ"
fi
๐Ÿ’ฌ Output:
Enter directory path: /opt/tomcat
Directory exists ๐Ÿ“

๐Ÿง  Example 4: Check User Login

#!/bin/bash
read -p "Enter username: " user
if id "$user" &>/dev/null; then
  echo "User $user exists ๐Ÿ‘ค"
else
  echo "User $user not found ⚠️"
fi
๐Ÿ’ฌ Output:
Enter username: devops
User devops exists ๐Ÿ‘ค

๐Ÿงฐ Example 5: Tomcat Status Checker

#!/bin/bash
pid=$(pgrep -f "org.apache.catalina.startup.Bootstrap")
if [ -n "$pid" ]; then
  echo "Tomcat is running (PID: $pid) ✅"
else
  echo "Tomcat is not running ❌"
fi
๐Ÿ’ฌ Output:
Tomcat is running (PID: 2145) ✅

๐Ÿ’พ Example 6: Disk Space Warning

#!/bin/bash
use=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
if [ $use -ge 90 ]; then
  echo "๐Ÿšจ Disk almost full: ${use}%"
elif [ $use -ge 70 ]; then
  echo "⚠️ Disk usage high: ${use}%"
else
  echo "✅ Disk usage normal: ${use}%"
fi
๐Ÿ’ฌ Output:
⚠️ Disk usage high: 75%

๐Ÿš€ Example 7: Deployment Environment Logic

#!/bin/bash
read -p "Enter environment (dev/test/prod): " env
if [ "$env" == "dev" ]; then
  echo "Deploying to Development ๐Ÿšง"
elif [ "$env" == "test" ]; then
  echo "Deploying to QA ๐Ÿงช"
elif [ "$env" == "prod" ]; then
  echo "Deploying to Production ⚙️"
else
  echo "Invalid environment ❌"
fi
๐Ÿ’ฌ Output:
Enter environment (dev/test/prod): prod
Deploying to Production ⚙️

๐Ÿ“ Example 8: Jenkins Job Success Check

#!/bin/bash
status=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/job/myjob/lastBuild/api/json)
if [ "$status" == "200" ]; then
  echo "✅ Jenkins job ran successfully."
else
  echo "❌ Jenkins job failed or not accessible."
fi
๐Ÿ’ฌ Output:
✅ Jenkins job ran successfully.

๐Ÿงฉ Example 9: Server Health Monitor

#!/bin/bash
ping -c1 google.com &>/dev/null
if [ $? -eq 0 ]; then
  echo "๐ŸŒ Internet reachable."
else
  echo "๐Ÿšซ No network connection."
fi
๐Ÿ’ฌ Output:
๐ŸŒ Internet reachable.

⚠️ What Happens When a Command Fails?

Every command returns an exit code: 0 → success ✅ | non-zero → failure ❌ $? stores this code. if uses it automatically to decide what to do next.

๐Ÿง  Example 10: File Read Failure Handling

#!/bin/bash
cat /etc/nofile &>/dev/null
if [ $? -eq 0 ]; then
  echo "✅ File read successful"
else
  echo "❌ Command failed with exit code $?"
fi
๐Ÿ’ฌ Output:
❌ Command failed with exit code 1

๐Ÿงฐ Example 11: Jenkins Trigger Failure

#!/bin/bash
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/job/demo/build
if [ $? -eq 0 ]; then
  echo "✅ Jenkins triggered."
else
  echo "๐Ÿšจ Jenkins trigger failed. Check server or credentials."
fi
๐Ÿ’ฅ Output (Jenkins down):
๐Ÿšจ Jenkins trigger failed. Check server or credentials.

๐Ÿ’พ Example 12: Stop Script on Any Failure (set -e)

#!/bin/bash
set -e
echo "Step 1: Creating directory..."
mkdir /tmp/demo_dir
echo "Step 2: Moving file..."
mv /file/does/not/exist /tmp/demo_dir
echo "✅ Script completed."
๐Ÿ’ฌ Output:
bash: mv: No such file or directory
Script exits immediately (set -e stops execution)

๐Ÿ Conclusion

  • if → runs when condition succeeds (exit 0)
  • elif → alternate logic branch
  • else → fallback on failure
  • $? & set -e → help handle failures cleanly

๐ŸŽฏ You now understand conditional logic like a DevOps pro!