Menu

Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

2 Nov 2025

🐧 Linux Shell Scripting for Beginners – Complete Tutorial Series.

  • A 13-part practical shell scripting course for DevOps and Middleware Engineers.

💡 How This Tutorial Series Will Help You

  • ✔ Build a strong foundation in Linux Shell scripting — from basics to automation.
  • ✔ Automate daily Middleware and DevOps tasks: monitoring, backups, and log management.
  • ✔ Write scripts for Middleware platforms like Tomcat, Jenkins, and WebSphere.
  • ✔ Debug, schedule, and manage production-ready shell scripts confidently.
  • ✔ Perfect for Sysadmins, Middleware, and DevOps Engineers upgrading to Cloud roles.

📚 Complete Course Index

🐚 Part 1: What is a Variable?
Learn how variables store data and simplify automation scripts. Foundation of all shell logic.
Make interactive scripts using read command with Jenkins and Tomcat examples.
Use decision-making in your scripts with examples checking files, services, and network health.
Automate repetitive DevOps tasks like log cleanup, backups, and status monitoring.
Write modular, reusable functions — restart Tomcat or back up Jenkins in one click.
📦 Part 6: Arrays & Arguments
Store multiple values, manage user inputs, and process server lists in scripts.
Learn how to read, write, append, and handle configuration or log files safely.
💡 Part 8: Debugging & Logging
Find and fix script errors, use logs, and enable debug mode for safe execution.
Use cron and @reboot jobs to run scripts automatically for health checks or backups.
Create a production-ready monitoring script for Tomcat, Jenkins, and NGINX with auto-restart.
Automate log rotation with logrotate and custom retention scripts for Middleware systems.
Perform automated backup and restore for Jenkins, Tomcat, and WebSphere with rollback support.
Run multiple jobs in parallel, monitor background processes, and speed up automation workflows.

🚀 Why MiddlewareBox Shell Scripting Series?

  • ✅ Step-by-step, beginner-friendly explanations.
  • ✅ Each topic includes real DevOps & Middleware context.
  • ✅ Examples with Output and Error handling.
  • ✅ Simple language for better understanding.

✨ Start Learning Now → Part 1: What is a Variable?
& build your own scripts from scratch! 💪

1 Nov 2025

⚙️ Shell Scripting for Beginners – Part 5: Functions in Shell (Reusable Scripts)

  • Functions let you organize and reuse code in your shell scripts like a Middleware and DevOps engineer.
  • Instead of repeating logic for Tomcat restarts or Jenkins backups, define it once — then call it from anywhere.
  • We'll also explore how to handle errors gracefully inside functions.

📑 Table of Contents


💡 What is a Function?

A function groups commands under one name so you can call them repeatedly. In Middleware automation, you can define restart, deploy, or monitor tasks as reusable functions.


🧩 Function Syntax

function_name() {
   commands
}
# OR
function function_name {
   commands
}

🎯 Example 1: Basic Function

#!/bin/bash
say_hello() {
  echo "Hello from MiddlewareBox!"
}
say_hello
💬 Output:
Hello from MiddlewareBox!

🧠 Example 2: Function with Parameters

#!/bin/bash
greet_user() {
  echo "Welcome $1! You are working on the $2 environment."
}
greet_user Pradeep DevOps
💬 Output:
Welcome Pradeep! You are working on the DevOps environment.

📦 Example 3: Function with Return Value

#!/bin/bash
add_numbers() {
  sum=$(( $1 + $2 ))
  echo $sum
}
result=$(add_numbers 25 30)
echo "The total is: $result"
💬 Output:
The total is: 55

🧰 Example 4: Restart Tomcat (with error handling)

#!/bin/bash
restart_tomcat() {
  echo "Stopping Tomcat..."
  /opt/tomcat/bin/shutdown.sh || { echo "❌ Failed to stop Tomcat"; return 1; }
  sleep 3
  echo "Starting Tomcat..."
  /opt/tomcat/bin/startup.sh || { echo "❌ Failed to start Tomcat"; return 2; }
  echo "✅ Tomcat restarted successfully!"
}
restart_tomcat
💬 Output:
Stopping Tomcat...
Starting Tomcat...
✅ Tomcat restarted successfully!
If Tomcat fails to start, check /opt/tomcat/logs/catalina.out.

💾 Example 5: Jenkins Backup Function (with error handling)

#!/bin/bash
backup_jenkins() {
  backup_dir="/var/jenkins_home/backup"
  mkdir -p $backup_dir || { echo "❌ Failed to create backup directory"; return 1; }
  tar -czf $backup_dir/jenkins_$(date +%F).tar.gz /var/jenkins_home || { echo "❌ Backup failed"; return 2; }
  echo "✅ Backup saved in $backup_dir"
}
backup_jenkins
If backup fails, ensure enough disk space and Jenkins home path is correct.

🧹 Example 6: WebSphere Log Cleanup

#!/bin/bash
cleanup_was_logs() {
  log_path="/opt/IBM/WebSphere/AppServer/logs"
  echo "Cleaning logs in $log_path"
  find $log_path -name "*.log" -type f -mtime +7 -exec rm -f {} \; || echo "⚠️ Cleanup error occurred"
  echo "🧹 Old WebSphere logs removed successfully."
}
cleanup_was_logs
If permissions fail, run with sudo or check AppServer/logs ownership.

🧠 Example 7: Database Health Check

#!/bin/bash
check_db() {
  if nc -zv dbserver 3306 >/dev/null 2>&1; then
    echo "✅ Database reachable"
  else
    echo "🚨 Database not reachable!"
    return 1
  fi
}
check_db || echo "⚙️ Please verify DB server or firewall."

🚀 Example 8: Modular CI/CD Wrapper

#!/bin/bash
deploy_app() {
  echo "Building app..."
  mvn clean package || { echo "❌ Build failed"; return 1; }
  echo "Deploying WAR..."
  scp target/app.war prod:/opt/tomcat/webapps/ || { echo "❌ Deploy failed"; return 2; }
  ssh prod "systemctl restart tomcat"
  echo "✅ Deployment complete!"
}
deploy_app
If deploy fails, verify SSH connectivity and WAR path.

🏁 Summary

  • Functions make scripts modular, reusable, and easier to maintain.
  • Handle errors using return and conditional checks.
  • Great for DevOps: automate Jenkins, Tomcat, WebSphere, and CI/CD pipelines.
  • Combine with loops and conditionals from previous lessons for full automation.

🔁 Shell Scripting for Beginners – Part 4: Loops in Shell (for, while, until)

  • In this part, you'll learn how loops help automate Middleware & DevOps tasks — from restarting Tomcat, cleaning WebSphere logs, checking NGINX uptime, or retrying Jenkins jobs.
  • We'll also cover how to detect and handle errors inside loops 💣 so your scripts don't fail silently.

📑 Table of Contents


1️⃣ The for Loop

The for loop executes a set of commands repeatedly for a list of items — servers, log files, users, etc. Perfect for middleware jobs like mass restarts or file cleanups.

🎯 Example 1: Basic For Loop

#!/bin/bash
for i in 1 2 3 4 5
do
  echo "Iteration number: $i"
done
💬 Output:
Iteration number: 1
Iteration number: 2
...
Iteration number: 5

💼 Example 2: Rotate WebSphere Logs

#!/bin/bash
log_dir="/opt/IBM/WebSphere/AppServer/logs"
for file in $log_dir/*.log
do
  echo "Archiving: $file"
  gzip "$file" || echo "⚠️ Failed to compress $file"
done
echo "✅ WebSphere logs rotated successfully!"
💬 Output:
Archiving: SystemOut.log
Archiving: SystemErr.log
✅ WebSphere logs rotated successfully!
⚠️ If a log file is locked by WebSphere, gzip fails. Use lsof $file to check which process is holding it.

🚀 Example 3: Deploy WAR to Multiple Tomcat Servers

#!/bin/bash
servers=("devtomcat" "testtomcat" "prodtomcat")
for srv in "${servers[@]}"
do
  echo "Deploying to $srv..."
  scp myapp.war $srv:/opt/tomcat/webapps/ || {
    echo "❌ Deployment failed on $srv, skipping..."
    continue
  }
  ssh $srv "systemctl restart tomcat"
done
echo "✅ Deployment completed."
💬 Output:
Deploying to devtomcat...
Deploying to testtomcat...
✅ Deployment completed.
❌ If SSH fails or disk is full, the continue keyword skips to the next server safely.

💣 Example 4: Handling Command Failures

#!/bin/bash
set -e
for s in server1 server2
do
  echo "Checking $s..."
  ssh $s "uptime" || { echo "⚠️ Error connecting to $s"; continue; }
done
echo "✅ All servers checked."
🧠 Use set -e to exit immediately on error or || continue to skip failed items.

2️⃣ The while Loop

The while loop runs as long as a condition remains true — great for service monitoring and retries.

🧮 Example 5: Simple Counter

#!/bin/bash
count=1
while [ $count -le 5 ]
do
  echo "Count = $count"
  ((count++))
done

🧠 Example 6: Tomcat Health Check

#!/bin/bash
while true
do
  if pgrep -f "org.apache.catalina.startup.Bootstrap" >/dev/null; then
    echo "✅ Tomcat running fine."
  else
    echo "🚨 Tomcat stopped! Restarting..."
    /opt/tomcat/bin/startup.sh || echo "❌ Failed to restart Tomcat."
  fi
  sleep 10
done
💡 Use pgrep + startup.sh for continuous checks. If restart fails, check logs in /opt/tomcat/logs/catalina.out.

🌍 Example 7: Auto-Restart NGINX if Down

#!/bin/bash
while true
do
  systemctl is-active nginx >/dev/null 2>&1
  if [ $? -ne 0 ]; then
    echo "🚨 NGINX down! Restarting..."
    systemctl restart nginx || echo "⚠️ Failed to restart NGINX!"
  else
    echo "🌐 NGINX OK"
  fi
  sleep 15
done
⚙️ If systemctl restart fails, check journal logs: journalctl -u nginx -xe

3️⃣ The until Loop

The until loop runs repeatedly until a condition becomes true. Best used to wait for Jenkins, DB, or a service to be ready.

🧩 Example 8: Wait for Jenkins to Start

#!/bin/bash
until curl -s http://localhost:8080 >/dev/null
do
  echo "Waiting for Jenkins to start..."
  sleep 5
done
echo "✅ Jenkins is running!"
🕒 If it loops forever, verify Jenkins logs with tail -f /var/log/jenkins/jenkins.log.

💾 Example 9: Wait for Database Connection

#!/bin/bash
until nc -zv dbserver 3306 >/dev/null 2>&1
do
  echo "⏳ Waiting for DB..."
  sleep 5
done
echo "✅ Database reachable!"
If connection keeps failing, check DB firewall or credentials in .env files.

🧱 Example 10: Wait Until Backup Completes

#!/bin/bash
until [ -f /opt/backups/backup_done.flag ]
do
  echo "Backup still running..."
  sleep 10
done
echo "✅ Backup completed successfully!"

🧠 Debugging & Error Handling Tips

  • Use set -x at top to debug each executed line.
  • Use || echo "Command failed" for inline failure tracking.
  • Use continue inside loops to skip failed items safely.
  • Use exit 1 if you want to terminate on critical errors.
  • Always log outputs: command >> /var/log/script.log 2>&1

🏁 Summary

  • for loop – Repeats tasks for lists (servers, files).
  • while loop – Keeps running while condition is true.
  • until loop – Keeps running until condition becomes true.
  • Error Handling – Use set -e, $?, ||, and continue for reliability.

🔥 Loops make your middleware automation resilient — whether it’s Tomcat restarts, WebSphere cleanups, or NGINX monitoring. Next up → Part 5: Shell Functions – Make Your Scripts Modular & Reusable ⚙️

🧩Shell Scripting for Beginners – Part 3: Conditional Statements (if, elif, else)

🚦 Shell Scripting for Beginners – Part 3: Conditional Statements (if, elif, else)

Welcome to Part 3 of the MiddlewareBox Shell Scripting Series! In this chapter, we’ll learn how your script can make decisions using if, elif, and else. These are the “logic gates” of automation. 🚦


📑 Table of Contents


💡 Basic Syntax

if [ condition ]; then
   commands
elif [ another_condition ]; then
   commands
else
   commands
fi
💬 Concept:
✅ True → runs block; ❌ False → moves to next or else.

🎯 Example 1: Simple Number Check

#!/bin/bash
read -p "Enter a number: " num
if [ $num -gt 0 ]; then
  echo "Positive number"
elif [ $num -lt 0 ]; then
  echo "Negative number"
else
  echo "Zero"
fi
💬 Output:
Enter a number: -5
Negative number

⚙️ Example 2: Check File Existence

#!/bin/bash
read -p "Enter file name: " file
if [ -f "$file" ]; then
  echo "File exists ✅"
else
  echo "File not found ❌"
fi
💬 Output:
Enter file name: test.sh
File exists ✅

📦 Example 3: Directory Check

#!/bin/bash
read -p "Enter directory path: " dir
if [ -d "$dir" ]; then
  echo "Directory exists 📁"
else
  echo "Directory not found 🚫"
fi
💬 Output:
Enter directory path: /opt/tomcat
Directory exists 📁

🧠 Example 4: Check User Login

#!/bin/bash
read -p "Enter username: " user
if id "$user" &>/dev/null; then
  echo "User $user exists 👤"
else
  echo "User $user not found ⚠️"
fi
💬 Output:
Enter username: devops
User devops exists 👤

🧰 Example 5: Tomcat Status Checker

#!/bin/bash
pid=$(pgrep -f "org.apache.catalina.startup.Bootstrap")
if [ -n "$pid" ]; then
  echo "Tomcat is running (PID: $pid) ✅"
else
  echo "Tomcat is not running ❌"
fi
💬 Output:
Tomcat is running (PID: 2145) ✅

💾 Example 6: Disk Space Warning

#!/bin/bash
use=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
if [ $use -ge 90 ]; then
  echo "🚨 Disk almost full: ${use}%"
elif [ $use -ge 70 ]; then
  echo "⚠️ Disk usage high: ${use}%"
else
  echo "✅ Disk usage normal: ${use}%"
fi
💬 Output:
⚠️ Disk usage high: 75%

🚀 Example 7: Deployment Environment Logic

#!/bin/bash
read -p "Enter environment (dev/test/prod): " env
if [ "$env" == "dev" ]; then
  echo "Deploying to Development 🚧"
elif [ "$env" == "test" ]; then
  echo "Deploying to QA 🧪"
elif [ "$env" == "prod" ]; then
  echo "Deploying to Production ⚙️"
else
  echo "Invalid environment ❌"
fi
💬 Output:
Enter environment (dev/test/prod): prod
Deploying to Production ⚙️

📁 Example 8: Jenkins Job Success Check

#!/bin/bash
status=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/job/myjob/lastBuild/api/json)
if [ "$status" == "200" ]; then
  echo "✅ Jenkins job ran successfully."
else
  echo "❌ Jenkins job failed or not accessible."
fi
💬 Output:
✅ Jenkins job ran successfully.

🧩 Example 9: Server Health Monitor

#!/bin/bash
ping -c1 google.com &>/dev/null
if [ $? -eq 0 ]; then
  echo "🌍 Internet reachable."
else
  echo "🚫 No network connection."
fi
💬 Output:
🌍 Internet reachable.

⚠️ What Happens When a Command Fails?

Every command returns an exit code: 0 → success ✅ | non-zero → failure ❌ $? stores this code. if uses it automatically to decide what to do next.

🧠 Example 10: File Read Failure Handling

#!/bin/bash
cat /etc/nofile &>/dev/null
if [ $? -eq 0 ]; then
  echo "✅ File read successful"
else
  echo "❌ Command failed with exit code $?"
fi
💬 Output:
❌ Command failed with exit code 1

🧰 Example 11: Jenkins Trigger Failure

#!/bin/bash
curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/job/demo/build
if [ $? -eq 0 ]; then
  echo "✅ Jenkins triggered."
else
  echo "🚨 Jenkins trigger failed. Check server or credentials."
fi
💥 Output (Jenkins down):
🚨 Jenkins trigger failed. Check server or credentials.

💾 Example 12: Stop Script on Any Failure (set -e)

#!/bin/bash
set -e
echo "Step 1: Creating directory..."
mkdir /tmp/demo_dir
echo "Step 2: Moving file..."
mv /file/does/not/exist /tmp/demo_dir
echo "✅ Script completed."
💬 Output:
bash: mv: No such file or directory
Script exits immediately (set -e stops execution)

🏁 Conclusion

  • if → runs when condition succeeds (exit 0)
  • elif → alternate logic branch
  • else → fallback on failure
  • $? & set -e → help handle failures cleanly

🎯 You now understand conditional logic like a DevOps pro!

📥 Shell Scripting for Beginners – Part 2: Reading User Input in Shell

🚀 Part 2: Making Scripts Interactive

  • Using the read command for user input
  • Creating scripts that listen to user responses
  • Essential for real-world scripting and automation

💡 What is the read Command?

The read command accepts user input. Think of it as your script saying — "Hey human, tell me something so I can automate it for you." 😎

read variable_name

Whatever the user types will be stored inside $variable_name.


🎯 Example 1: Basic User Input

#!/bin/bash
echo "What is your name?"
read name
echo "Hello $name, welcome to MiddlewareBox!"
💬 Output:
What is your name?
John
Hello John, welcome to MiddlewareBox!

⚙️ Example 2: Read with Prompt in Same Line

#!/bin/bash
read -p "Enter your environment (dev/stage/prod): " env
echo "Deploying application to $env environment..."
💬 Output:
Enter your environment (dev/stage/prod): dev
Deploying application to dev environment...

🔐 Example 3: Hidden Input (Password)

#!/bin/bash
read -sp "Enter your Jenkins admin password: " password
echo
echo "Password received! (Not displayed for security)"
💬 Output:
Enter your Jenkins admin password: ********
Password received! (Not displayed for security)

⏰ Example 4: Set Timeout for Input

#!/bin/bash
read -t 5 -p "Enter server name within 5 seconds: " server
echo "Server chosen: $server"
💬 Output:
Enter server name within 5 seconds: middlewarebox01
Server chosen: middlewarebox01

📦 Example 5: Default Value if User Skips Input

#!/bin/bash
read -p "Enter build version (default: 1.0): " version
version=${version:-1.0}
echo "Deploying build version $version..."
💬 Output:
Enter build version (default: 1.0):
Deploying build version 1.0...

💾 Example 6: Take Multiple Inputs Together

#!/bin/bash
read -p "Enter username and group: " user group
echo "Creating user: $user in group: $group"
💬 Output:
Enter username and group: devops admins
Creating user: devops in group: admins

🧰 Example 7: Restart Tomcat with User Confirmation

#!/bin/bash
read -p "Do you want to restart Tomcat? (yes/no): " choice
if [[ "$choice" == "yes" ]]; then
  echo "Stopping Tomcat..."
  sh /opt/tomcat/bin/shutdown.sh
  sleep 3
  echo "Starting Tomcat..."
  sh /opt/tomcat/bin/startup.sh
  echo "✅ Tomcat restarted successfully!"
else
  echo "❌ Operation cancelled."
fi
💬 Output:
Do you want to restart Tomcat? (yes/no): yes
Stopping Tomcat...
Starting Tomcat...
✅ Tomcat restarted successfully!

🧠 Example 8: Jenkins Backup Trigger

#!/bin/bash
read -p "Do you want to back up Jenkins data now? (y/n): " answer
if [[ "$answer" == "y" ]]; then
  backup_dir="/var/jenkins_home/backup"
  mkdir -p $backup_dir
  tar -czf $backup_dir/jenkins_backup_$(date +%F).tar.gz /var/jenkins_home
  echo "✅ Jenkins backup created in $backup_dir"
else
  echo "🕒 Backup skipped by user."
fi
💬 Output:
Do you want to back up Jenkins data now? (y/n): y
✅ Jenkins backup created in /var/jenkins_home/backup

🌍 Example 9: Deployment Environment Selector

#!/bin/bash
read -p "Enter deployment environment (dev/test/prod): " env
case $env in
  dev)
    echo "🚀 Deploying to Development servers..."
    ;;
  test)
    echo "🧪 Deploying to Testing servers..."
    ;;
  prod)
    echo "⚠️ Deploying to Production environment..."
    ;;
  *)
    echo "❌ Invalid environment!"
    ;;
esac
💬 Output:
Enter deployment environment (dev/test/prod): prod
⚠️ Deploying to Production environment...

📁 Example 10: Check and Read File Path Before Processing

#!/bin/bash
read -p "Enter the log file path: " logfile
if [[ -f "$logfile" ]]; then
  echo "Analyzing log file..."
  grep -i "error" $logfile | head -5
else
  echo "❌ File not found: $logfile"
fi
💬 Output:
Enter the log file path: /var/log/nginx/error.log
Analyzing log file...
[error] 1054#1054: *1 connect() failed (111: Connection refused)
[error] 1055#1055: *2 open() "/var/www/html/404.html" failed (2: No such file)

🧩 Example 11: Take User Input in a CI/CD Script

#!/bin/bash
read -p "Enter Git branch to deploy: " branch
read -p "Enter version tag: " version
echo "Cloning branch '$branch'..."
git clone -b $branch https://github.com/middlewarebox/myapp.git
echo "Deploying version $version..."
echo "✅ Deployment completed!"
💬 Output:
Enter Git branch to deploy: main
Enter version tag: v2.0
Cloning branch 'main'...
✅ Deployment completed!

🧠 Example 12: User Confirmation for Rollback

#!/bin/bash
read -p "Rollback to previous version? (yes/no): " rollback
if [[ "$rollback" == "yes" ]]; then
  echo "🔁 Rolling back..."
  echo "✅ Rollback completed successfully!"
else
  echo "👍 Staying on current version."
fi
💬 Output:
Rollback to previous version? (yes/no): yes
🔁 Rolling back...
✅ Rollback completed successfully!

🏁 Conclusion

In DevOps and Middleware environments, interactive scripts save time and prevent mistakes.

29 Oct 2025

🐧 Linux Shell Scripting for Beginners – Part 1: What is a Variable?💻

Welcome to this Shell Scripting series! In this first part, we will learn about variables in a very simple and easy-to-understand way.

📑 Table of Contents



🌟 What is a Variable?

A variable is like a small box where you store some information. Later, you can open the box and use that information again in your script.

✏️ How to Create a Variable

name="Pradeep"
course="Shell Scripting"
count=10

Important: There should be no space before or after the = sign.

❌ Wrong

name = "Pradeep"

✅ Correct

name="Pradeep"

📢 How to Use a Variable

echo "Hello $name, welcome to $course training!"
Hello Pradeep, welcome to Shell Scripting training!

🧊 Variables That Cannot Change (readonly)

readonly company="MiddlewareBox"

If you try to change it again, the script will not allow it.


🌍 Global and Local Variables

name="Pradeep"   # Global variable

greet() {
  local name="DevOps Engineer"
  echo "Inside function: $name"
}

greet
echo "Outside function: $name"
Inside function: DevOps Engineer
Outside function: Pradeep

🧠 Store Command Output in a Variable

today=$(date)
echo "Today is: $today"

💻 Simple Practical Example: System Info

host=$(hostname)
os=$(uname -s)
kernel=$(uname -r)

echo "Hostname: $host"
echo "OS: $os"
echo "Kernel: $kernel"

🚀 Simple Use Case: App Deployment Path

app="orderservice"
version="v2.3"
path="/opt/apps/$app"

echo "Deploying $app version $version"
mkdir -p $path
cp $app-$version.jar $path/


🧵 Multi-Word Values (Use Quotes)

msg="Hello Shell Scripting Learners"
echo "$msg"

🔁 Use Variables as Counters

count=1

while [ $count -le 3 ]; do
  echo "Count: $count"
  count=$((count+1))
done

❓ Check if a Variable is Empty

if [ -z "$name" ]; then
  echo "Name is empty"
else
  echo "Name is $name"
fi

⏱️ Use Time in Variable (Backup File Name)

time=$(date +%Y-%m-%d_%H-%M-%S)
file="backup_$time.tar.gz"
echo "Backup file name: $file"

🧰 Small Real-Life Script: Create User and Log

username="john"
log="/var/log/user_creation.log"

echo "Creating user: $username" | tee -a $log
useradd $username
echo "User $username created at $(date)" | tee -a $log

👍 Quick Summary

TopicMeaning
VariableA box to store data
$variableUse the value
readonlyLock the value
localUse only inside a function
$(command)Save output of a command