Menu

Showing posts with label DevOps. Show all posts
Showing posts with label DevOps. Show all posts

3 Feb 2026

🌩️ Azure Session 1 – Cloud Computing Basics

Cloud Computing Basics

Cloud Computing Basics

Target Audience: Beginners, DevOps Engineers, Middleware Administrators, AZ-900 & AZ-104 aspirants.


☁️ What is Cloud Computing?

Cloud Computing is the delivery of computing services such as servers, storage, databases, networking and software over the internet instead of using physical hardware in your own data center.

  • No need to buy physical servers
  • Resources can be created in minutes
  • Scalable and flexible
  • Pay only for what you use

☁️ Cloud Service Models

  • IaaS (Infrastructure as a Service): Virtual servers, storage and networks.
    Use: Full control over OS and applications (Azure Virtual Machines).
  • PaaS (Platform as a Service): Platform and runtime without managing servers.
    Use: Fast application deployment (Azure App Service, Azure SQL).
  • SaaS (Software as a Service): Fully managed software.
    Use: Ready-to-use applications (Microsoft 365).

☁️ Cloud Deployment Models

  • Public Cloud: Shared cloud infrastructure.
    Use: Cost-effective and scalable. like AWS, Azure, GCP [Pay As you Go]
  • Private Cloud: Dedicated infrastructure.
    Use: High security and compliance. like Azure Stack and AWS Outposts
  • Hybrid Cloud: On-premise + cloud.
    Use: Enterprise flexibility.
  • Multi-Cloud: Multiple cloud providers.
    Use: Avoid vendor lock-in. if one providers gets stuck others can be present

⚡ Cloud Characteristics (NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY- NIST)

  • On-Demand Self Service
  • Broad Network Access
  • Resource Pooling
  • Rapid Elasticity
  • Measured Service (Pay-as-you-go)

🌍 User Location & Latency

Latency: Delay between request and response.
Use: Deploy applications near users for better performance.


πŸ“Š Data Types in Cloud

  • Structured Data: SQL databases
  • Unstructured Data: Images, videos, logs
  • Semi-Structured Data: JSON, XML

πŸ›️ Compliance & Data Residency

Some industries require data to stay in specific countries due to legal rules.
Use: Banking, Finance, Healthcare, Government.


πŸ” Conditional Access

Controls access based on user identity, device and location.
Use: Block access from unknown or insecure devices.


πŸ—️ Azure Resource Hierarchy

  • Management Groups
  • Subscriptions
  • Resource Groups
  • Resources

🌐 Hub-and-Spoke Network Model

Central hub contains firewall and security services, while spokes host applications.
Use: Cost saving and centralized security.


πŸ“Š Logs vs Metrics vs Alerts

  • Logs: Event records
  • Metrics: Performance values (CPU, memory)
  • Alerts: Notifications when limits crossed

☁️ Cloud Bursting

Extra workload moves to cloud when on-premise capacity is full.
Use: Payroll processing, exam systems.


🏒 Business Continuity vs Disaster Recovery

  • Business Continuity (BCP): Keep business running
  • Disaster Recovery (DR): Recover systems after failure

πŸ§‘‍πŸ’Ό Identity & Access Management (IAM)

Manages users and their access to resources.
Use: Azure Active Directory, RBAC, MFA.


πŸ“Š Subscription Limits & Quotas

Limits resources to prevent misuse and control cost.


πŸ’° Cloud Pricing Models

  • Pay-as-you-go: Pay only for usage
  • Reserved Instances: Discount for long-term usage
  • Spot Instances: Low-cost unused capacity

🌐 Cloud Networking

  • VNet (Virtual Network): Private network for cloud resources.
    Use: Secure communication.
  • Subnet: Smaller network inside VNet.
    Use: Separate web, app and database layers.
  • NSG (Network Security Group): Firewall rules.
    Use: Allow or block traffic.
  • Load Balancer: Distributes traffic.
    Use: High availability.
  • VPN (Virtual Private Network): Secure tunnel to cloud.
    Use: Connect office to cloud.
  • ExpressRoute: Private dedicated connection.
    Use: Faster enterprise connectivity.

🚚 Cloud Migration – 6R Strategy

  • Rehost (Lift and Shift):
    Move applications from on-premise to cloud without changing their design.
    Use: Fast migration when you want minimal changes and quick cloud adoption.

  • Refactor:
    Modify application architecture to use cloud-native features like auto-scaling and managed databases.
    Use: Improve performance, scalability, and long-term cost efficiency.

  • Replatform:
    Move applications to cloud with small optimizations but without full redesign.
    Use: Gain some cloud benefits while keeping application structure mostly same.

  • Retire:
    Remove applications that are no longer needed or used.
    Use: Reduce cost and simplify IT environment by eliminating unused systems.

  • Retain:
    Keep certain applications on-premise due to security, compliance, or technical reasons.
    Use: When apps cannot be moved to cloud immediately.

  • Replace:
    Replace existing applications with cloud-based SaaS solutions.
    Use: Modernize systems using ready-made cloud software (example: replace on-prem email with Microsoft 365).

πŸ“ˆ Scaling in Cloud

  • Vertical Scaling: Increase CPU or RAM
  • Horizontal Scaling: Add more servers
  • Auto Scaling: Automatic scaling based on load

πŸ” Cloud Security Concepts

  • HTTPS (HyperText Transfer Protocol Secure)
  • RBAC (Role-Based Access Control)
  • NSG (Network Security Group)
  • JIT (Just-In-Time)
  • MFA (Multi-Factor Authentication)

🧠 Cloud Computing Interview Questions

What is Cloud Computing?

Delivering IT services over the internet using pay-as-you-go model.

Difference between Backup and DR?

Backup stores data; DR restores systems.

What is Hybrid Cloud?

Combination of on-premise and cloud.

What is RPO and RTO?

RPO - Recovery Point Objective = How much data loss (time wise) is acceptable if a disaster occurs

RTO -Recovery Time Objective = How quickly the system must be back online after a failure.


πŸ’Ό Enterprise Real-World Scenarios

  • Banking app deployed in multi-region
  • Jenkins CI/CD hosted on Azure VM
  • Healthcare data stored in India region
  • E-learning platform auto scales
  • VPN connection from office to cloud

✅ Key Takeaways

  • Cloud computing is scalable and cost-effective
  • Security and compliance are critical
  • Understanding fundamentals helps interviews
  • Azure supports enterprise workloads

6 Nov 2025

πŸ“ŒWebSphere Outbound SSL & SNI – Troubleshooting Guide

  • ERROR : javax.net.ssl.SSLHandshakeException: No name matching found.
  • This guide helps you diagnose and fix WebSphere outbound HTTPS failures when the target requires SNI.
  • You'll get exact OpenSSL checks, JVM flags, and a support matrix.

WebSphere Outbound SSL & SNI – Troubleshooting Guide

πŸ“‘ Table of Contents

πŸ”Ž What is SNI?

  • Server Name Indication (SNI) is a TLS extension
  • Client includes the target hostname in the ClientHello message
  • Allows servers to present the correct certificate when multiple virtual hosts share the same IP address
Without SNI you often receive a default certificate → CN/SAN mismatch → hostname validation fails even if the trust chain is fine.

🎯 Why SNI matters in WebSphere

  • Outbound calls from WebSphere to cloud APIs, SaaS, and WAF/CDN fronted apps often terminate on shared VIPs
  • Older or non-default Java settings may not send SNI (Server Name Indication).
  • Without SNI, the remote server sends a default certificate.
  • Your client fails with hostname mismatch error.

🚨 Common error patterns

javax.net.ssl.SSLHandshakeException: No name matching <api.company.com> found
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
PKIX path building failed: unable to find valid certification path to requested target
CWPKI0022E: SSL HANDSHAKE FAILURE
GSK_ERROR_BAD_CERT

πŸ§ͺ Confirming an SNI issue (OpenSSL)

Compare results with and without SNI from the WebSphere host:

# Baseline ( May still send SNI implicitly)
openssl s_client -connect api.company.com:443 -showcerts

# Force SNI (ClientHello includes hostname) openssl s_client -connect api.company.com:443 -servername api.company.com -showcerts
πŸ’¬ Interpretation
If CN/SAN only matches when -servername is used, the endpoint requires SNI and your client must send it.

⚙️ How to enable SNI in WebSphere

Add this JVM system property to the server's Generic JVM Arguments:

-Djsse.enableSNIExtension=true

Console path: Servers → Server Types → WebSphere application servers → <server> → Java and Process Management → Process Definition → Java Virtual Machine → Generic JVM Arguments

Save, synchronize nodes, restart the JVM.


Optional: align TLS baseline

-Dhttps.protocols=TLSv1.2

🧭 Best practices checklist

  • Verify CN/SAN with openssl s_client -servername before go-live or on UAT
  • Standardize JVM args across environments; document SNI-dependent endpoints.
  • Keep Java 8+; enforce TLS1.2+ to match common provider baselines.

❓ FAQ

Question Answer
What is SNI and why does it matter? SNI makes the server present the right certificate on shared IPs. Without it, you'll likely hit hostname mismatch in WebSphere even if trust is correct.
How do I enable SNI? Add -Djsse.enableSNIExtension=true to Generic JVM Arguments, then save, sync, and restart.
Which versions support SNI? WAS 8.5.5.x (Java 8) and WAS 9.x (Java 8/11) support SNI; WAS 7.x doesn't; WAS 8.0.x is partial/inconsistent.
How do I confirm an SNI issue? Compare openssl s_client with/without -servername. If only the SNI run shows the correct CN/SAN, you need SNI.

4 Nov 2025

✨ Worked on VMC Today – A Simple Explanation for Everyone

How Brands Show Logo in Gmail Using VMC & BIMI (SPF, DKIM, DMARC, SVG Tiny PS) | MiddlewareBox

Today I discovered something intresting that we encounter daily but rarely understand – how company logos appear in your Mails or Gmail inbox. 🀯

Have you noticed when you receive emails from major brands like Google, Amazon, or your bank, their official logo displays right next to the sender's name? Ever wondered how that works?

I uncovered the answer today, and it's called VMC – Verified Mark Certificate. It's actually much simpler than it sounds! I'm sharing everything I learned here in easy-to-understand language so you can implement this too. 😊


πŸ” What You Need Before Getting VMC – Verified Mark Certificate
  • Your company logo must be trademarked.
    • Official government trademark registration is required.
  • Logo must be in .SVG Tiny PS format.
    • SVG: Scalable Vector Graphics
    • Tiny PS: Tiny Portable/Secure profile (lightweight & email-safe)
  • Email authentication must be properly configured:
    • SPF: Sender Policy Framework
    • DKIM: DomainKeys Identified Mail
    • DMARC: Domain-based Message Authentication, Reporting & Conformance
    • DMARC policy must be: p=quarantine or p=reject with pct=100
  • You will receive a .pem VMC certificate from the Certificate Authority.
    • PEM: Privacy Enhanced Mail format



Use this Link :
  • bimigroup.org/bimi-generator/

  • πŸ”Ž What is VMC – Verified Mark Certificate?

    • Some emails display the company's logo next to the sender's name — that’s VMC in action.
    • VMC (Verified Mark Certificate) allows your official company logo to appear in Gmail, Yahoo, Apple Mail, etc.
    • It verifies that your email is genuinely sent by your organization and not a phishing attempt.
    • Think of it as a ✔️ verification badge for your business emails.

    🎯 Why Organizations Implement VMC

    BenefitExplanation
    Display Logo in InboxYour brand appears professional and trustworthy in every email.
    Build Customer TrustRecipients can instantly identify legitimate emails from your organization.
    Prevent Email SpoofingProtects customers from scammers impersonating your brand.
    Increase Email EngagementVerified logos lead to higher open rates and reduced spam filtering.

    πŸ” Getting a VMC Certificate

    • Choose a Certificate Authority: DigiCert, Entrust, Sectigo.
    • Provide trademark proof — your logo must be officially registered. - Organisation Verification.
    • Prepare your logo in SVG Tiny PS/S format.
    • Complete CA organization & trademark validation.
    • Upon approval, you’ll receive the vmc.pem certificate file.

    πŸ“¦ Understanding the .PEM Certificate & Hosting Options

    • After approval, you get a file: vmc.pem.
    • Both logo and pem must be publicly accessible:
    • logo.svg
      vmc.pem

    Option 1: Hosted by Certificate Authority by Digicert or Entrust (Easy)

    https://vmc.digicert.com/<your-id>.svg
    https://vmc.digicert.com/<your-id>.pem
    

    Option 2: Self-Host on Your Domain (Recommended)

    https://brand.yourdomain.com/bimi/logo.svg
    https://brand.yourdomain.com/bimi/vmc.pem
    

    Directory structure:

    /bimi
    ├── logo.svg
    └── vmc.pem
    

    🧩 Creating Your BIMI DNS Record

    • Once files are hosted, create a BIMI TXT record.
    • Host: default._bimi
    • Type: TXT

    BIMI TXT Value:

    v=BIMI1; l=https://brand.yourdomain.com/bimi/logo.svg; a=https://brand.yourdomain.com/bimi/vmc.pem; avp=digicert
    

    Note: avp is optional. It indicates the certificate authority.


    πŸ§ͺ Testing Your BIMI & VMC Implementation


    πŸ’Ž Pro Tip: Use crt.sh for Certificate Search

    • crt.sh helps you search SSL/TLS & VMC certificates.
    • Search by domain, CA, or organization.
    • Link: crt.sh

    ❓ Frequently Asked Questions

    • Do I configure VMC in DigiCert?
      πŸ”₯ NO – You only obtain the certificate from DigiCert. DNS configuration is YOUR responsibility.
    • Why does my URL show vmc.digicert.com or bimi.entrust.com?
      πŸ”₯ NORMAL – DigiCert/Entrust hosts your BIMI files. This is STANDARD OPERATION and completely acceptable.
    • Logo not showing?
      πŸ”₯ TROUBLESHOOT – Verify:
      • DMARC policy (p=quarantine or p=reject)
      • BIMI record syntax
      • SVG format compliance
      • DKIM authentication
    • Is BIMI supported everywhere?
      πŸ”₯ MAJOR PROVIDERS – Gmail, Yahoo, and others support it. Coverage is EXPANDING but not universal.

    ✨ Final Tip: Add BIMI TXT → Validate using tools → Send a test email to Gmail/Yahoo to see your logo in inbox!


    2 Nov 2025

    🐧 Linux Shell Scripting for Beginners – Complete Tutorial Series.

    • A 13-part practical shell scripting course for DevOps and Middleware Engineers.

    πŸ’‘ How This Tutorial Series Will Help You

    • ✔ Build a strong foundation in Linux Shell scripting — from basics to automation.
    • ✔ Automate daily Middleware and DevOps tasks: monitoring, backups, and log management.
    • ✔ Write scripts for Middleware platforms like Tomcat, Jenkins, and WebSphere.
    • ✔ Debug, schedule, and manage production-ready shell scripts confidently.
    • ✔ Perfect for Sysadmins, Middleware, and DevOps Engineers upgrading to Cloud roles.

    πŸ“š Complete Course Index

    🐚 Part 1: What is a Variable?
    Learn how variables store data and simplify automation scripts. Foundation of all shell logic.
    πŸ“₯ Part 2: Reading User Input in Shell
    Make interactive scripts using read command with Jenkins and Tomcat examples.
    Use decision-making in your scripts with examples checking files, services, and network health.
    Automate repetitive DevOps tasks like log cleanup, backups, and status monitoring.
    Write modular, reusable functions — restart Tomcat or back up Jenkins in one click.
    πŸ“¦ Part 6: Arrays & Arguments
    Store multiple values, manage user inputs, and process server lists in scripts.
    πŸ“ Part 7: File Handling in Shell
    Learn how to read, write, append, and handle configuration or log files safely.
    πŸ’‘ Part 8: Debugging & Logging
    Find and fix script errors, use logs, and enable debug mode for safe execution.
    πŸ•’ Part 9: Scheduling & Automation
    Use cron and @reboot jobs to run scripts automatically for health checks or backups.
    Create a production-ready monitoring script for Tomcat, Jenkins, and NGINX with auto-restart.
    🧾 Part 11: Log Management & Rotation
    Automate log rotation with logrotate and custom retention scripts for Middleware systems.
    Perform automated backup and restore for Jenkins, Tomcat, and WebSphere with rollback support.
    Run multiple jobs in parallel, monitor background processes, and speed up automation workflows.

    πŸš€ Why MiddlewareBox Shell Scripting Series?

    • ✅ Step-by-step, beginner-friendly explanations.
    • ✅ Each topic includes real DevOps & Middleware context.
    • ✅ Examples with Output and Error handling.
    • ✅ Simple language for better understanding.

    ✨ Start Learning Now → Part 1: What is a Variable?
    & build your own scripts from scratch! πŸ’ͺ

    ⚙️ Shell Scripting for Beginners – Part 13: Subshells, Process Control & Parallel Execution.

    • Now let's level up and make your shell scripts multitask like a Middleware and DevOps pro.
    • In this part, you'll learn how to run processes in the background, manage subshells, and execute multiple tasks in parallel for faster automation.

    πŸ“‘ Table of Contents


    1️⃣ What is a Subshell?

    A subshell is a new child shell launched from your current shell. It allows running commands in isolation — any variable or environment change inside doesn’t affect the parent shell.

    #!/bin/bash
    VAR="ParentShell"
    ( VAR="SubShell"; echo "Inside subshell: $VAR" )
    echo "Outside subshell: $VAR"
    
    πŸ’¬ Output:
    Inside subshell: SubShell
    Outside subshell: ParentShell

    2️⃣ Running Commands in the Background

    Add & at the end of a command to run it in the background. This allows your script to keep running while other tasks continue behind the scenes.

    #!/bin/bash
    echo "Starting Jenkins backup..."
    tar -czf /opt/backups/jenkins_backup.tar.gz /var/lib/jenkins & 
    echo "Backup started in background (PID: $!)"
    
    πŸ’¬ Output:
    Starting Jenkins backup...
    Backup started in background (PID: 2356)

    3️⃣ Managing Jobs

    When you run background processes, you can manage them using built-in commands:

    • jobs → List current background jobs
    • fg %1 → Bring job 1 to foreground
    • bg %1 → Resume job 1 in background
    • kill %1 → Kill job 1
    #!/bin/bash
    long_task() {
      sleep 30 &
    }
    long_task
    jobs
    
    πŸ’¬ Output:
    [1]+ Running sleep 30 &

    4️⃣ Parallel Execution in DevOps

    Let’s say you want to restart multiple Tomcat servers or back up several apps simultaneously. We can use background jobs with a simple for loop.

    #!/bin/bash
    servers=("devbox" "testbox" "prodbox")
    for srv in "${servers[@]}"; do
      echo "Restarting Tomcat on $srv..."
      ssh $srv "systemctl restart tomcat" &
    done
    
    wait  # Wait for all background jobs to finish
    echo "✅ All Tomcat servers restarted in parallel!"
    
    πŸ’¬ Output:
    Restarting Tomcat on devbox...
    Restarting Tomcat on testbox...
    Restarting Tomcat on prodbox...
    ✅ All Tomcat servers restarted in parallel!

    5️⃣ nohup & disown

    Sometimes, you need your scripts to survive terminal disconnections — for instance, a long-running Jenkins restore or DB migration.

    #!/bin/bash
    nohup ./db_migration.sh > /var/log/db_migration.log 2>&1 &
    disown
    echo "🧠 DB migration running safely in background, even if you close terminal."
    
    πŸ’¬ Output:
    nohup: ignoring input and appending output to 'nohup.out'
    🧠 DB migration running safely in background, even if you close terminal.

    6️⃣ Real-World DevOps Examples

    🧩 Example 1: Parallel Log Collection

    #!/bin/bash
    servers=("dev" "test" "prod")
    for s in "${servers[@]}"; do
      scp $s:/var/log/tomcat/catalina.out /opt/logs/$s.log &
    done
    wait
    echo "✅ Logs collected from all servers."
    

    🧰 Example 2: Monitor Jenkins & Docker in Background

    #!/bin/bash
    ( while true; do pgrep -f jenkins >/dev/null || systemctl restart jenkins; sleep 10; done ) &
    ( while true; do docker ps | grep -q "webapp" || docker start webapp; sleep 15; done ) &
    echo "🩺 Jenkins and Docker monitored in background."
    

    πŸ“¦ Example 3: Run Multiple CI Jobs Simultaneously

    #!/bin/bash
    for job in build-test build-stage build-prod; do
      curl -s -X POST "http://jenkins.local/job/$job/build" &
    done
    wait
    echo "✅ All Jenkins build jobs triggered in parallel!"
    

    🧠 Pro Tips & Troubleshooting

    🧩 Process Control Best Practices

    • Use wait to synchronize parallel tasks before continuing.
    • Log PIDs ($!) to monitor background tasks easily.
    • Redirect background job output to separate log files using > file 2>&1 &.
    • For persistent background services, prefer systemd over manual nohup scripts.

    ⚙️ Debugging Issues

    • Run ps -ef | grep scriptname to confirm background processes.
    • Use set -x or bash -x script.sh for detailed command tracing.
    • Use trap to clean up child processes on script exit.
    • If parallel SSH commands hang, add -o ConnectTimeout=5 in SSH for safety.

    🏁 Summary

    • ✅ Learned to create and manage subshells.
    • ⚙️ Mastered process control using &, jobs, fg, bg, and kill.
    • πŸš€ Ran real-world DevOps tasks in parallel — Jenkins builds, Tomcat restarts, Docker checks.
    • πŸ’‘ Used nohup and disown for persistent automation tasks.

    πŸ’Ύ Shell Scripting for Beginners – Part 12: Backup & Restore Automation Project (DevOps & Middleware Edition)

    • Here we'll automate backup and restore tasks for real middleware & DevOps systems like Jenkins, Tomcat, WebSphere, MySQL, Docker, and Apache.
    • Each example includes a working restore script and troubleshooting tips. ⚙️

    πŸ“‘ Table of Contents


    1️⃣ Jenkins Backup & Restore

    #!/bin/bash
    JENKINS_HOME="/var/lib/jenkins"
    BACKUP_DIR="/opt/backups/jenkins"
    mkdir -p $BACKUP_DIR
    tar -czf $BACKUP_DIR/jenkins_$(date +%F_%H-%M).tar.gz -C $JENKINS_HOME .
    echo "✅ Jenkins backup created in $BACKUP_DIR"
    

    ♻️ Restore Jenkins

    #!/bin/bash
    BACKUP_FILE="/opt/backups/jenkins/jenkins_2025-11-02_02-10.tar.gz"
    service jenkins stop
    tar -xzf "$BACKUP_FILE" -C /var/lib/jenkins
    chown -R jenkins:jenkins /var/lib/jenkins
    service jenkins start
    echo "♻️ Jenkins restored successfully."
    

    2️⃣ Tomcat Backup & Restore

    #!/bin/bash
    TOMCAT_HOME="/opt/tomcat"
    BACKUP_DIR="/opt/backups/tomcat"
    mkdir -p $BACKUP_DIR
    tar -czf $BACKUP_DIR/tomcat_$(date +%F).tar.gz $TOMCAT_HOME/webapps $TOMCAT_HOME/conf
    echo "πŸ“¦ Tomcat backup completed."
    

    ♻️ Restore Tomcat

    #!/bin/bash
    BACKUP_FILE="/opt/backups/tomcat/tomcat_2025-11-02.tar.gz"
    service tomcat stop
    tar -xzf "$BACKUP_FILE" -C /
    service tomcat start
    echo "♻️ Tomcat restored and restarted."
    

    3️⃣ WebSphere Backup & Restore

    #!/bin/bash
    WAS_PROFILE="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"
    BACKUP_DIR="/opt/backups/websphere"
    mkdir -p $BACKUP_DIR
    tar -czf $BACKUP_DIR/was_$(date +%F_%H-%M).tar.gz -C $WAS_PROFILE .
    echo "πŸ“ WebSphere configuration backup done."
    

    ♻️ Restore WebSphere

    #!/bin/bash
    BACKUP_FILE="/opt/backups/websphere/was_2025-11-02_03-00.tar.gz"
    service was stop
    tar -xzf "$BACKUP_FILE" -C /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
    service was start
    echo "♻️ WebSphere restored successfully."
    

    4️⃣ MySQL Database Backup & Restore

    #!/bin/bash
    DB="middlewaredb"
    USER="root"
    PASS="Secret123"
    BACKUP_DIR="/opt/backups/mysql"
    mkdir -p $BACKUP_DIR
    mysqldump -u $USER -p$PASS $DB > $BACKUP_DIR/${DB}_$(date +%F).sql
    gzip $BACKUP_DIR/${DB}_$(date +%F).sql
    echo "πŸ—„️ MySQL backup completed."
    

    ♻️ Restore MySQL

    #!/bin/bash
    BACKUP_FILE="/opt/backups/mysql/middlewaredb_2025-11-02.sql.gz"
    gunzip "$BACKUP_FILE"
    mysql -u root -pSecret123 middlewaredb < /opt/backups/mysql/middlewaredb_2025-11-02.sql
    echo "✅ Database restored successfully."
    

    5️⃣ Docker Containers Backup & Restore

    #!/bin/bash
    BACKUP_DIR="/opt/backups/docker"
    mkdir -p $BACKUP_DIR
    for cid in $(docker ps -q); do
      cname=$(docker inspect --format='{{.Name}}' $cid | cut -d'/' -f2)
      docker export $cid > $BACKUP_DIR/${cname}_$(date +%F).tar
    done
    echo "🐳 Docker containers exported successfully."
    

    ♻️ Restore Docker Container

    #!/bin/bash
    BACKUP_FILE="/opt/backups/docker/myapp_2025-11-02.tar"
    docker import "$BACKUP_FILE" myapp_restored:latest
    docker run -d --name myapp_restored myapp_restored:latest
    echo "♻️ Docker container restored and running."
    

    6️⃣ Apache Webserver Backup & Restore

    #!/bin/bash
    BACKUP_DIR="/opt/backups/apache"
    mkdir -p $BACKUP_DIR
    tar -czf $BACKUP_DIR/apache_$(date +%F).tar.gz /etc/apache2 /var/www/html /etc/ssl
    echo "🌐 Apache configuration, website, and SSL backed up."
    

    ♻️ Restore Apache

    #!/bin/bash
    BACKUP_FILE="/opt/backups/apache/apache_2025-11-02.tar.gz"
    tar -xzf "$BACKUP_FILE" -C /
    service apache2 restart
    echo "✅ Apache restored and running."
    

    7️⃣ Automating Backups

    Schedule automatic backups using cron so your systems stay protected 24×7.

    # Run daily at 2 AM
    0 2 * * * /opt/scripts/backup_all.sh >> /var/log/backup_all.log 2>&1
    
    # Run on every reboot
    @reboot /opt/scripts/backup_all.sh >> /var/log/backup_boot.log 2>&1
    
    πŸ’¬ Output:
    [CRON] Scheduled backup started…
    ✅ Jenkins, Tomcat, WebSphere, MySQL, Docker, Apache backed up successfully.

    🧠 Pro Tips & Troubleshooting

    ✅ General Best Practices

    • Use a dedicated /opt/backups partition or NFS mount to avoid filling system drives.
    • Compress large backups with gzip or zstd [Developed by Meta] for better space efficiency.
    • Store daily, weekly, and monthly copies separately (retention policy).
    • Automate restores in a test environment weekly to ensure recovery works.

    ⚙️ Troubleshooting Common Issues

    • Permission Denied: Run backups as sudo or fix ownership using chown.
    • πŸ“¦ Disk Full: Use df -h to check space before backup. Add cleanup logic for old files.
    • 🧾 Log Rotation: Add logrotate entry to keep backup logs small and readable.

    🏁 Summary

    • ✅ Full backup & restore automation for all major middleware and DevOps tools.
    • 🧩 Scripts are modular, reusable, and easy to integrate in CI/CD pipelines.
    • ⚙️ Cron ensures zero manual intervention for daily protection.

    1 Nov 2025

    🧾 Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

    • Logs are the heartbeat of your servers πŸ’“ — but if left unmanaged, they'll crash your systems.
    • In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

    πŸ“‘ Table of Contents


    1️⃣ Log Cleanup Scripts

    #!/bin/bash
    # Delete logs older than 7 days for Tomcat & Jenkins
    find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
    find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
    echo "🧹 Tomcat & Jenkins logs older than 7 days removed!"
    
    πŸ’¬ Output:
    🧹 Tomcat & Jenkins logs older than 7 days removed!

    2️⃣ Apache & Middleware Examples

    #!/bin/bash
    ARCHIVE="/opt/log_archive/apache"
    mkdir -p $ARCHIVE
    find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
    mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
    echo "πŸ“¦ Apache logs compressed and archived in $ARCHIVE"
    
    πŸ’¬ Output:
    πŸ“¦ Apache logs compressed and archived in /opt/log_archive/apache

    3️⃣ Docker & Kubernetes Log Cleanup

    #!/bin/bash
    echo "🐳 Cleaning Docker & K8s logs..."
    docker system prune -af
    find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
    echo "✅ Docker and Kubernetes logs cleaned!"
    
    πŸ’¬ Output:
    ✅ Docker and Kubernetes logs cleaned!

    4️⃣ logrotate Configuration

    # /etc/logrotate.d/devops-services
    /opt/tomcat/logs/*.log
    /var/log/apache2/*.log
    /var/lib/jenkins/logs/*.log
    /var/log/docker/*.log {
        weekly
        rotate 6
        compress
        missingok
        notifempty
        sharedscripts
        postrotate
            systemctl reload apache2 2>/dev/null || true
            systemctl restart tomcat 2>/dev/null || true
        endscript
    }
    
    πŸ’¬ Result:
    πŸ” Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

    5️⃣ How to Run & Test logrotate

    Once your config is ready, you can run logrotate manually or let cron handle it.

    🧠 Step 1 – Check Syntax of Your Config

    sudo logrotate -d /etc/logrotate.d/devops-services
    πŸ’¬ Output:
    Reading config file /etc/logrotate.d/devops-services
    Handling /opt/tomcat/logs/*.log
    Handling /var/log/apache2/*.log
    **dry-run mode (no rotation performed)**

    ⚙️ Step 2 – Run logrotate Manually

    sudo logrotate -f /etc/logrotate.d/devops-services
    πŸ’¬ Output:
    Rotating logs for Tomcat, Apache, Jenkins...
    Compression complete.
    Rotation successful ✅

    ⏰ Step 3 – Schedule via Cron (Daily or Weekly)

    # /etc/cron.daily/logrotate
    /usr/sbin/logrotate /etc/logrotate.conf

    πŸ’‘ By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.


    🏁 Summary

    • ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
    • ✅ Created unified logrotate configuration.
    • ✅ Learned to manually test (-d) and force-run (-f) logrotate.
    • ✅ Automated rotation through cron for 24×7 log hygiene.

    🎯Shell Scripting for Beginners – Part 10: Middleware Health Monitoring Project (Mixed Example).

    • This is our final, beginner-friendly project 🎯 — where we'll combine everything you've learned: variables, if-else, loops, functions, logging, and cron automation.
    • Let's create a simple Middleware Health Monitoring script that checks Tomcat, Jenkins, Docker, MySQL, and SSL expiry.

    πŸ“‘ Table of Contents


    1️⃣ Overview

    This script uses:

    • Variables – store names, logs, and domains.
    • Functions – reusable checks for each service.
    • If-Else – to decide if a service needs restart.
    • For loop – to check multiple Docker containers.
    • While loop – to retry connection.
    • Logs – write output to a file.

    2️⃣ Full Middleware Health Monitoring Script

    #!/bin/bash
    # MiddlewareBox - Simple Health Monitor
    
    LOG="/var/log/middleware_health.log"
    domain="middlewarebox.com"
    echo "===== Middleware Health Check =====" >> $LOG
    date >> $LOG
    
    # --- Function to check Tomcat ---
    check_tomcat() {
      echo "Checking Tomcat..."
      if pgrep -f "org.apache.catalina.startup.Bootstrap" > /dev/null; then
         echo "✅ Tomcat is running" | tee -a $LOG
      else
         echo "🚨 Tomcat is down, restarting..." | tee -a $LOG
         /opt/tomcat/bin/startup.sh
         echo "✅ Tomcat restarted" | tee -a $LOG
      fi
    }
    
    # --- Function to check Jenkins ---
    check_jenkins() {
      echo "Checking Jenkins..."
      if systemctl is-active --quiet jenkins; then
         echo "✅ Jenkins is active" | tee -a $LOG
      else
         echo "🚨 Jenkins not running, restarting..." | tee -a $LOG
         systemctl restart jenkins
         echo "✅ Jenkins restarted" | tee -a $LOG
      fi
    }
    
    # --- Function to check MySQL ---
    check_db() {
      echo "Checking MySQL..."
      if mysql -u root -pSecret -e "show databases;" > /dev/null 2>&1; then
         echo "✅ MySQL connection successful" | tee -a $LOG
      else
         echo "🚨 MySQL connection failed" | tee -a $LOG
         echo "Retrying in 5 seconds..."
         sleep 5
         mysql -u root -pSecret -e "show databases;" && echo "✅ MySQL reconnected" | tee -a $LOG
      fi
    }
    
    # --- Check Docker Containers using FOR Loop ---
    check_docker() {
      echo "Checking Docker containers..."
      for c in nginx webapp db
      do
        status=$(docker inspect -f '{{.State.Status}}' $c 2>/dev/null)
        if [ "$status" != "running" ]; then
          echo "🚨 Container $c is $status, restarting..." | tee -a $LOG
          docker start $c
        else
          echo "✅ Container $c is running" | tee -a $LOG
        fi
      done
    }
    
    # --- Check SSL certificate expiry ---
    check_ssl() {
      echo "Checking SSL expiry..."
      expiry=$(echo | openssl s_client -servername $domain -connect $domain:443 2>/dev/null | openssl x509 -noout -dates | grep notAfter | cut -d= -f2)
      days_left=$(( ($(date -d "$expiry" +%s) - $(date +%s)) / 86400 ))
      if [ $days_left -le 15 ]; then
        echo "⚠️ SSL expires in $days_left days!" | tee -a $LOG
      else
        echo "✅ SSL valid for $days_left days" | tee -a $LOG
      fi
    }
    
    # --- MAIN EXECUTION FLOW ---
    check_tomcat
    check_jenkins
    check_db
    check_docker
    check_ssl
    
    echo "✅ Health check completed successfully!" | tee -a $LOG
    echo "========================================" >> $LOG
    
    πŸ’¬ Example Output:
    ✅ Tomcat is running
    ✅ Jenkins is active
    ✅ MySQL connection successful
    ✅ Container nginx is running
    ✅ SSL valid for 70 days
    ✅ Health check completed successfully!

    3️⃣ Automate the Script

    # Run every 30 minutes
    */30 * * * * /opt/scripts/middleware_health.sh >> /var/log/health_cron.log 2>&1
    
    # Run automatically on system reboot
    @reboot /opt/scripts/middleware_health.sh >> /var/log/boot_health.log 2>&1
    
    πŸ’¬ Output (Cron log):
    [BOOT] Health check executed
    ✅ All services healthy

    🏁 Summary

    • ✅ Combined all major shell scripting concepts.
    • ✅ Checks Tomcat, Jenkins, Docker, MySQL, and SSL in one file.
    • ✅ Uses if, for, functions, and basic while retry logic.
    • ✅ Can run manually or via cron / reboot automation.

    πŸ•’ Shell Scripting for Beginners – Part 9: Scheduling & Automation

    • In DevOps and Middleware environments, automation isn't complete until it's scheduled.
    • Here, we master cron, at, @reboot, and Jenkins integration — running scripts automatically for backups, restarts, monitoring, and cleanups.

    πŸ“‘ Table of Contents


    ⏰ 1️⃣ Automating with Cron

    • cron executes scripts periodically.
    • Perfect for recurring jobs like backups, cleanup, and health checks.

    Example 1: Daily Jenkins Backup at 3 AM

    0 3 * * * /opt/scripts/jenkins_backup.sh >> /var/log/jenkins_backup.log 2>&1
    πŸ’¬ Output (log):
    [03:00] Jenkins backup started
    [03:03] ✅ Jenkins backup completed

    Example 2: Weekly Tomcat Log Cleanup

    0 2 * * 0 /opt/scripts/cleanup_tomcat_logs.sh >> /var/log/tomcat_cleanup.log 2>&1
    πŸ’¬ Output:
    🧹 Deleted old logs from /opt/tomcat/logs
    ✅ Weekly cleanup done

    πŸ• 2️⃣ One-Time Tasks with At

    at executes a job once — ideal for maintenance windows, patches, or urgent restarts.

    Example 3: Restart WebSphere at 11:30 PM

    echo "/opt/IBM/WebSphere/AppServer/bin/stopServer.sh server1 && sleep 15 && /opt/IBM/WebSphere/AppServer/bin/startServer.sh server1" | at 23:30
    πŸ’¬ Output:
    job 17 at Sat Nov 1 23:30:00 2025

    Example 4: Run Backup Once in 2 Hours

    echo "/opt/scripts/backup_db.sh" | at now + 2 hours
    πŸ’¬ Output:
    job 18 at Sat Nov 1 05:30:00 2025

    πŸ”„ 3️⃣ Auto-Execute on System Reboot

    The @reboot keyword in crontab ensures your scripts run every time Linux boots. Perfect for restarting services, mounting drives, or initializing environments.


    Example 5: Auto-Start Tomcat After Reboot

    @reboot /opt/tomcat/bin/startup.sh >> /var/log/tomcat_reboot.log 2>&1
    πŸ’¬ Output (after reboot):
    ✅ Tomcat started automatically at boot.

    Example 6: Health Check Script at Boot

    @reboot /opt/scripts/health_check.sh >> /var/log/health_boot.log 2>&1
    πŸ’¬ Output (log):
    [BOOT] Checking WebSphere, Jenkins, NGINX...
    ✅ All services active.
    πŸ’‘ To verify reboot jobs, check grep CRON /var/log/syslog after restart.

    ⚙️ 4️⃣ Jenkins Scheduler Integration

    In CI/CD, Jenkins provides cron-like scheduling for build pipelines.

    Example 7: Daily Jenkins Build at 2 AM

    pipeline {
      triggers { cron('H 2 * * *') }
      stages {
        stage('Nightly Build') {
          steps {
            sh '/opt/scripts/build_app.sh'
          }
        }
      }
    }
    πŸ’¬ Output:
    ✅ Jenkins triggered build_app.sh
    Build completed successfully.

    🧰 5️⃣ Real Middleware & DevOps Automation Examples

    Example 8: NGINX Health Monitor Every 10 Minutes

    */10 * * * * /opt/scripts/nginx_health.sh >> /var/log/nginx_monitor.log 2>&1
    πŸ’¬ Output:
    [02:40] 🌐 NGINX is active
    [02:50] 🚨 NGINX down – restarted automatically.

    Example 9: Database Backup Every 6 Hours

    0 */6 * * * mysqldump -u root -pSecret middlewaredb > /backup/db_$(date +%F_%H).sql
    πŸ’¬ Output:
    ✅ Backup completed: db_2025-11-01_06.sql

    Example 10: Restart Jenkins After Crash Detection

    */15 * * * * pgrep jenkins >/dev/null || systemctl restart jenkins
    πŸ’¬ Output:
    [03:15] 🚨 Jenkins was down, restarted successfully.

    Example 11: Auto-Remount Missing Volume

    @reboot mount | grep /mnt/backup >/dev/null || mount /dev/sdb1 /mnt/backup
    πŸ’¬ Output:
    ✅ /mnt/backup auto-mounted on system startup.

    🧠 6️⃣ Tips & Troubleshooting

    • πŸ“„ Check logs → grep CRON /var/log/syslog
    • πŸ”’ Ensure permissions → chmod +x /opt/scripts/*.sh
    • ⚙️ Test reboot jobs manually → sudo run-parts /etc/cron.d/
    • πŸ“§ Use MAILTO in crontab for notifications
    • 🚨 For Jenkins jobs, check /var/lib/jenkins/jobs/*/builds

    🏁 Summary

    • Cron → Schedules recurring jobs
    • At → Runs one-time tasks
    • @reboot → Executes scripts on startup
    • Jenkins → Manages enterprise-level schedules
    • πŸ’‘ Combine them to create self-healing, auto-starting Middleware environments.

    πŸ’‘Shell Scripting for Beginners – Part 8: Debugging & Logging

    • By now, your scripts deploy apps, manage servers, and clean logs.
    • But what happens when things go wrong?
    • Welcome to Debugging & Logging, where we make your scripts smarter and traceable like a pro DevOps engineer! πŸ•΅️‍♂️

    πŸ“‘ Table of Contents


    🐞 1️⃣ Debugging with set -x

    set -x shows every command the script executes — perfect for troubleshooting silent errors.

    #!/bin/bash
    set -x  # enable debugging
    echo "Starting deployment..."
    mkdir /opt/app/deploy
    cp /tmp/app.war /opt/app/deploy/
    set +x  # disable debugging
    echo "Deployment finished!"
    
    πŸ’¬ Output:
    + echo "Starting deployment..."
    Starting deployment...
    + mkdir /opt/app/deploy
    + cp /tmp/app.war /opt/app/deploy/
    + echo "Deployment finished!"
    Deployment finished!
    πŸ’‘ Use bash -x script.sh to debug a script without editing it.

    πŸͺ€ 2️⃣ Using trap to Handle Failures

    The trap command catches unexpected exits (Ctrl+C, errors) and executes custom recovery code. Perfect for cleanup or service restarts.

    #!/bin/bash
    trap 'echo "⚠️ Script interrupted! Cleaning up..."; rm -f /tmp/tempfile' EXIT
    echo "Creating temporary file..."
    touch /tmp/tempfile
    echo "Simulating work..."
    sleep 3
    exit 1  # simulate failure
    
    πŸ’¬ Output:
    Creating temporary file...
    Simulating work...
    ⚠️ Script interrupted! Cleaning up...
    πŸ’‘ Always trap EXIT or ERR to prevent orphan temp files or hanging processes.

    πŸšͺ 3️⃣ Exit Codes & Error Messages

    Every command returns an exit code — 0 means success, non-zero means failure.

    #!/bin/bash
    cp /tmp/app.war /opt/app/deploy/
    if [ $? -ne 0 ]; then
      echo "❌ Copy failed! Check file path or permissions."
      exit 1
    else
      echo "✅ Copy successful!"
    fi
    
    πŸ’¬ Output:
    ❌ Copy failed! Check file path or permissions.
    or
    ✅ Copy successful!
    Use echo $? after any command to view its exit code.

    🧾 4️⃣ Logging with Syslog & Custom Files

    Example 1: Using logger Command

    #!/bin/bash
    logger "Starting Middleware backup..."
    tar -czf /tmp/backup.tar.gz /opt/middleware/ || logger "Backup failed!"
    logger "Middleware backup completed!"
    
    πŸ’¬ Syslog Output (in /var/log/syslog):
    Nov 1 02:15:00 server logger: Starting Middleware backup...
    Nov 1 02:15:03 server logger: Middleware backup completed!

    Example 2: Redirecting Logs to a File

    #!/bin/bash
    exec >> /opt/logs/deploy.log 2>&1
    echo "[$(date)] Starting deployment..."
    echo "[$(date)] Copying WAR..."
    cp /tmp/app.war /opt/tomcat/webapps/
    echo "[$(date)] Deployment done ✅"
    
    πŸ’¬ deploy.log Content:
    [Sat Nov 1 02:20:00 2025] Starting deployment...
    [Sat Nov 1 02:20:01 2025] Copying WAR...
    [Sat Nov 1 02:20:02 2025] Deployment done ✅

    ⚙️ 5️⃣ Real Middleware & DevOps Examples

    Example 1: Jenkins Job Failure Recovery

    #!/bin/bash
    set -e
    trap 'echo "🚨 Jenkins build failed, restarting service..."; systemctl restart jenkins' ERR
    curl -s -X POST "http://localhost:8080/job/myjob/build" || false
    echo "✅ Jenkins job triggered successfully!"
    
    πŸ’¬ Output:
    🚨 Jenkins build failed, restarting service...
    ✅ Jenkins job triggered successfully!

    Example 2: Tomcat Log Watcher with Debug Info

    #!/bin/bash
    set -x
    tail -Fn0 /opt/tomcat/logs/catalina.out | \
    while read line; do
      echo "$line" | grep "ERROR" &>/dev/null && echo "πŸ”₯ Found error: $line" >> /opt/logs/tomcat_error.log
    done
    
    πŸ’¬ Output:
    [DEBUG] + grep "ERROR" catalina.out
    πŸ”₯ Found error: Database connection timeout

    Example 3: NGINX Health Check with Logging

    #!/bin/bash
    while true; do
      if ! systemctl is-active nginx >/dev/null; then
        echo "$(date): 🚨 NGINX is down, restarting..." >> /var/log/nginx_monitor.log
        systemctl restart nginx
      fi
      sleep 10
    done
    
    πŸ’¬ Log Output:
    Sat Nov 1 03:00:00 2025: 🚨 NGINX is down, restarting...

    🏁 Summary

    • set -x → Debug command execution flow.
    • trap → Catch exits & clean up automatically.
    • exit codes → Detect and handle failures gracefully.
    • logger → Send messages to syslog or custom logs.
    • πŸ’‘ Always include logging in production-ready scripts for audit and rollback.