Menu

Showing posts with label Docker. Show all posts
Showing posts with label Docker. Show all posts

15 Jun 2026

⚖️ JWT vs Session vs Cookies Explained - Part 5

JWT vs Session vs Cookies Explained
  • Welcome to Part 5 of the Authentication & Identity Security series.
  • This article explains the difference between JWT, Sessions and Cookies.
  • Designed for Middleware, DevOps, Cloud, API, and Application Support Engineers.
  • Includes enterprise examples using WebSphere, JBoss, Tomcat, Docker, Azure Entra ID, APIs, and Load Balancers.


Introduction

In previous parts, we learned about sessions, cookies, JSESSIONID, stateful applications, stateless applications, and JWT-based authentication.

Now it is important to clearly understand the difference between Cookies, Sessions and JWT. Many engineers confuse these terms and use them interchangeably, but they are different concepts.

Simple Understanding:
A cookie is stored in the browser. A session is stored on the server. A JWT is a signed token usually used for stateless authentication.

A cookie is a small piece of data stored in the user's browser. The server sends a cookie to the browser, and the browser sends it back with future requests.

Cookies are commonly used to store identifiers, preferences, tracking values, and session IDs.

Example Cookie

Set-Cookie: JSESSIONID=ABC123XYZ789; Path=/app; Secure; HttpOnly

Browser Sends Cookie Back

GET /app/dashboard HTTP/1.1
Host: www.company.com
Cookie: JSESSIONID=ABC123XYZ789
Important:
A cookie is only a storage mechanism in the browser. It is not the same as a session.

What is a Session?

A session is server-side storage used to maintain user information after login.

In Java enterprise applications, the session is commonly identified using JSESSIONID.

Session Data Example

Session ID: ABC123XYZ789

Stored on Server:
- userId = pradeep
- role = admin
- loginTime = 10:30 AM
- applicationAccess = policy-portal

The browser does not usually store this full session data. The browser only stores the JSESSIONID cookie. The actual session data remains inside the application server memory or external session store.

Common Platforms

  • IBM WebSphere Application Server
  • WebSphere Liberty
  • Apache Tomcat
  • JBoss EAP
  • WildFly

What is JWT?

JWT stands for JSON Web Token. It is a signed token that contains claims about the user or application.

JWT is commonly used in APIs, microservices, mobile applications, Docker-based applications, and cloud-native systems.

JWT Example

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
eyJzdWIiOiJwcmFkZWVwIiwicm9sZSI6ImFkbWluIn0.
signature

JWT Payload Example

{
  "sub": "pradeep",
  "role": "admin",
  "iss": "https://login.company.com",
  "aud": "api.company.com",
  "exp": 1789459200
}
Important:
JWT payload is encoded, not encrypted by default. Do not store passwords, secrets, or sensitive data inside JWT.

Cookie, Session and JWT Flow

Session + Cookie Flow

User Login
   │
Application Server Authenticates User
   │
Server Creates Session
   │
Server Generates JSESSIONID
   │
Browser Stores JSESSIONID Cookie
   │
Browser Sends Cookie With Every Request
   │
Server Finds Session Using JSESSIONID

JWT Flow

User Login
   │
Authentication Server Validates User
   │
JWT Generated
   │
Client Stores JWT
   │
Client Sends JWT With API Request
   │
API Validates JWT Signature and Expiry
   │
Access Granted

JWT vs Session vs Cookies Comparison

Feature Cookie Session JWT
Basic Meaning Browser storage mechanism Server-side user state Signed authentication token
Stored In Browser Application Server / External Store Client side
Common Example JSESSIONID cookie User session in JVM heap Bearer token
Stateful / Stateless Storage only Stateful Stateless
Server Memory Required No Yes, unless externalized No session memory required
Scaling Easy Requires sticky session or replication Easier horizontal scaling
Best For Session ID, preferences Traditional web applications APIs, mobile apps, microservices
Security Risk Cookie theft Session hijacking Token theft

Enterprise Middleware Example

Traditional WebSphere / JBoss / Tomcat Application

Browser
   │
JSESSIONID Cookie
   │
IBM HTTP Server / NGINX
   │
WebSphere / JBoss / Tomcat
   │
Session Stored in JVM Memory

This architecture is common in enterprise web applications. The application server stores session data and the browser stores only the JSESSIONID.

Common Requirements

  • Session timeout configuration
  • Sticky session at load balancer
  • Session replication in cluster
  • Secure and HttpOnly cookie flags
  • Logout session invalidation

Docker and Cloud Perspective

Docker does not automatically make an application stateless. If the application running inside the container stores sessions in JVM memory, it is still stateful.

Stateful Docker Application

Browser
   │
JSESSIONID Cookie
   │
Docker Container
   │
Tomcat / JBoss / WebSphere Liberty
   │
Session Stored in Container JVM Memory

If the container restarts or is replaced, in-memory sessions may be lost.

Better Cloud-Native Options

  • Use JWT-based stateless authentication
  • Use Redis as external session store
  • Use database-backed session persistence
  • Use short-lived tokens with refresh tokens
  • Use API Gateway for centralized token validation

Azure Entra ID Perspective

Azure Entra ID commonly issues tokens for modern enterprise applications and APIs.

User
 │
Application
 │
Azure Entra ID Login
 │
Access Token / ID Token
 │
Application or API Access

In this model, Azure Entra ID handles authentication and issues tokens. Applications validate the token before granting access.

Enterprise Use Case:
Azure Entra ID + OAuth2/OIDC + JWT is commonly used for SSO, API authentication, Azure applications, and cloud-native platforms.

Which One Should You Use?

Application Type Recommended Approach
Traditional WebSphere / JBoss / Tomcat web app Session + Cookie
REST API JWT / OAuth2 Access Token
Docker-based application JWT or Redis-backed session
Single Page Application OIDC with secure token handling
Enterprise SSO Azure Entra ID / SAML / OIDC
Legacy Java application Session-based authentication

Common Misconceptions

Misconception 1: Cookie and Session are Same

This is incorrect. A cookie is stored in the browser. A session is stored on the server.

Misconception 2: JWT Always Replaces Session

JWT is useful for APIs and stateless architectures, but traditional web applications may still use sessions effectively.

Misconception 3: JWT is Always More Secure

JWT security depends on implementation. Poor token storage, long expiry, missing validation, or weak signing keys can create serious risks.

Misconception 4: Docker Makes Applications Stateless

Docker only packages and runs applications. If the application stores sessions in memory, it remains stateful.


Security Best Practices

Cookie Best Practices

  • Use Secure flag for HTTPS-only transmission.
  • Use HttpOnly flag to reduce JavaScript access.
  • Use SameSite attribute to reduce CSRF risk.
  • Set proper cookie path and domain.

Session Best Practices

  • Configure proper session timeout.
  • Invalidate session after logout.
  • Regenerate session ID after login.
  • Use session replication or external session store for HA.
  • Avoid storing unnecessary large objects in session.

JWT Best Practices

  • Use HTTPS for all token communication.
  • Keep access tokens short-lived.
  • Validate signature, expiry, issuer, and audience.
  • Never store passwords or secrets in JWT payload.
  • Use refresh tokens carefully.
  • Rotate signing keys periodically.

Common Production Issues

Issue Possible Cause
User logged out randomly Session timeout, server restart, or sticky session issue
Session works on one node but fails on another No session replication or affinity
JWT gives 401 Unauthorized Expired token, invalid signature, wrong audience or issuer
Cookie not sent to server Wrong domain, path, Secure flag, or SameSite setting
High JVM memory usage Too many sessions or large session objects
Login loop Cookie, SSO, reverse proxy, or token validation issue

Key Takeaways

  • A cookie is browser-side storage.
  • A session stores user data on the server.
  • JSESSIONID is commonly stored inside a cookie.
  • JWT is a signed token used for stateless authentication.
  • Sessions are common in WebSphere, JBoss, and Tomcat applications.
  • JWT is common in APIs, Docker applications, microservices, and cloud-native systems.
  • Cookies, sessions, and JWT can all be secure or insecure depending on implementation.

What’s Next?

Next Article:
Part 6 – API Authentication & API Gateway Security

In the next article, we will understand API authentication methods such as API keys, Basic Authentication, JWT, OAuth2, and how API Gateways secure backend services.


Series: Authentication & Identity Security for Middleware, DevOps & Cloud Engineers
Author: Pradeep V
Blog: MiddlewareBox.com


🎫 JWT & Token-Based Authentication Explained - Part 4

JWT & Token-Based Authentication Explained
  • Welcome to Part 4 of the Authentication & Identity Security series.
  • This article explains JWT, access tokens, refresh tokens, and bearer tokens.
  • Designed for Middleware, DevOps, Cloud, API, and Application Support Engineers.
  • Includes enterprise examples using APIs, Docker, Azure Entra ID, API Gateway, and modern authentication flows.


Introduction

In Part 3, we learned the difference between stateful and stateless applications. Traditional enterprise applications commonly use sessions and JSESSIONID, while modern applications and APIs commonly use tokens.

One of the most popular token formats used in modern authentication is JWT.

JWT Full Form:
JWT stands for JSON Web Token.

What is JWT?

JWT, or JSON Web Token, is a compact and signed token format used to securely exchange information between a client and a server.

A JWT can carry identity and authorization information such as:

  • User ID
  • Email address
  • User role
  • Issuer
  • Audience
  • Token expiry time

JWT is commonly used in APIs, microservices, cloud applications, mobile applications, and Single Sign-On systems.


Why JWT is Used

JWT is widely used because it supports stateless authentication. The application does not need to store the user session in server memory.

  • Useful for REST APIs
  • Works well with Docker-based applications
  • Supports microservices architecture
  • Easy to validate at API Gateway level
  • Used by OAuth2 and OpenID Connect flows
  • Used by identity providers such as Azure Entra ID
Cloud View:
JWT helps applications scale horizontally because any backend server can validate the token without depending on server-side session memory.

JWT Structure

A JWT contains three parts separated by dots.

Header.Payload.Signature

Example JWT format:

xxxxx.yyyyy.zzzzz

1. Header

The header contains token type and signing algorithm.

{
  "alg": "HS256",
  "typ": "JWT"
}

2. Payload

The payload contains claims. Claims are information about the user or token.

{
  "sub": "pradeep",
  "role": "admin",
  "iss": "https://login.company.com",
  "aud": "api.company.com",
  "exp": 1789459200
}

Common JWT Claims

Claim Meaning
subSubject or User ID
issIssuer of the token
audAudience or target application
expToken expiry time
iatIssued at time
roleUser role or permission

3. Signature

The signature is used to verify that the token has not been modified.

HMACSHA256(
  base64UrlEncode(header) + "." + base64UrlEncode(payload),
  secret
)
Important:
JWT payload is encoded, not encrypted by default. Do not store passwords, secrets, or sensitive personal data inside JWT payload.

How JWT Works

User
 │
 ▼
Login Request
 │
 ▼
Authentication Server
 │
 ▼
Credentials Validated
 │
 ▼
JWT Generated
 │
 ▼
JWT Returned To Client
 │
 ▼
Client Sends JWT With API Requests
 │
 ▼
API Validates JWT
 │
 ▼
Access Granted

Step 1: User Login

POST /login
username=pradeep
password=********

Step 2: JWT Returned

{
  "access_token": "eyJhbGciOiJIUzI1NiIs..."
}

Step 3: API Request With JWT

GET /api/customer
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

The API validates the token signature, expiry, issuer, audience, and claims before allowing access.


Is JWT Generated Every Time?

No. JWT is not generated for every API request.

A JWT is usually generated once during successful login or authentication. The same token is reused for multiple API requests until it expires.

Login
 │
JWT Generated Once
 │
Same JWT Sent With Every API Request
 │
Token Expires
 │
New Token Required

When is a New JWT Generated?

  • When the user logs in again
  • When the access token expires and a refresh token is used
  • When the authentication server rotates or renews tokens
  • When the user signs out and signs in again
Important:
The client sends the same JWT with every request using the Authorization header until the token expires.

Access Token, Refresh Token & Bearer Token

Token Type Purpose Typical Lifetime
Access Token Used to access APIs and protected resources Short-lived
Refresh Token Used to get a new access token Longer-lived
Bearer Token Token sent in Authorization header Depends on token expiry

Bearer Token Example

Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

Bearer means whoever has the token can use it. That is why token storage and transport security are very important.


Enterprise Middleware Example

Traditional Session-Based Application

Browser
   │
JSESSIONID Cookie
   │
IBM HTTP Server
   │
WebSphere / JBoss / Tomcat
   │
Session Stored in JVM Memory

This is commonly used in traditional enterprise web applications.

Modern JWT-Based API

Client Application
   │
Authorization: Bearer JWT
   │
API Gateway
   │
Backend API / Microservice
   │
Token Validated

This is commonly used in APIs, microservices, Docker deployments, and cloud-native applications.


Azure Entra ID Example

Azure Entra ID can act as an identity provider and issue tokens after successful authentication.

User
 │
 ▼
Application
 │
 ▼
Azure Entra ID Login
 │
 ▼
MFA / Conditional Access
 │
 ▼
Access Token Issued
 │
 ▼
Application Calls API

The API validates the token issued by Azure Entra ID before allowing access to protected resources.

Enterprise Use Case:
Azure Entra ID tokens are commonly used for Microsoft 365, Azure Portal, enterprise applications, APIs, and Single Sign-On integrations.

JWT Security Best Practices

  • Always use HTTPS to protect tokens in transit.
  • Keep access tokens short-lived.
  • Use refresh tokens carefully and store them securely.
  • Validate token signature on every request.
  • Validate issuer and audience claims.
  • Never store passwords or secrets inside JWT payload.
  • Use secure storage for tokens in browser or mobile applications.
  • Rotate signing keys periodically.
  • Use API Gateway or identity middleware for centralized validation.

Session vs JWT

Point Session JWT
ArchitectureStatefulStateless
User DataStored on serverStored as token claims
IdentifierJSESSIONIDBearer Token
ScalingNeeds sticky session or replicationEasier horizontal scaling
Common UseTraditional web appsAPIs and microservices
Example PlatformWebSphere, JBoss, TomcatAPI Gateway, Docker, Cloud APIs

Common JWT Issues

Issue Possible Cause
401 UnauthorizedToken expired or invalid
Invalid signatureWrong signing key or modified token
Audience validation failedToken issued for different API
Issuer validation failedWrong identity provider configured
Token too largeToo many claims added
Token theft riskInsecure storage or missing HTTPS

Key Takeaways

  • JWT stands for JSON Web Token.
  • JWT contains Header, Payload, and Signature.
  • JWT is commonly used for stateless authentication.
  • A JWT is usually generated during login and reused until expiry.
  • Access tokens are used to access APIs.
  • Refresh tokens are used to get new access tokens.
  • Bearer tokens are sent in the Authorization header.
  • JWT is widely used in APIs, Docker applications, cloud platforms, and microservices.

What’s Next?

Next Article:
Part 5 – JWT vs Session vs Cookies

In the next article, we will compare JWT, sessions, and cookies in detail and understand when to use each approach in enterprise applications.


Series: Authentication & Identity Security for Middleware, DevOps & Cloud Engineers
Author: Pradeep V
Blog: MiddlewareBox.com


πŸ”„ Stateful vs Stateless Applications Explained - Part 3

Stateful vs Stateless Applications Explained
  • Welcome to Part 3 of the Authentication & Identity Security series.
  • This article explains the difference between stateful and stateless applications.
  • Designed for Middleware, DevOps, Cloud, and Application Support Engineers.
  • Includes enterprise examples using WebSphere, JBoss, Tomcat, Docker, JWT (JWT = JSON Web Token), APIs, and Load Balancers.


Introduction

In Part 2, we learned how sessions, cookies, and JSESSIONID help applications remember users after login. This leads to an important architecture concept: Stateful vs Stateless Applications.

Understanding this difference is very important for Middleware, DevOps, and Cloud Engineers because it impacts application scaling, load balancing, failover, session management, and cloud migration.

Key Concept:
Stateful applications remember user/session information on the server. Stateless applications do not store user state on the server between requests.

What is State?

State means information that the application remembers about a user, request, or transaction.

Examples of state include:

  • User login session
  • Shopping cart data
  • User role and permissions
  • Workflow step
  • Temporary transaction data

If the server stores this information between requests, the application is usually considered stateful.


What is a Stateful Application?

A stateful application stores user/session data on the server side. The server remembers the user's previous activity.

Example

User Login
   │
   ▼
Application Server Creates Session
   │
   ▼
Session Stored in JVM Memory
   │
   ▼
User Continues Application Access

Traditional Java enterprise applications running on WebSphere, JBoss, or Tomcat are commonly stateful when they use server-side HTTP sessions.

Common Stateful Technologies

  • WebSphere HTTP Sessions
  • JBoss / Tomcat Sessions
  • JSESSIONID-based applications
  • Session replication
  • Sticky sessions
  • Database-backed sessions
Middleware View:
If your application depends on JSESSIONID and server-side session memory, it is usually stateful.

What is a Stateless Application?

A stateless application does not store user/session data on the application server between requests. Every request contains enough information for the server to process it independently.

Example

Client Request
   │
   ▼
Authorization: Bearer JWT Token
   │
   ▼
API Server Validates Token
   │
   ▼
Response Sent Back

Modern APIs and microservices commonly use stateless authentication with JWT tokens.

Common Stateless Technologies

  • JWT Tokens
  • REST APIs
  • OAuth2 Access Tokens
  • API Gateway Authentication
  • Cloud-native applications
Cloud View:
Stateless applications are easier to scale because any server can process any request.

Stateful vs Stateless Flow

Stateful Flow

Browser
   │
Cookie: JSESSIONID
   │
Load Balancer
   │
WebSphere / Tomcat / JBoss
   │
Session Stored in Server Memory

Stateless Flow

Client
   │
Authorization: Bearer JWT
   │
API Gateway
   │
Backend API
   │
No Server-Side Session Required

Enterprise Middleware Example

Stateful WebSphere Application

Browser
   │
IBM HTTP Server
   │
WebSphere Cluster
   │
Application Session in JVM Heap

In this model, the user session is stored inside the WebSphere JVM. If the next request goes to another cluster member, the user session may be lost unless sticky session or session replication is configured.

Stateful Example Issue

User logs in through WAS01
Next request goes to WAS02
Session not found
User gets logged out

Solution

  • Enable sticky sessions at IBM HTTP Server or Load Balancer.
  • Configure memory-to-memory session replication.
  • Use database or external session persistence.

Docker Example

Docker itself does not make an application stateless. The behavior depends on how the application stores state.

Stateful Docker Application

Browser
   │
Docker Container
   │
Tomcat / JBoss / WebSphere Liberty
   │
Session Stored in Container JVM Memory

If the container is restarted or replaced, in-memory sessions may be lost.

Stateless Docker Application

Client
   │
JWT Token
   │
Docker Container
   │
API Service
   │
No Local Session Storage

A stateless containerized application uses JWT tokens or external systems to avoid storing user sessions inside the container.

Docker Best Practice:
For scalable Docker-based applications, avoid storing critical session state only inside container memory. Use Redis, database persistence, or JWT-based stateless authentication.

Stateful vs Stateless Comparison

Point Stateful Application Stateless Application
Session Storage Stored on server Not stored on server
Example JSESSIONID session JWT token
Scaling More complex Easier
Load Balancer May require sticky sessions Any server can handle request
Failover Needs replication/persistence Easier if token is valid
Common Use Traditional web applications APIs and microservices
Middleware Example WebSphere / Tomcat session API with JWT

Session vs JWT

One of the most common questions in modern application architecture is whether to use traditional sessions or JWT-based authentication.

Feature Session-Based Authentication JWT-Based Authentication
Architecture TypeStatefulStateless
User StateStored on ServerStored in Token
IdentifierJSESSIONIDBearer Token
Storage LocationApplication ServerClient Side
ScalingMore ComplexEasier
Load Balancer DependencyOften Requires Sticky SessionsNo Sticky Session Required
Common UsageTraditional Web ApplicationsAPIs & Microservices
Middleware Perspective:
Traditional WebSphere, JBoss, and Tomcat applications commonly use sessions. Modern APIs, Docker deployments, cloud-native applications, and microservices commonly use JWT.

Where to Use Stateful and Stateless

Use Case Recommended Approach
Traditional enterprise web application Stateful session with sticky session or replication
Banking portal with complex workflow Stateful or hybrid approach
REST API Stateless with JWT/OAuth2 token
Microservices Stateless preferred
Docker-based application Stateless preferred, or external session store
Legacy WebSphere application Stateful session management

Common Production Issues

Middleware and DevOps teams frequently troubleshoot issues related to stateful and stateless application design.

Issue Possible Cause
User logged out after failover Session stored only in one JVM
Intermittent session loss Sticky session not configured
Container restart logs out users Session stored inside container memory
API request fails with 401 JWT token expired or invalid
Scaling issue Stateful design without external session store
High JVM memory usage Too many active sessions stored in heap

Key Takeaways

  • State means information remembered by the application.
  • Stateful applications store session data on the server.
  • Stateless applications process each request independently.
  • JSESSIONID-based applications are usually stateful.
  • JWT-based APIs are usually stateless.
  • Docker does not automatically make applications stateless.
  • Stateless design is preferred for APIs, microservices, and scalable cloud applications.
  • Stateful applications need sticky sessions, replication, or external session storage.

What’s Next?

Next Article:
Part 4 – JWT & Token-Based Authentication Explained

In the next article, we will understand JWT, access tokens, refresh tokens, bearer tokens, and how token-based authentication works in modern applications and APIs.


Series: Authentication & Identity Security for Middleware, DevOps & Cloud Engineers
Author: Pradeep V
Blog: MiddlewareBox.com


2 Nov 2025

🐧 Linux Shell Scripting for Beginners – Complete Tutorial Series.

  • A 13-part practical shell scripting course for DevOps and Middleware Engineers.

πŸ’‘ How This Tutorial Series Will Help You

  • ✔ Build a strong foundation in Linux Shell scripting — from basics to automation.
  • ✔ Automate daily Middleware and DevOps tasks: monitoring, backups, and log management.
  • ✔ Write scripts for Middleware platforms like Tomcat, Jenkins, and WebSphere.
  • ✔ Debug, schedule, and manage production-ready shell scripts confidently.
  • ✔ Perfect for Sysadmins, Middleware, and DevOps Engineers upgrading to Cloud roles.

πŸ“š Complete Course Index

🐚 Part 1: What is a Variable?
Learn how variables store data and simplify automation scripts. Foundation of all shell logic.
πŸ“₯ Part 2: Reading User Input in Shell
Make interactive scripts using read command with Jenkins and Tomcat examples.
Use decision-making in your scripts with examples checking files, services, and network health.
Automate repetitive DevOps tasks like log cleanup, backups, and status monitoring.
Write modular, reusable functions — restart Tomcat or back up Jenkins in one click.
πŸ“¦ Part 6: Arrays & Arguments
Store multiple values, manage user inputs, and process server lists in scripts.
πŸ“ Part 7: File Handling in Shell
Learn how to read, write, append, and handle configuration or log files safely.
πŸ’‘ Part 8: Debugging & Logging
Find and fix script errors, use logs, and enable debug mode for safe execution.
πŸ•’ Part 9: Scheduling & Automation
Use cron and @reboot jobs to run scripts automatically for health checks or backups.
Create a production-ready monitoring script for Tomcat, Jenkins, and NGINX with auto-restart.
🧾 Part 11: Log Management & Rotation
Automate log rotation with logrotate and custom retention scripts for Middleware systems.
Perform automated backup and restore for Jenkins, Tomcat, and WebSphere with rollback support.
Run multiple jobs in parallel, monitor background processes, and speed up automation workflows.

πŸš€ Why MiddlewareBox Shell Scripting Series?

  • ✅ Step-by-step, beginner-friendly explanations.
  • ✅ Each topic includes real DevOps & Middleware context.
  • ✅ Examples with Output and Error handling.
  • ✅ Simple language for better understanding.

✨ Start Learning Now → Part 1: What is a Variable?
& build your own scripts from scratch! πŸ’ͺ

⚙️ Shell Scripting for Beginners – Part 13: Subshells, Process Control & Parallel Execution.

  • Now let's level up and make your shell scripts multitask like a Middleware and DevOps pro.
  • In this part, you'll learn how to run processes in the background, manage subshells, and execute multiple tasks in parallel for faster automation.

πŸ“‘ Table of Contents


1️⃣ What is a Subshell?

A subshell is a new child shell launched from your current shell. It allows running commands in isolation — any variable or environment change inside doesn’t affect the parent shell.

#!/bin/bash
VAR="ParentShell"
( VAR="SubShell"; echo "Inside subshell: $VAR" )
echo "Outside subshell: $VAR"
πŸ’¬ Output:
Inside subshell: SubShell
Outside subshell: ParentShell

2️⃣ Running Commands in the Background

Add & at the end of a command to run it in the background. This allows your script to keep running while other tasks continue behind the scenes.

#!/bin/bash
echo "Starting Jenkins backup..."
tar -czf /opt/backups/jenkins_backup.tar.gz /var/lib/jenkins & 
echo "Backup started in background (PID: $!)"
πŸ’¬ Output:
Starting Jenkins backup...
Backup started in background (PID: 2356)

3️⃣ Managing Jobs

When you run background processes, you can manage them using built-in commands:

  • jobs → List current background jobs
  • fg %1 → Bring job 1 to foreground
  • bg %1 → Resume job 1 in background
  • kill %1 → Kill job 1
#!/bin/bash
long_task() {
  sleep 30 &
}
long_task
jobs
πŸ’¬ Output:
[1]+ Running sleep 30 &

4️⃣ Parallel Execution in DevOps

Let’s say you want to restart multiple Tomcat servers or back up several apps simultaneously. We can use background jobs with a simple for loop.

#!/bin/bash
servers=("devbox" "testbox" "prodbox")
for srv in "${servers[@]}"; do
  echo "Restarting Tomcat on $srv..."
  ssh $srv "systemctl restart tomcat" &
done

wait  # Wait for all background jobs to finish
echo "✅ All Tomcat servers restarted in parallel!"
πŸ’¬ Output:
Restarting Tomcat on devbox...
Restarting Tomcat on testbox...
Restarting Tomcat on prodbox...
✅ All Tomcat servers restarted in parallel!

5️⃣ nohup & disown

Sometimes, you need your scripts to survive terminal disconnections — for instance, a long-running Jenkins restore or DB migration.

#!/bin/bash
nohup ./db_migration.sh > /var/log/db_migration.log 2>&1 &
disown
echo "🧠 DB migration running safely in background, even if you close terminal."
πŸ’¬ Output:
nohup: ignoring input and appending output to 'nohup.out'
🧠 DB migration running safely in background, even if you close terminal.

6️⃣ Real-World DevOps Examples

🧩 Example 1: Parallel Log Collection

#!/bin/bash
servers=("dev" "test" "prod")
for s in "${servers[@]}"; do
  scp $s:/var/log/tomcat/catalina.out /opt/logs/$s.log &
done
wait
echo "✅ Logs collected from all servers."

🧰 Example 2: Monitor Jenkins & Docker in Background

#!/bin/bash
( while true; do pgrep -f jenkins >/dev/null || systemctl restart jenkins; sleep 10; done ) &
( while true; do docker ps | grep -q "webapp" || docker start webapp; sleep 15; done ) &
echo "🩺 Jenkins and Docker monitored in background."

πŸ“¦ Example 3: Run Multiple CI Jobs Simultaneously

#!/bin/bash
for job in build-test build-stage build-prod; do
  curl -s -X POST "http://jenkins.local/job/$job/build" &
done
wait
echo "✅ All Jenkins build jobs triggered in parallel!"

🧠 Pro Tips & Troubleshooting

🧩 Process Control Best Practices

  • Use wait to synchronize parallel tasks before continuing.
  • Log PIDs ($!) to monitor background tasks easily.
  • Redirect background job output to separate log files using > file 2>&1 &.
  • For persistent background services, prefer systemd over manual nohup scripts.

⚙️ Debugging Issues

  • Run ps -ef | grep scriptname to confirm background processes.
  • Use set -x or bash -x script.sh for detailed command tracing.
  • Use trap to clean up child processes on script exit.
  • If parallel SSH commands hang, add -o ConnectTimeout=5 in SSH for safety.

🏁 Summary

  • ✅ Learned to create and manage subshells.
  • ⚙️ Mastered process control using &, jobs, fg, bg, and kill.
  • πŸš€ Ran real-world DevOps tasks in parallel — Jenkins builds, Tomcat restarts, Docker checks.
  • πŸ’‘ Used nohup and disown for persistent automation tasks.

πŸ’Ύ Shell Scripting for Beginners – Part 12: Backup & Restore Automation Project (DevOps & Middleware Edition)

  • Here we'll automate backup and restore tasks for real middleware & DevOps systems like Jenkins, Tomcat, WebSphere, MySQL, Docker, and Apache.
  • Each example includes a working restore script and troubleshooting tips. ⚙️

πŸ“‘ Table of Contents


1️⃣ Jenkins Backup & Restore

#!/bin/bash
JENKINS_HOME="/var/lib/jenkins"
BACKUP_DIR="/opt/backups/jenkins"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/jenkins_$(date +%F_%H-%M).tar.gz -C $JENKINS_HOME .
echo "✅ Jenkins backup created in $BACKUP_DIR"

♻️ Restore Jenkins

#!/bin/bash
BACKUP_FILE="/opt/backups/jenkins/jenkins_2025-11-02_02-10.tar.gz"
service jenkins stop
tar -xzf "$BACKUP_FILE" -C /var/lib/jenkins
chown -R jenkins:jenkins /var/lib/jenkins
service jenkins start
echo "♻️ Jenkins restored successfully."

2️⃣ Tomcat Backup & Restore

#!/bin/bash
TOMCAT_HOME="/opt/tomcat"
BACKUP_DIR="/opt/backups/tomcat"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/tomcat_$(date +%F).tar.gz $TOMCAT_HOME/webapps $TOMCAT_HOME/conf
echo "πŸ“¦ Tomcat backup completed."

♻️ Restore Tomcat

#!/bin/bash
BACKUP_FILE="/opt/backups/tomcat/tomcat_2025-11-02.tar.gz"
service tomcat stop
tar -xzf "$BACKUP_FILE" -C /
service tomcat start
echo "♻️ Tomcat restored and restarted."

3️⃣ WebSphere Backup & Restore

#!/bin/bash
WAS_PROFILE="/opt/IBM/WebSphere/AppServer/profiles/AppSrv01"
BACKUP_DIR="/opt/backups/websphere"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/was_$(date +%F_%H-%M).tar.gz -C $WAS_PROFILE .
echo "πŸ“ WebSphere configuration backup done."

♻️ Restore WebSphere

#!/bin/bash
BACKUP_FILE="/opt/backups/websphere/was_2025-11-02_03-00.tar.gz"
service was stop
tar -xzf "$BACKUP_FILE" -C /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
service was start
echo "♻️ WebSphere restored successfully."

4️⃣ MySQL Database Backup & Restore

#!/bin/bash
DB="middlewaredb"
USER="root"
PASS="Secret123"
BACKUP_DIR="/opt/backups/mysql"
mkdir -p $BACKUP_DIR
mysqldump -u $USER -p$PASS $DB > $BACKUP_DIR/${DB}_$(date +%F).sql
gzip $BACKUP_DIR/${DB}_$(date +%F).sql
echo "πŸ—„️ MySQL backup completed."

♻️ Restore MySQL

#!/bin/bash
BACKUP_FILE="/opt/backups/mysql/middlewaredb_2025-11-02.sql.gz"
gunzip "$BACKUP_FILE"
mysql -u root -pSecret123 middlewaredb < /opt/backups/mysql/middlewaredb_2025-11-02.sql
echo "✅ Database restored successfully."

5️⃣ Docker Containers Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/docker"
mkdir -p $BACKUP_DIR
for cid in $(docker ps -q); do
  cname=$(docker inspect --format='{{.Name}}' $cid | cut -d'/' -f2)
  docker export $cid > $BACKUP_DIR/${cname}_$(date +%F).tar
done
echo "🐳 Docker containers exported successfully."

♻️ Restore Docker Container

#!/bin/bash
BACKUP_FILE="/opt/backups/docker/myapp_2025-11-02.tar"
docker import "$BACKUP_FILE" myapp_restored:latest
docker run -d --name myapp_restored myapp_restored:latest
echo "♻️ Docker container restored and running."

6️⃣ Apache Webserver Backup & Restore

#!/bin/bash
BACKUP_DIR="/opt/backups/apache"
mkdir -p $BACKUP_DIR
tar -czf $BACKUP_DIR/apache_$(date +%F).tar.gz /etc/apache2 /var/www/html /etc/ssl
echo "🌐 Apache configuration, website, and SSL backed up."

♻️ Restore Apache

#!/bin/bash
BACKUP_FILE="/opt/backups/apache/apache_2025-11-02.tar.gz"
tar -xzf "$BACKUP_FILE" -C /
service apache2 restart
echo "✅ Apache restored and running."

7️⃣ Automating Backups

Schedule automatic backups using cron so your systems stay protected 24×7.

# Run daily at 2 AM
0 2 * * * /opt/scripts/backup_all.sh >> /var/log/backup_all.log 2>&1

# Run on every reboot
@reboot /opt/scripts/backup_all.sh >> /var/log/backup_boot.log 2>&1
πŸ’¬ Output:
[CRON] Scheduled backup started…
✅ Jenkins, Tomcat, WebSphere, MySQL, Docker, Apache backed up successfully.

🧠 Pro Tips & Troubleshooting

✅ General Best Practices

  • Use a dedicated /opt/backups partition or NFS mount to avoid filling system drives.
  • Compress large backups with gzip or zstd [Developed by Meta] for better space efficiency.
  • Store daily, weekly, and monthly copies separately (retention policy).
  • Automate restores in a test environment weekly to ensure recovery works.

⚙️ Troubleshooting Common Issues

  • Permission Denied: Run backups as sudo or fix ownership using chown.
  • πŸ“¦ Disk Full: Use df -h to check space before backup. Add cleanup logic for old files.
  • 🧾 Log Rotation: Add logrotate entry to keep backup logs small and readable.

🏁 Summary

  • ✅ Full backup & restore automation for all major middleware and DevOps tools.
  • 🧩 Scripts are modular, reusable, and easy to integrate in CI/CD pipelines.
  • ⚙️ Cron ensures zero manual intervention for daily protection.

1 Nov 2025

🧾 Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

  • Logs are the heartbeat of your servers πŸ’“ — but if left unmanaged, they'll crash your systems.
  • In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

πŸ“‘ Table of Contents


1️⃣ Log Cleanup Scripts

#!/bin/bash
# Delete logs older than 7 days for Tomcat & Jenkins
find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
echo "🧹 Tomcat & Jenkins logs older than 7 days removed!"
πŸ’¬ Output:
🧹 Tomcat & Jenkins logs older than 7 days removed!

2️⃣ Apache & Middleware Examples

#!/bin/bash
ARCHIVE="/opt/log_archive/apache"
mkdir -p $ARCHIVE
find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
echo "πŸ“¦ Apache logs compressed and archived in $ARCHIVE"
πŸ’¬ Output:
πŸ“¦ Apache logs compressed and archived in /opt/log_archive/apache

3️⃣ Docker & Kubernetes Log Cleanup

#!/bin/bash
echo "🐳 Cleaning Docker & K8s logs..."
docker system prune -af
find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
echo "✅ Docker and Kubernetes logs cleaned!"
πŸ’¬ Output:
✅ Docker and Kubernetes logs cleaned!

4️⃣ logrotate Configuration

# /etc/logrotate.d/devops-services
/opt/tomcat/logs/*.log
/var/log/apache2/*.log
/var/lib/jenkins/logs/*.log
/var/log/docker/*.log {
    weekly
    rotate 6
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2 2>/dev/null || true
        systemctl restart tomcat 2>/dev/null || true
    endscript
}
πŸ’¬ Result:
πŸ” Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

5️⃣ How to Run & Test logrotate

Once your config is ready, you can run logrotate manually or let cron handle it.

🧠 Step 1 – Check Syntax of Your Config

sudo logrotate -d /etc/logrotate.d/devops-services
πŸ’¬ Output:
Reading config file /etc/logrotate.d/devops-services
Handling /opt/tomcat/logs/*.log
Handling /var/log/apache2/*.log
**dry-run mode (no rotation performed)**

⚙️ Step 2 – Run logrotate Manually

sudo logrotate -f /etc/logrotate.d/devops-services
πŸ’¬ Output:
Rotating logs for Tomcat, Apache, Jenkins...
Compression complete.
Rotation successful ✅

⏰ Step 3 – Schedule via Cron (Daily or Weekly)

# /etc/cron.daily/logrotate
/usr/sbin/logrotate /etc/logrotate.conf

πŸ’‘ By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.


🏁 Summary

  • ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
  • ✅ Created unified logrotate configuration.
  • ✅ Learned to manually test (-d) and force-run (-f) logrotate.
  • ✅ Automated rotation through cron for 24×7 log hygiene.

🎯Shell Scripting for Beginners – Part 10: Middleware Health Monitoring Project (Mixed Example).

  • This is our final, beginner-friendly project 🎯 — where we'll combine everything you've learned: variables, if-else, loops, functions, logging, and cron automation.
  • Let's create a simple Middleware Health Monitoring script that checks Tomcat, Jenkins, Docker, MySQL, and SSL expiry.

πŸ“‘ Table of Contents


1️⃣ Overview

This script uses:

  • Variables – store names, logs, and domains.
  • Functions – reusable checks for each service.
  • If-Else – to decide if a service needs restart.
  • For loop – to check multiple Docker containers.
  • While loop – to retry connection.
  • Logs – write output to a file.

2️⃣ Full Middleware Health Monitoring Script

#!/bin/bash
# MiddlewareBox - Simple Health Monitor

LOG="/var/log/middleware_health.log"
domain="middlewarebox.com"
echo "===== Middleware Health Check =====" >> $LOG
date >> $LOG

# --- Function to check Tomcat ---
check_tomcat() {
  echo "Checking Tomcat..."
  if pgrep -f "org.apache.catalina.startup.Bootstrap" > /dev/null; then
     echo "✅ Tomcat is running" | tee -a $LOG
  else
     echo "🚨 Tomcat is down, restarting..." | tee -a $LOG
     /opt/tomcat/bin/startup.sh
     echo "✅ Tomcat restarted" | tee -a $LOG
  fi
}

# --- Function to check Jenkins ---
check_jenkins() {
  echo "Checking Jenkins..."
  if systemctl is-active --quiet jenkins; then
     echo "✅ Jenkins is active" | tee -a $LOG
  else
     echo "🚨 Jenkins not running, restarting..." | tee -a $LOG
     systemctl restart jenkins
     echo "✅ Jenkins restarted" | tee -a $LOG
  fi
}

# --- Function to check MySQL ---
check_db() {
  echo "Checking MySQL..."
  if mysql -u root -pSecret -e "show databases;" > /dev/null 2>&1; then
     echo "✅ MySQL connection successful" | tee -a $LOG
  else
     echo "🚨 MySQL connection failed" | tee -a $LOG
     echo "Retrying in 5 seconds..."
     sleep 5
     mysql -u root -pSecret -e "show databases;" && echo "✅ MySQL reconnected" | tee -a $LOG
  fi
}

# --- Check Docker Containers using FOR Loop ---
check_docker() {
  echo "Checking Docker containers..."
  for c in nginx webapp db
  do
    status=$(docker inspect -f '{{.State.Status}}' $c 2>/dev/null)
    if [ "$status" != "running" ]; then
      echo "🚨 Container $c is $status, restarting..." | tee -a $LOG
      docker start $c
    else
      echo "✅ Container $c is running" | tee -a $LOG
    fi
  done
}

# --- Check SSL certificate expiry ---
check_ssl() {
  echo "Checking SSL expiry..."
  expiry=$(echo | openssl s_client -servername $domain -connect $domain:443 2>/dev/null | openssl x509 -noout -dates | grep notAfter | cut -d= -f2)
  days_left=$(( ($(date -d "$expiry" +%s) - $(date +%s)) / 86400 ))
  if [ $days_left -le 15 ]; then
    echo "⚠️ SSL expires in $days_left days!" | tee -a $LOG
  else
    echo "✅ SSL valid for $days_left days" | tee -a $LOG
  fi
}

# --- MAIN EXECUTION FLOW ---
check_tomcat
check_jenkins
check_db
check_docker
check_ssl

echo "✅ Health check completed successfully!" | tee -a $LOG
echo "========================================" >> $LOG
πŸ’¬ Example Output:
✅ Tomcat is running
✅ Jenkins is active
✅ MySQL connection successful
✅ Container nginx is running
✅ SSL valid for 70 days
✅ Health check completed successfully!

3️⃣ Automate the Script

# Run every 30 minutes
*/30 * * * * /opt/scripts/middleware_health.sh >> /var/log/health_cron.log 2>&1

# Run automatically on system reboot
@reboot /opt/scripts/middleware_health.sh >> /var/log/boot_health.log 2>&1
πŸ’¬ Output (Cron log):
[BOOT] Health check executed
✅ All services healthy

🏁 Summary

  • ✅ Combined all major shell scripting concepts.
  • ✅ Checks Tomcat, Jenkins, Docker, MySQL, and SSL in one file.
  • ✅ Uses if, for, functions, and basic while retry logic.
  • ✅ Can run manually or via cron / reboot automation.

26 Oct 2025

πŸš€DevOps vs DevSecOps Explained: Simple Workflows, Tools, and Learning Path for Beginners

devSecops
DevOps vs DevSecOps
  • In today’s fast-moving IT world, DevOps and DevSecOps form the backbone of modern and efficient software delivery.
  • This blog explains both step-by-step — using simple workflows, real-world examples, and common tool references.

1️⃣ Why DevOps Exists

  • Before DevOps, software teams worked in silos — developers built code while operations deployed and maintained it.
  • This caused delays, inconsistent environments, and deployment failures.
  • Testing and production behaved differently, leading to rework and slow releases.
  • DevOps emerged to bridge this gap using automation, collaboration, and continuous feedback.

2️⃣ DevOps in Simple Terms

  • DevOps combines development and operations into a single continuous workflow.
  • It focuses on automation, collaboration, and continuous improvement.
  • It reduces manual effort, minimizes human error, and ensures consistent releases.
  • The goal: faster delivery, higher quality, and continuous innovation.

3️⃣ DevOps Workflow (Simple View)

πŸ‘¨‍πŸ’» Developer → πŸͺ£ Git (Commit Code) → ⚙️ Jenkins (Build & Test) → πŸ§ͺ Docker (Package App) → ☸️ Kubernetes (Deploy to Cluster) → πŸ“ˆ Grafana (Monitor)

This is how DevOps automates the pipeline — from writing code to deploying and monitoring applications.

4️⃣ Common DevOps Tools

🧩 Real-World Example: DevOps

  • A Fintech company automates deployments using GitHub → Jenkins → Docker → Kubernetes → Grafana.
  • Each code push triggers a CI/CD pipeline that builds, tests, and deploys containers — reducing deployment time from hours to minutes.

6️⃣ What is DevSecOps?

  • DevSecOps integrates security practices into every stage of the DevOps pipeline.
  • It ensures that security is a shared responsibility across development, operations, and security teams.
  • Vulnerabilities are detected early in the build, deploy, or run phase — not after release.
  • Tools like SonarQube, Snyk, and Trivy automate vulnerability detection, while Vault protects secrets and Falco monitors runtime threats.

7️⃣ DevOps vs DevSecOps Workflow

DevOps: πŸ‘¨‍πŸ’» Developer → Git → Jenkins → Docker → Kubernetes → Grafana

DevSecOps: πŸ‘¨‍πŸ’» Developer → Git + SonarQube → Jenkins + Snyk → Docker + Trivy → Kubernetes + Vault → Falco → Grafana

8️⃣ Common DevSecOps Tools

  • SonarQube: Scans source code for vulnerabilities.
  • Snyk: Detects insecure dependencies during builds.
  • Trivy: Scans Docker images for known vulnerabilities.
  • Vault: Manages application secrets securely.
  • Falco: Detects abnormal runtime behavior in containers.

🧩 Real-World Example: DevSecOps

  • A Healthcare provider integrates security into its CI/CD pipeline using SonarQube, Snyk, Trivy, and Vault.
  • Falco monitors runtime threats — ensuring compliance and secure releases.

πŸ”Ÿ Key Differences Between DevOps & DevSecOps

Aspect DevOps DevSecOps
Focus Speed & automation Speed with security
Security Added later in the process Integrated from the start (shift-left)
Goal Deliver software quickly Deliver software quickly & securely
Tools Git, Jenkins, Docker, Kubernetes SonarQube, Snyk, Trivy, Vault, Falco
Responsibility Dev & Ops teams Dev, Ops & Security teams
Outcome Fast delivery Secure & compliant delivery

1️⃣1️⃣ Summary & Next Blog

  • DevOps = Automate everything for speed and collaboration.
  • DevSecOps = Add security automation to every stage.
  • πŸ’‘ Learn DevOps first, then extend it to DevSecOps for end-to-end security integration.