Menu

Showing posts with label Kubernetes. Show all posts
Showing posts with label Kubernetes. Show all posts

1 Nov 2025

🧾 Shell Scripting for Beginners – Part 11: Log Management & Rotation for Middleware & DevOps Projects

  • Logs are the heartbeat of your servers πŸ’“ — but if left unmanaged, they'll crash your systems.
  • In this part, you'll learn how to clean, compress, and rotate logs for Tomcat, Jenkins, Apache, Docker, Kubernetes, and system services using shell scripts and logrotate.

πŸ“‘ Table of Contents


1️⃣ Log Cleanup Scripts

#!/bin/bash
# Delete logs older than 7 days for Tomcat & Jenkins
find /opt/tomcat/logs/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
find /var/log/jenkins/ -type f -name "*.log" -mtime +7 -exec rm -f {} \;
echo "🧹 Tomcat & Jenkins logs older than 7 days removed!"
πŸ’¬ Output:
🧹 Tomcat & Jenkins logs older than 7 days removed!

2️⃣ Apache & Middleware Examples

#!/bin/bash
ARCHIVE="/opt/log_archive/apache"
mkdir -p $ARCHIVE
find /var/log/apache2/ -type f -name "*.log" -mtime +3 -exec gzip {} \;
mv /var/log/apache2/*.gz $ARCHIVE 2>/dev/null
echo "πŸ“¦ Apache logs compressed and archived in $ARCHIVE"
πŸ’¬ Output:
πŸ“¦ Apache logs compressed and archived in /opt/log_archive/apache

3️⃣ Docker & Kubernetes Log Cleanup

#!/bin/bash
echo "🐳 Cleaning Docker & K8s logs..."
docker system prune -af
find /var/log/containers/ -type f -name "*.log" -mtime +5 -exec rm -f {} \;
echo "✅ Docker and Kubernetes logs cleaned!"
πŸ’¬ Output:
✅ Docker and Kubernetes logs cleaned!

4️⃣ logrotate Configuration

# /etc/logrotate.d/devops-services
/opt/tomcat/logs/*.log
/var/log/apache2/*.log
/var/lib/jenkins/logs/*.log
/var/log/docker/*.log {
    weekly
    rotate 6
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload apache2 2>/dev/null || true
        systemctl restart tomcat 2>/dev/null || true
    endscript
}
πŸ’¬ Result:
πŸ” Logs for Apache, Tomcat, Jenkins, and Docker will rotate weekly and auto-restart services.

5️⃣ How to Run & Test logrotate

Once your config is ready, you can run logrotate manually or let cron handle it.

🧠 Step 1 – Check Syntax of Your Config

sudo logrotate -d /etc/logrotate.d/devops-services
πŸ’¬ Output:
Reading config file /etc/logrotate.d/devops-services
Handling /opt/tomcat/logs/*.log
Handling /var/log/apache2/*.log
**dry-run mode (no rotation performed)**

⚙️ Step 2 – Run logrotate Manually

sudo logrotate -f /etc/logrotate.d/devops-services
πŸ’¬ Output:
Rotating logs for Tomcat, Apache, Jenkins...
Compression complete.
Rotation successful ✅

⏰ Step 3 – Schedule via Cron (Daily or Weekly)

# /etc/cron.daily/logrotate
/usr/sbin/logrotate /etc/logrotate.conf

πŸ’‘ By default, most Linux systems already run logrotate daily using this cron job. You can verify the last run log at /var/lib/logrotate/status.


🏁 Summary

  • ✅ Cleaned and archived logs for Tomcat, Jenkins, Apache, Docker, and K8s.
  • ✅ Created unified logrotate configuration.
  • ✅ Learned to manually test (-d) and force-run (-f) logrotate.
  • ✅ Automated rotation through cron for 24×7 log hygiene.

26 Oct 2025

πŸš€DevOps vs DevSecOps Explained: Simple Workflows, Tools, and Learning Path for Beginners

devSecops
DevOps vs DevSecOps
  • In today’s fast-moving IT world, DevOps and DevSecOps form the backbone of modern and efficient software delivery.
  • This blog explains both step-by-step — using simple workflows, real-world examples, and common tool references.

1️⃣ Why DevOps Exists

  • Before DevOps, software teams worked in silos — developers built code while operations deployed and maintained it.
  • This caused delays, inconsistent environments, and deployment failures.
  • Testing and production behaved differently, leading to rework and slow releases.
  • DevOps emerged to bridge this gap using automation, collaboration, and continuous feedback.

2️⃣ DevOps in Simple Terms

  • DevOps combines development and operations into a single continuous workflow.
  • It focuses on automation, collaboration, and continuous improvement.
  • It reduces manual effort, minimizes human error, and ensures consistent releases.
  • The goal: faster delivery, higher quality, and continuous innovation.

3️⃣ DevOps Workflow (Simple View)

πŸ‘¨‍πŸ’» Developer → πŸͺ£ Git (Commit Code) → ⚙️ Jenkins (Build & Test) → πŸ§ͺ Docker (Package App) → ☸️ Kubernetes (Deploy to Cluster) → πŸ“ˆ Grafana (Monitor)

This is how DevOps automates the pipeline — from writing code to deploying and monitoring applications.

4️⃣ Common DevOps Tools

🧩 Real-World Example: DevOps

  • A Fintech company automates deployments using GitHub → Jenkins → Docker → Kubernetes → Grafana.
  • Each code push triggers a CI/CD pipeline that builds, tests, and deploys containers — reducing deployment time from hours to minutes.

6️⃣ What is DevSecOps?

  • DevSecOps integrates security practices into every stage of the DevOps pipeline.
  • It ensures that security is a shared responsibility across development, operations, and security teams.
  • Vulnerabilities are detected early in the build, deploy, or run phase — not after release.
  • Tools like SonarQube, Snyk, and Trivy automate vulnerability detection, while Vault protects secrets and Falco monitors runtime threats.

7️⃣ DevOps vs DevSecOps Workflow

DevOps: πŸ‘¨‍πŸ’» Developer → Git → Jenkins → Docker → Kubernetes → Grafana

DevSecOps: πŸ‘¨‍πŸ’» Developer → Git + SonarQube → Jenkins + Snyk → Docker + Trivy → Kubernetes + Vault → Falco → Grafana

8️⃣ Common DevSecOps Tools

  • SonarQube: Scans source code for vulnerabilities.
  • Snyk: Detects insecure dependencies during builds.
  • Trivy: Scans Docker images for known vulnerabilities.
  • Vault: Manages application secrets securely.
  • Falco: Detects abnormal runtime behavior in containers.

🧩 Real-World Example: DevSecOps

  • A Healthcare provider integrates security into its CI/CD pipeline using SonarQube, Snyk, Trivy, and Vault.
  • Falco monitors runtime threats — ensuring compliance and secure releases.

πŸ”Ÿ Key Differences Between DevOps & DevSecOps

Aspect DevOps DevSecOps
Focus Speed & automation Speed with security
Security Added later in the process Integrated from the start (shift-left)
Goal Deliver software quickly Deliver software quickly & securely
Tools Git, Jenkins, Docker, Kubernetes SonarQube, Snyk, Trivy, Vault, Falco
Responsibility Dev & Ops teams Dev, Ops & Security teams
Outcome Fast delivery Secure & compliant delivery

1️⃣1️⃣ Summary & Next Blog

  • DevOps = Automate everything for speed and collaboration.
  • DevSecOps = Add security automation to every stage.
  • πŸ’‘ Learn DevOps first, then extend it to DevSecOps for end-to-end security integration.