Module 20: Containers & Orchestration Basics

1. Module Title & Overview

• Title: Module 20: Containers & Orchestration Basics
• Overview: This module teaches the fundamentals of application isolation, containerization, and cluster-level orchestration. Students will master container image compilation, analyze Docker host boundaries, and design Kubernetes topology layouts featuring service routing, scaling configurations, and deployment strategies.

2. Learning Objectives

• Construct Optimized Container Images: Build minimal, secure container files using multi-stage compilation patterns and secure user permissions.
• Formulate Kubernetes Routing Topologies: Design pod-to-service and ingress-to-pod networking pathways utilizing K8s Services, DNS resolution, and Ingress rules.
• Establish Application Health Lifecycles: Configure Startup, Liveness, and Readiness probes to enable self-healing and zero-downtime rolling updates.
• Implement Autoscaling and Resource Management: Write resource request/limit specifications for cluster workloads and configure Horizontal Pod Autoscaling (HPA) policies.

3. Prerequisite Statement

Requires Module 4 (Network Primitives & Traffic Management) and Module 18 (Cloud Architecture Fundamentals). Students must understand load balancer layer distributions, IP subnets, and cloud compute resource scaling rules before building container scheduling rules.

4. Content Outline

Section 20.1: Containerization Mechanics & Runtime Isolation

• Concepts: Linux Namespaces (Process, Network, Mount), Control Groups (cgroups), Container Runtimes (containerd, CRI-O), and Union File Systems.
• Deep Dive: Under-the-hood difference between VM hypervisors (hardware emulation) and container engine virtualization (OS kernel sharing). Root file system isolation boundaries, process signals (SIGTERM handling), and Docker daemon structures.
• Architectural Trade-offs: Compiling thin, lightweight container images using distroless/Alpine baselines increases download speed and minimizes attack surface but limits debug tool availability inside active containers.
• Physical Constraints: Shared host OS kernel version compatibility, local namespace context isolation, and disk storage throughput on image layering read-writes.

Section 20.2: Kubernetes Cluster Architecture & Control Plane Mechanics

• Concepts: Control Plane nodes, worker nodes, API Server, etcd, Controller Manager, Kube-Scheduler, Kubelet, and Kube-Proxy.
• Deep Dive: Declarative State loops. The lifecycle of a Kubernetes resource: from client manifest submission via kubectl to etcd persistence, scheduler filtering, and kubelet state creation.
• Architectural Trade-offs: Maintaining a single, centralized database state in etcd simplifies consistency but creates a critical checkpoint that bottleneck-limits cluster write scale capacity.
• Physical Constraints: etcd database write latency (requires fast SSD storage), network bandwidth limits on control plane communication, and node scheduling limits.

Section 20.3: Pod Topologies & Networking Pathways

• Concepts: Pods, ReplicaSets, Deployments, ClusterIP, NodePort, LoadBalancer, Ingress Controllers, CoreDNS, and CNI (Container Network Interface).
• Deep Dive: Pod-to-Pod network communication models (virtual network bridges and overlay networks). Port mapping mechanics, DNS record updates inside the cluster, and packet routing through iptables rules.
• Architectural Trade-offs: NodePort service routes expose direct external IP entry lines to workloads but restrict available host port ranges (30000–32767) and present load imbalance vulnerabilities. Ingress Controllers route traffic through a unified gateway, reducing IP costs but adding routing hops.
• Physical Constraints: CNI IP address allocation limits, port reuse limits, and routing lookup latency on large scale-out clusters.

Section 20.4: Pod Lifecycles & Self-Healing Probes

• Concepts: Pod phases (Pending, Running, Succeeded, Failed), Startup Probes, Liveness Probes, Readiness Probes, Graceful Shutdown periods, and PreStop hooks.
• Deep Dive: Designing robust application startup verification paths. The mechanical difference between Liveness (Should we kill this pod?) and Readiness (Should we send traffic to this pod?).
• Architectural Trade-offs: Aggressive health probe checks (short timeouts/high frequencies) isolate deadlocks fast but trigger cascading restart loops during temporary database network hiccups. Loose probe policies delay error recovery.
• Physical Constraints: Application startup times, probe execution CPU usage, and network timeout limits.

Section 20.5: Resource Allocation & Horizontal Pod Autoscaling (HPA)

• Concepts: Resource Requests, Resource Limits, CPU Throttling, OOMKilled (Out of Memory Killer), HPA, and Metrics Server.
• Deep Dive: How Linux cgroups enforce memory limits (hard limits leading to OOM termination) vs. CPU limits (soft limits leading to thread throttling). Configuring HPA using CPU/Memory metrics and custom Prometheus queries.
• Architectural Trade-offs: Under-provisioning CPU limits avoids cluster cost waste but leads to application request latency spikes during spikes due to CPU throttling. Over-provisioning increases costs.
• Physical Constraints: Host VM memory capacities, host VM CPU cores, and autoscaler scale-out rate limits.

Section 20.6: Orchestrated Deployment Patterns

• Concepts: Recreate, Rolling Updates (maxSurge, maxUnavailable), Canary Deployments, and Blue-Green swaps.
• Deep Dive: Executing zero-downtime rolling updates. Service routing changes during rollout states, database schema compatibility rules (backward-compatible writes), and rollback mechanics.
• Architectural Trade-offs: Rolling updates dynamically balance cluster workloads during deployment but require database schemas to support concurrent old/new code versions. Recreate deployments avoid schema issues but introduce downtime.
• Physical Constraints: Image download latency during deployment steps, network routing update propagation delays, and warm-up latency of new containers.

5. Key Concepts

• Namespace: A Linux kernel feature isolating system resources (processes, network interfaces, mount points) from other processes on the same host.
• Cgroup (Control Group): A Linux kernel feature limiting, accounting for, and isolating the resource usage (CPU, memory, disk I/O, network) of a collection of processes.
• Pod: The smallest deployable unit of computing that can be created and managed in Kubernetes, containing one or more containers sharing storage and network.
• ReplicaSet: A Kubernetes controller that maintains a stable set of replica Pods running at any given time.
• ClusterIP: The default Kubernetes Service type, exposing the service on a cluster-internal IP address, accessible only within the cluster.
• Ingress Controller: A specialized load balancer application that manages external access to HTTP/HTTPS services in a Kubernetes cluster.
• Liveness Probe: A check determining whether a container needs to be restarted because it has entered a deadlocked state.
• Readiness Probe: A check determining whether a container is ready to accept network traffic. If it fails, the container is removed from the service endpoint list.
• OOMKilled: A container termination state triggered when a container exceeds its allocated cgroup memory limit.
• Rolling Update: A deployment strategy replacing instances of the old version of an application with instances of the new version without downtime.
• CNI (Container Network Interface): The specification and libraries that define how network plugins configure networking for containers.

6. Practice Section Description

• Practice Exercise: Designing a Scalable Kubernetes Topology for a E-Commerce System.
• Scenario: A store database-backed frontend service experiences massive load spikes during seasonal sales. It needs to scale from 3 replicas to 50 replicas in response to CPU load, but must avoid overloading the backend database.
• Challenge: Students must design the deployment configuration in a K8s network diagram (via the diagram editor). They must layout the Ingress layer, routing services, pod replicas, resource limits, and health probes.
• Constraints: Must use Readiness Probes to protect the system from routing traffic to uninitialized app containers. Must configure CPU request limits to prevent OOM termination of the database pods. Must configure rolling update params (maxSurge and maxUnavailable) to guarantee zero-downtime rollouts.

flowchart TD
    subgraph Internet
        User[Client Browser]
    end

    subgraph Kubernetes Cluster Boundary
        Ingress[Ingress Controller]
        
        subgraph Frontend Service Tier [Namespace: App]
            SVC_Front[Service: frontend-svc - ClusterIP]
            Pod_F1[Pod: frontend-deployment - Replica 1]
            Pod_F2[Pod: frontend-deployment - Replica 2]
            HPA[Horizontal Pod Autoscaler]
        end

        subgraph Database Tier [Namespace: Data]
            SVC_DB[Service: db-svc - ClusterIP]
            Pod_DB[Pod: postgres-db-0 - StatefulSet]
        end
    end

    %% Routing Flow
    User -->|HTTPS| Ingress
    Ingress -->|Route to Port 80| SVC_Front
    SVC_Front -->|Load Balances| Pod_F1
    SVC_Front -->|Load Balances| Pod_F2

    Pod_F1 -->|Port 5432| SVC_DB
    Pod_F2 -->|Port 5432| SVC_DB
    SVC_DB --> Pod_DB

    %% Management Connections
    HPA -.->|Scrapes CPU Metrics| Pod_F1
    HPA -.->|Scrapes CPU Metrics| Pod_F2
    HPA -.->|Adjusts Replicas| SVC_Front

7. Deliverable/Documentation

• Deliverable Name: Deployment Architecture & Kubernetes Topology Spec
• Description: A detailed deployment mapping document containing:
  1. A C4 Level 2 Container Diagram showing the mapping to Kubernetes workloads (Deployments, Services, ConfigMaps).
  2. A comprehensive YAML manifest suite defining the application deployment, service ports, resource limits (Requests/Limits), HPA policies, and health probe metrics.
  3. A mitigation analysis matrix detailing the cluster response to three failure cases: pod process crash, node VM out of memory, and API server communication failure.

Code Snippet: Kubernetes Deployment Manifest for a Resilient C# Service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mpc-order-processor
  namespace: core-apps
  labels:
    app: order-processor
    tier: application
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0
  selector:
    matchLabels:
      app: order-processor
  template:
    metadata:
      labels:
        app: order-processor
    spec:
      containers:
      - name: processor
        image: mpc-registry.internal/order-processor:v2.4.1
        ports:
        - containerPort: 8080
          name: http
        resources:
          requests:
            memory: "256Mi"
            cpu: "200m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        startupProbe:
          httpGet:
            path: /health/startup
            port: 8080
          failureThreshold: 30
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 15
          timeoutSeconds: 2
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
          timeoutSeconds: 2

8. Integration Notes

• Curriculum Placement: Extends Module 4 (Network Primitives & Traffic Management) and Module 18 (Cloud Architecture Fundamentals). Bridges the gap between bare networks and containerized scheduling infrastructure.
• hiring_signal: "Understands how application code interacts with the underlying operating system container namespace boundaries and can write Kubernetes resource limits, scaling configurations, and deployment strategies that prevent cluster outages."