Skip to main content

Kubernetes Infrastructure FDD

Overview​

This FDD describes the Kubernetes infrastructure setup for running Arcana services in production. The infrastructure provides container orchestration, networking, ingress control, and SSL/TLS certificate management.

Status​

  • Basic Kubernetes cluster setup
  • Flannel CNI networking
  • Nginx ingress controller
  • Let's Encrypt SSL certificates
  • Monitoring and logging NEXT_TASK: Implement Kubernetes monitoring and logging stack with Prometheus, Grafana, and ELK
  • Automatic certificate renewal
  • Horizontal pod autoscaling
  • Backup and disaster recovery

Components​

Core Infrastructure​

  • Single-node Kubernetes cluster on Hetzner
  • Flannel CNI for pod networking
  • Nginx ingress controller for traffic routing
  • Let's Encrypt certificates for HTTPS

Networking​

# Flannel Configuration
networking:
  podSubnet: 10.244.0.0/16
  mtu: 1450
  ipMasq: true

Ingress​

# Nginx Ingress Configuration
ingress:
  controller:
    kind: DaemonSet
    hostNetwork: true
    ports:
      http: 80
      https: 443
    config:
      disable-default-backend: 'true'
      ssl-redirect: 'true'

SSL/TLS​

# Certificate Configuration
certificates:
  provider: letsEncrypt
  domains:
    - mercury.arcana.cfd
  secretName: mercury-tls
  renewBefore: 720h # 30 days

Technical Implementation​

Pod Networking​

  1. Flannel CNI plugin installed and configured
  2. Required kernel modules loaded:
    • br_netfilter
    • overlay
  3. Network policies to be implemented for security

Ingress Controller​

  1. Nginx ingress controller deployed as DaemonSet
  2. Host networking enabled for direct port binding
  3. Default backend disabled
  4. SSL redirect enforced

Certificate Management​

  1. Let's Encrypt certificates obtained via certbot
  2. Certificates stored as Kubernetes secrets
  3. TLS termination at ingress level
  4. Manual renewal process (automation planned)

Configuration​

Required Environment Variables​

# Node Configuration
KUBERNETES_VERSION=1.28
NODE_IP=95.217.130.127
POD_NETWORK_CIDR=10.244.0.0/16

# Ingress Configuration
INGRESS_CLASS=nginx
INGRESS_CONTROLLER_PORT_HTTP=80
INGRESS_CONTROLLER_PORT_HTTPS=443

# Certificate Configuration
CERT_DOMAIN=mercury.arcana.cfd
[email protected]

Network Requirements​

  • Port 80/443 open for HTTP/HTTPS
  • Port 6443 for Kubernetes API
  • Pod network CIDR: 10.244.0.0/16
  • Service CIDR: 10.96.0.0/12

Development Guidelines​

Local Development​

  1. Use minikube or kind for local testing
  2. Match production Kubernetes version
  3. Use ingress-nginx addon in minikube

Deployment Process​

  1. Update kernel modules if needed
  2. Apply Flannel CNI configuration
  3. Deploy ingress controller
  4. Configure SSL certificates
  5. Deploy application services

Troubleshooting​

  1. Check pod networking:

    kubectl get pods -A
    kubectl describe pod <pod-name>

  2. Verify ingress:

    kubectl get ingress -A
    kubectl describe ingress <ingress-name>

  3. Certificate issues:

    kubectl get secrets
    kubectl describe secret <tls-secret>

Security Considerations​

  1. Network Policies

    • Implement default deny
    • Allow only required traffic
    • Separate critical services

  2. Certificate Management

    • Regular rotation
    • Secure storage of secrets
    • Automated renewal (planned)

  3. Access Control

    • RBAC for service accounts
    • Minimal privileges principle
    • Regular audit of permissions

Monitoring and Maintenance​

Health Checks​

  • Node status
  • Pod status
  • Service endpoints
  • Certificate expiration

Backup Strategy​

  • etcd snapshots
  • Certificate backups
  • Configuration backups

Upgrade Process​

  1. Backup current state
  2. Update node components
  3. Update CNI plugins
  4. Update ingress controller
  5. Verify functionality

Future Improvements​

  1. High Availability

    • Multi-node setup
    • Load balancing
    • Failover configuration

  2. Automation

    • Certificate renewal
    • Backup process
    • Health monitoring

  3. Security

    • Network policy implementation
    • Regular security audits
    • Vulnerability scanning

References​

  1. Kubernetes Documentation
  2. Flannel CNI
  3. Nginx Ingress Controller
  4. Let's Encrypt