Kubernetes TLS-Offloading with istio

I spent some time to find out how to enable TLS for all of my Kubernetes services, which are available on my website. Many of the pods reference reverse-proxy functionality for TLS termination. This would require additional pods running Nginx (reverse proxy) having a maintained TLS certificate for the same domain (*.fas-consulting.de). Next to inbound traffic, I focus on TLS communication between pods only. Therefore I checked for a service mash such as mist.io and istio.io. Currently my services include the following pods:

• Product Management
• Bastion Host
• Patch Management
• Source Code Repository
• Automation Server
• Code Quality
• etc.

You can find those modules on top of my website for demonstration purposes (see first row):

Solution: Istio Service Mash

Istio offers service mash capabilities. Ingress traffic consists of multiple components:

• Edge Proxy
  Part of the istio namespace. Provides a Node-Port to access the Kubernetes-Cluster.
• Gateway
  Normally configured to be part of the isito namespace. Provides TLS termination and central networking capabilities.
• Virtual Service
  Contains concrete matches and routings within application context. This module can be in separate namespaces:
  • istio-system namespace (InfraOps)
    centrally managed as part of the istio namespace. Not recommended by istio development teams.
  • separate routing namespace (SecOps)
    centrally managed as part of an own namespace. Recommended to separate between InfraOps, SecOps and DevOps.
  • application namespace (DevSecOps)
    application managed as part of the DevSecOps team. Recommended for smaller organizations, which do not provide dedicated SecOps (due to low workload).

The Edge-Gateway is  part of the istio-system namespace and correlates through a node port to the network:

$ kubectl edit svc istio-ingressgateway -n istio-system
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"istio-ingressgateway","chart":"gateways","heritage":"Tiller","istio":"ingressgateway","release":"istio"},"name":"istio-ingressgateway","namespace":"istio-system"},"spec":{"ports":[{"name":"status-port","port":15020,"targetPort":15020},{"name":"http2","nodePort":31380,"port":80,"targetPort":80},{"name":"https","nodePort":31390,"port":443},{"name":"tcp","nodePort":31400,"port":31400},{"name":"https-kiali","port":15029,"targetPort":15029},{"name":"https-prometheus","port":15030,"targetPort":15030},{"name":"https-grafana","port":15031,"targetPort":15031},{"name":"https-tracing","port":15032,"targetPort":15032},{"name":"tls","port":15443,"targetPort":15443}],"selector":{"app":"istio-ingressgateway","istio":"ingressgateway","release":"istio"},"type":"LoadBalancer"}}
  creationTimestamp: "2019-10-22T14:05:30Z"
  labels:
    app: istio-ingressgateway
    chart: gateways
    heritage: Tiller
    istio: ingressgateway
    release: istio
  name: istio-ingressgateway
  namespace: istio-system
  resourceVersion: "660121"
  selfLink: /api/v1/namespaces/istio-system/services/istio-ingressgateway
  uid: 591234e7-eef7-44f8-8d0f-c7b8af0a274a
  spec:
  clusterIP: 10.105.42.214
  externalTrafficPolicy: Cluster
  ports:
  - name: status-port
    nodePort: 31245
    port: 15020
    protocol: TCP
    targetPort: 15020
  - name: http2
    nodePort: 31380
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    nodePort: 31390
    port: 443
    protocol: TCP
    targetPort: 443
  - name: tcp
    nodePort: 31400
    port: 31400
    protocol: TCP
    targetPort: 31400
  - name: https-kiali
    nodePort: 31851
    port: 15029
    protocol: TCP
    targetPort: 15029
  - name: https-prometheus
    nodePort: 32279
    port: 15030
    protocol: TCP
    targetPort: 15030
  - name: https-grafana
    nodePort: 32650
    port: 15031
    protocol: TCP
    targetPort: 15031
  - name: https-tracing
    nodePort: 30791
    port: 15032
  - name: tls
    nodePort: 31931
    port: 15443
    protocol: TCP
    targetPort: 15443
  - name: https-applications  # https-Prefix for TLS traffic (istio 1.3.3)
    nodePort: 20000           # Node-Port of Edge Proxy
    port: 15000               # Port of the Gateway
    protocol: TCP
    targetPort: 15000         # Port of the Gateway
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
    release: istio
  sessionAffinity: None
  type: LoadBalancer
  status:
  loadBalancer: {}

The Edge-Gateway forwards the traffic to a dedicated gateway, which is recommended to be allocated within the istio-system namespace. This is due to the current (istio 1.3.3) limitation that only one application gateway is allowed.

$ vim application-gateway.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: application-gw
  namespace: istio-system
  spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 15000
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"
    

For proper functionality, don’t forget to create the related Istio secret while keeping all files at /etc/istio/ingressgateway-certs/:

kubectl create -n istio-system secret tls istio-ingressgateway-certs --key tls.key --cert tls.crt

As part of the configuration, Virtual Services references the related Gateway and attach themselves to it. The application or SecOps team forwards the traffic to the applications service:

$ vim jenkins-virtualservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: jenkins-vs
  namespace: jenkins
  spec:
  hosts:
  - jenkins.fas-consulting.de
  gateways:
    - istio-system/application-gw
  http:
  - match:
    route:
    - destination:
        port:
          number: 8080
        host: jenkins-svc.jenkins.svc.cluster.local