Accessing a service in a different namespace from a single ingress in Kubernetes

IPv4. Its rare when its public, and annoying when its private. So we try and conserve this precious resource. One of the things that makes it complex is Kubernetes namespaces. A Kubernetes Ingress controller is not namespace aware (you can’t have a shared Ingress that has services in multiple namespaces). Or can you?

What if I told you you could install a single Ingress (and cert-manager etc) and then have a service in each namespace served by it? Would you rejoice over saving a few $100/mo on public IP rental in ‘the cloud’?

Lets dig in. Imagine we have 3 namespaces with interesting services. ‘foo’, ‘bar’ and ‘kube-system’ (which has our dashboard).

Lets assume we have ‘kibana’ running in kube-system. We want to expose this to the ‘public internet’. Likely we would also use oauth2 proxy here to sign in, but I’ll ignore auth for now.  We are going to use a new service (synthetic) which lives in the default namespace alongside our Ingress controller as ‘glue’. Its kind of like a DNS CNAME.

First we install a single global ingress. Lets use helm:

helm install stable/nginx-ingress --name ingress --set controller.service.externalTrafficPolicy=Local --set rbac.create=true

Wait for the LoadBalancer to get a public IP, register it in DNS. You can either use a wildcard (*.something.MYDOMAIN.CA) or register each service, your call. All will use the same IP.

(To avoid complicating this, I’ll show the cert-manager etc at the end, but its optional, we just need the next step with the Ingress + Service)

Once we have installed the below yaml we can now browse https://kibana.MYDOMAIN.CA/ and we are there. Repeat for the other services. Done! We have a single public IP

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kibana
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: 'true'
    nginx.ingress.kubernetes.io/tls-acme: 'true'
spec:
  tls:
  - hosts:
    - kibana.MYDOMAIN.CA
    secretName: tls-secret-kibana
  rules:
  - host: kibana.MYDOMAIN.CA
    http:
      paths:
      - path: /
        backend:
          serviceName: kibana
          servicePort: 5601
---
kind: Service
apiVersion: v1
metadata:
  name: kibana
  namespace: default
spec:
  type: ExternalName
  externalName: kibana.kube-system.svc.cluster.local
  ports:
  - port: 5601

Now, to complete this and show with SSL certificates. You don’t need this, above is all you need to expose the service, but why not do it on TLS at the same time? Its free!.

helm install stable/cert-manager --namespace kube-system --set ingressShim.defaultIssuerName=letsencrypt --set ingressShim.defaultIssuerKind=ClusterIssuer --name cert

cat << EOF | kubectl -n kube-system apply -f -
apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: don@agilicus.com
    privateKeySecretRef:
      name: letsencrypt
    http01: {}
EOF