In this tutorial, we will install Ingress-Nginx controller on a self-managed Kubernetes cluster. We will also test the ingress by deploying a tiny web application on the cluster.

Ingress-Nginx controller is an open-source Kubernetes Ingress controller maintained by the Kubernetes project. This is different from the Nginx ingress controller which is a commercial Ingress Controller solution offered by Nginx.

Prerequisites

1. A self-managed Kubernetes cluster.
2. kubectl (Kubernetes CLI) installed and configured to access the Kubernetes cluster.

If you do not have a self-managed Kubernetes cluster, check out this tutorial to quickly deploy a Kubernetes cluster on your laptop.

Outline of steps

• Step-1: Install Ingress-Nginx controller
• Step-2: Install MetalLB load balancer
• Step-3: Configure MetalLB IP address pool
• Step-4: Advertise MetalLB IP address pool
• Step-5: Test the Nginx ingress
• Wrap up

Step-1: Install Ingress-Nginx controller

Log in to the workstation where kubectl is set up.

Install Ingress-nginx controller via Kubernetes manifest published on GitHub by Ingress-Nginx controller project:

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.1/deploy/static/provider/cloud/deploy.yaml

Check the status of Ingress-Nginx controller:

$ kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-hrdx2        0/1     Completed   0          5m
ingress-nginx-admission-patch-x297h         0/1     Completed   0          5m
ingress-nginx-controller-6d675964ff-qlh49   1/1     Running     0          5m

The ingress-nginx-admission-create and ingress-nginx-admission-patch are two Kubernetes Jobs for setting up the Ingress-Nginx controller and must be Completed.

The ingress controller Pod, ingress-nginx-controller will be in Running status.

It will take several minutes for Kubernetes to download and run the images. So, if your ingress-nginx-controller is not yet running allow several more minutes and check again.

Step-2: Install MetalLB load balancer

The Ingress-Nginx controller resides inside the Kubernetes cluster and needs a method to communicate with the clients outside the cluster.

There are three options to facilitate this:

1. Kubernetes port forwarding:

   Good for quick testing but not for production.

2. NodePort service:

   Expose ingress controller via a Kubernetes NodePort service. The clients must use a non-standard TCP port (30000 to 32767) to access applications inside the cluster.

3. Load balancer:

   Expose the ingress controller to outside clients via a load balancer.

We are going to use the Load balancer method with MetalLB which is an open-source load balancer for Kubernetes.

MetalLB is a CNCF sandbox project but is considered production-ready. MetalLB is included in several on-premise Kubernetes distributions.

Install MetalLB via the Kubernetes manifest provided by the MetalLB project:

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml

Step-3: Configure MetalLB IP address pool

For MetalLB to work as a load balancer, it needs an IP address pool.

Create Kubernetes manifest metallb-ip.yml to create an IPAddressPool:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default
  namespace: metallb-system
spec:
  addresses:
  - 10.200.0.10-10.200.0.20
  autoAssign: true

We have used the IP address range 10.200.0.10-10.200.0.20. You can use any IP address range that does not conflict with:

1. Kubernetes node IP addresses
2. External client IP addresses that access applications inside the cluster

To access the application inside the cluster directly from the Internet, you must use a public IP address range for the load balancer IP pool.

Create the IPAddressPool:

$ kubectl apply -f metallb-ip.yml

Step-4: Advertise MetalLB IP address pool

To advertise the IP addresses, MetalLB has two options:

1. Layer 2 mode:

   One node in the cluster takes the responsibility of advertising the load balancer IP address pool and advertises the IP addresses via ARP. This method is limited in scalability as all traffic must pass through this specific node in the cluster.

2. BGP mode:

   In the BGP mode, each node in the Kubernetes cluster establishes a BGP session with an external node, like a data center gateway router (aka DC Gateway). The BGP mode is more scalable than the Layer 2 mode but requires a router that supports BGP.

For this tutorial, we will use the Layer 2 mode.

Create the Kubernetes manifest metallb-config.yml:

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
  - default

Create the L2Advertisement:

$ kubectl apply -f metallb-config.yml

Step-5: Test the Nginx ingress

We will test the Nginx ingress by deploying qube-server which is a tiny web server that listens on port 8080.

Create Kubernetes manifest qube-server-deployment.yml:

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: qube-server
  name: qube-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: qube-server
  template:
    metadata:
      labels:
        app: qube-server
    spec:
      containers:
      - image: cloudqubes/qube-server:1.0.0
        name: qube-server
        ports:
        - containerPort: 8080

Create qube-server-deployment:

$ kubectl apply -f qube-server-deployment.yml

Check the deployment status:

$ kubectl get deployments -o=wide
NAME          READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS    IMAGES                         SELECTOR
qube-server   1/1     1            1           30s   qube-server   cloudqubes/qube-server:1.0.0   app=qube-server

Create the Kubernetes manifest qube-server-service.yml for Kubernetes Service:

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: qube-server
  name: qube-server
spec:
  ports:
  - name: qube-server-port
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: qube-server
  type: ClusterIP

Create qube-server-service:

$ kubectl apply -f qube-server-service.yml

Check the service status:

$ kubectl describe service qube-server
Name:              qube-server
Namespace:         default
Labels:            app=qube-server
Annotations:       <none>
Selector:          app=qube-server
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.96.132.60
IPs:               10.96.132.60
Port:              qube-server-port  8080/TCP
TargetPort:        8080/TCP
Endpoints:         192.168.119.234:8080
Session Affinity:  None
Events:            <none>

Note the Endpoints value is set to pod-ip-address:port. This association is created because we set the labels in the Service. If your labels in the Service, did not match the matchLabels in the Deployment, the Endpoints will not be correctly set.

Create Kubernetes manifest qube-server-ingress.yml for Kubernetes ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: null
  name: qube-server
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: qube-server
            port:
              number: 8080

Create the Kubernetes ingress:

$ kubectl apply -f qube-server-ingress.yml

Check the Kubernetes ingress:

$ kubectl get ingress
NAME     CLASS   HOSTS   ADDRESS       PORTS   AGE
qube-server   nginx   *       10.200.0.10   80      13s

The ingress is assigned with an IP address from the pool of load balancer IP addresses.

Kubernetes may take about one minute to assign this IP address to the ingress. If you don’t see an IP address assigned immediately after creating a new ingress object, wait a few minutes and check the status again.

Test the application with curl:

$ curl http://10.200.0.10
Hello cloud

Replace 10.200.0.10 with the IP address assigned to the ingress in your cluster.

We are now accessing the qube-server successfully via ingress.

Troubleshooting

Two common errors that you can encounter:

1. Ingress does not get an IP address

   Check that the IP address pool and the L2 advertising are properly configured.

2. You get an HTTP 5xx error response from the ingress controller

   This happens because the ingress cannot route HTTP requests to the qube-server. Check the TCP port numbers in the Deployment, Service, and Ingress are matching.

Wrap up

In this tutorial, we installed Ingress-Nginx controller on a self-managed Kubernetes cluster.

A self-managed Kubernetes cluster is a Kubernetes cluster installed from the open-source Kubernetes binaries. It’s a DIY installation as opposed to managed kubernetes from cloud providers like AWS, AKS, GKE, etc.

Ingress-Nginx controller is compatible with managed Kubernetes also. But the installation instructions are different from what we did here. Check out the docs for more details on how to deploy Ingress-Nginx controller on managed Kubernetes clusters.