How to Run NGINX Web Server

Trying to run NGINX web server is common test to deploy application/service into Kubernetes. It seems to be an easy task as NGINX is simple to run and moreover, there is docker image already prepared. However, deployment can fail for several reasons, therefore, we provide some notes, how to run at at first try.

Privileged vs. Unprivileged

This Kubernetes platform is multitenant, i.e., it is shared by many users and therefore, it has some restrictions. One of them is necessity to set security context because privilege escalation is not allowed. Setting the security context to unprivileged mode sometimes requires creating own docker image for the application. Luckily, for NGINX, there is already prepared unprivileged image. Its reference id is: nginxinc/nginx-unprivileged. This image is fully compatible with our platform as is, no changes into image are needed.

Running User ID

The above mentioned security context requires a correct user id and group id to be set (those RunAsUser, RunAsGroup fields). For the nginx-unprivileged image, the correct value for both of them is 101. This value can be simply obtained using docker as follows. See the uid=101 and gid=101:

docker run -it --rm --entrypoint /bin/sh nginxinc/nginx-unprivileged
$ id
uid=101(nginx) gid=101(nginx) groups=101(nginx)
$

This example assumes that the image contains /bin/sh and id binaries. Mainly containerized Go Lang applications usualy contain only the Go application and nothing else. In such a case, Dockerfile used for image build or project documentation must be consulted.

Deployment Manifest

The following manifest can be used to run the NGINX instance. The name nginx can be replaced with any other valid string (small letters, numbers, dash). Also resources part should be set accordingly to expected usage (memory and CPU).

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      securityContext:
        fsGroupChangePolicy: OnRootMismatch
        fsGroup: 101
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: nginx
        image: nginxinc/nginx-unprivileged
        imagePullPolicy: IfNotPresent
        securityContext:
          runAsUser: 101
          runAsGroup: 101
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
        resources:
          requests:
            cpu: 500m
            memory: 2Gi
          limits:
            cpu: 2
            memory: 8Gi

Adding Persistent Storage

The deployment above will work, however, no useful content will be served. There is just hello page. Also, the image does not expect that any content is added later by copying data into web root (/usr/share/nginx/html), it is not writable by default.

Optimal way to change served content is to add PVC volume, see more about PVC.

We can create ad-hoc PVC using the following manifest:

pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-nginx
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 2Gi
  storageClassName: nfs-csi

Data on this PVC are preserved until the PVC is deleted.

If the PVC has been created, we need to change the deployment so that PVC is used:

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      securityContext:
        fsGroupChangePolicy: OnRootMismatch
        fsGroup: 101
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: nginx
        image: nginxinc/nginx-unprivileged
        imagePullPolicy: IfNotPresent
        securityContext:
          runAsUser: 101
          runAsGroup: 101
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
        resources:
          requests:
            cpu: 500m
            memory: 2Gi
          limits:
            cpu: 2
            memory: 8Gi
        volumeMounts:
        - name: vol-1
          mountPath: /usr/share/nginx/html
      volumes:
      - name: vol-1
        persistentVolumeClaim:
          claimName: test-nginx

The PVC name (name: test-nginx) must match the claimName (claimName: test-nginx).

Directory index is not enabled by default for the NGINX. Therefore, if there is no index.html file uploaded to the PVC, the NGINX will return error 403 Forbidden if the URL explained below will be opened in browser.

Exposing the NGINX

To make the running NGINX accessible from internet, see Exposing applications section. Basically, two additional simple manifests are needed:

Service

The service manifest route traffic into the running Pod. It needs correct target port and application label. In our case, we chose app: nginx as application label (see metadata labels section of the deployment manifest), and from nginx-unprivileged documentation we can see, that the NGINX is using 8080 port, so the Service manifest looks as follows:

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
    - port: 8080
      targetPort: 8080
      protocol: TCP
  selector:
    app: nginx

Ingress

The last required manifest is the Ingress manifest. It contains the hostname of the application and it can also provide the Let’s Encrypt certificate.

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
   - hosts:
       - "[some-name].dyn.cloud.e-infra.cz"
     secretName: [some-name]-dyn-cloud-e-infra-cz
  rules:
  - host: [some-name].dyn.cloud.e-infra.cz
    http:
      paths:
        - pathType: Prefix
          path: "/"
          backend:
            service:
              name: nginx
              port:
                number: 8080

Replace the [some-name] with some real name. Do not use generic names such as test, example, and so on, they can be already taken.

The service name and port number must match values from the Service object.