Basically, there are two kinds of exposable applications: web-based applications, other applications. The difference between these types is in required IP address. The Web-based application shares IP address with other Web-based applications while the other applications require new IP address for each service. Number of IP addresses is limited so if possible, use the Web-based approach.
In the following documentation, there are YAML fragments, they are meant to be deployed using the
kubectl like this:
kubectl create -f file.yaml -n namespace where
file.yaml contains those YAMLs below and
namespace is the namespace with the application. Furthermore, this part of documentation suppose, that the application (deployment) already exists. If not sure, read again hello example.
Web-based applications are those that communicate via
HTTP protocol. These applications are exposed using
This kind of applications require a service that binds a port with application and ingress rules that expose the service to the Internet.
Suppose, we have an application that is ready to serve on port 8080. We create a service:
apiVersion: v1 kind: Service metadata: name: application-svc spec: type: ClusterIP ports: - name: application-port port: 80 targetPort: 8080 selector: app: application
app: application must match application name in deployment and
application-port are arbitrary names.
Once we have a service, we create ingress:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: application-ingress annotations: kubernetes.io/ingress.class: "nginx" kubernetes.io/tls-acme: "true" cert-manager.io/cluster-issuer: "letsencrypt-prod" spec: tls: - hosts: - "application.dyn.cloud.e-infra.cz" secretName: application-dyn-clout-e-infra-cz-tls rules: - host: "application.dyn.cloud.e-infra.cz" http: paths: - backend: service: name: application-svc port: number: 80 pathType: ImplementationSpecific
name: application-svc must match metadata name from the Service. This Ingress exposes the application on web URL
application.dyn.cloud.e-infra.cz. Note: domain name is automatically registered if it is in
dyn.cloud.e-infra.cz domain. The
tls section creates Lets Encrypt certificate, no need to maintain it.
tls section is used, TLS is terminated at system NGINX Ingress. Application must be set to communicate via
HTTP. Communication between Kubernetes and a user is via
TLS is terminated in NGINX Ingress at cluster boundary. Communication inside cluster is not encrypted, mainly not inside a single cluster node. If this is a problem, user needs to omit
tls section and
anotations section and provide a certificate and key on his/her own to the Pod.
Some applications are confused that they are configured as
HTTP but exposed as
HTTPS. If an application generates absolute URLs, it must generate
HTTPS URLs and not
HTTP. Kubernetes Ingress sets
HTTP_X_SCHEME header to
HTTPS if it is TLS terminated. E.g., for django, user must set:
SECURE_PROXY_SSL_HEADER = ("HTTP_X_SCHEME", "https"). Common header used for this situation is
X_FORWARDED_PROTO but this is not set by Kubernetes NGINX. It is possible to expose also ‘HTTPS’ configured applications. See below HTTPS Target.
Custom domain name (FQDN)
It is possible to use also custom domain name (any name basically).
CNAMEDNS record is required. For
- Ingress object is in the form:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: application-ingress annotations: kubernetes.io/ingress.class: "nginx" kubernetes.io/tls-acme: "true" cert-manager.io/cluster-issuer: "letsencrypt-prod" spec: tls: - hosts: - "my-name" secretName: my-name-tls rules: - host: "my-name" http: paths: - backend: service: name: application-svc port: number: 80 pathType: ImplementationSpecific
CNAME must be set in advance and must be propagated so that Let’s Encrypt service is able to verify domain in case, TLS is requested. It means that seamless migration from any system to our infrastructure is not easily possible.
Ingress can request user authentication and thus provide restricted access to the
Service. Authentication can be set using a
Secret and annotations.
First, setup a secret:
apiVersion: v1 kind: Secret metadata: name: secretref type: Opaque data: auth: password
secretref is arbitrary name and password is
htpasswd. Password can be generated by this command:
htpasswd -n username | base64 -w0 where
username is desired login username. It outputs text like
Zm9vOiRhcHIxJGhOZVRzdngvJEkyVk9NbEhHZDE1N1gvYTN2aktYSDEKCg== which shall be put instead of the
password into the secret above.
The following two annotations are required in the
nginx.ingress.kubernetes.io/auth-type: basic nginx.ingress.kubernetes.io/auth-secret: secretref
secretref must match tne metadata name of the
The password protection is applied only on external traffic, i.e., user will be prompted for a password. However, traffic from other pods bypasses authentication if it communicates directly with the
Service IP. This can be mitigated applying
Network Policy. See Security.
Big Data Upload
If big data upload is expected, the following two
Ingress annotations might be necessary to deal with upload size limit.
nginx.ingress.kubernetes.io/proxy-body-size: "600m" nginx.org/client-max-body-size: "600m"
600m value with desired max value for upload data.
Ingress object can expose applications use HTTPS procol instead of HTTP, i.e., communication between ingress and application is encrypted as well. In this case, you need to add
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" annotation to the Ingress object.
Applications that to not use
HTTP protocol are exposed directly via
apiVersion: v1 kind: Service metadata: name: application-svc annotations: external-dns.alpha.kubernetes.io/hostname: application.dyn.cloud.e-infra.cz spec: type: LoadBalancer ports: - port: 22 targetPort: 2222 selector: app: application
app: application must match application name in deployment and
application-svc is arbitrary name. If annotation
external-dns.alpha.kubernetes.io/hostname is provided and value i in domain
dyn.cloud.e-infra.cz, the name is registered in DNS.
Deploying this Service exposes the application on a public IP. One can check the IP with
kubectl get svc -n namespace where
namespace is namespace of the Service and application.
It is also possible to expose application on MUNI private IP. In this case, the application will be reachable from MUNI network or using MUNI VPN. This type of exposing is selected using annotation
metallb.universe.tf/address-pool: privmuni. This case is preferred as it does not consume public IP. Full example is below.
apiVersion: v1 kind: Service metadata: name: application-svc annotations: metallb.universe.tf/address-pool: privmuni external-dns.alpha.kubernetes.io/hostname: application.dyn.cloud.e-infra.cz spec: type: LoadBalancer ports: - port: 22 targetPort: 2222 selector: app: application