In this post you’ll learn how to write Kubernetes Admission webhooks using OpenFaaS functions
Introduction to Kubernetes Admission webhooks
Admission webhooks are HTTP callbacks that receive admission requests and do something with them. You can define two types of admission webhooks, validating admission webhook and mutating admission webhook. Mutating admission webhooks are invoked first, and can modify objects sent to the API server to enforce custom defaults. After all object modifications are complete, and after the incoming object is validate by the API server, validating admission webhooks are invoked and can reject requests to enforce custom policies.
Using OpenFaaS in this design, we can focus on our core logic more than designing the microservice itself and simply create application without being worry about how to build and deploy.
The Scenario
Let’s assume, in our company, we have some requirements that we must meet while deploying applications onto the Kubernetes cluster. We need to set some required labels to our Kubernetes manifest. Unless we specify the required labels our request will reject.
So, in order to apply those requirements to the Kubernetes cluster to ensure the best practices, we can use Kubernetes ValidatingAdmissionWebhook and OpenFaaS together. Since ValidatingAdmissionWebhooks intercepts requests to the apiserver, OpenFaaS functions includes a little code to check required labels and determines the request either allowed or not.
Webhook Admission Server is just plain http server that adhere to Kubernetes API. For each Pod create request to the apiserver(I said Pod because we specify which kind of resources that we consider while registering our webhook to the apiserver using ValidatingWebhookConfiguration resource) the ValidatingAdmissionWebhook sends an admissionReview(API for reference) to the relevant webhook admission server. The webhook admission server gathers information like object, oldobject, and userInfo from admissionReview’s AdmissionRequest and sends AdmissionRequest to the serverless function through the OpenFaaS Gateway. The function checks the required labels exist on Pod and determines the request either valid or not and then sends back the AdmissionResponse whose Allowed and Result fields are filled with the admission decision to the webhook admission server then the webhook admission servers sends back a admissionReview to the apiserver.
- Kubernetes API -> Webhook (w/TLS) -> OpenFaaS Gateway (w/HTTP) -> OpenFaaS Function
Credit: https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/
Supporting TLS for external webhook server is also required because admission is a high security operation. As part of the process, we need to create a TLS certificate signed by the Kubernetes CA to secure the communication between the webhook server and apiserver.
Prerequisites
Arkade
- arkade is The OpenFaaS community built tool for Kubernetes developers, with arkade you can easily install all necessary cli tools to your host and deploy apps to the cluster.
$ curl -sLS https://get.arkade.dev | sudo sh
KinD (Kubernetes in Docker)
- Kubernetes is our recommendation for teams running at scale, but in this demo we will be using KinD for the sake of simplicity.
$ arkade get kind
kubectl
- You can control your cluster using kubectl CLI.
$ arkade get kubectl
faas-cli
- faas-cli is an official CLI for OpenFaaS , with “faas-cli” you can build and deploy functions easily.
$ arkade get faas-cli
Setup
1. Setup a Kubernetes Cluster with KinD
You can start a Kubernetes cluster with KinD if you don’t have one already
$ arkade get kind
$ kind create cluster
2. Deploy OpenFaaS to our local Kubernetes Cluster with arkade:
- Install a OpenFaaS
$ arkade install openfaas
Read the output from the installation and run the commands given to you.
You can access them again at any time with arkade info openfaas
3. Clone the project
- Clone the sample from GitHub
$ git clone https://github.com/developer-guy/admission-webhook-example-with-openfaas
$ cd admission-webhook-example-with-openfaas
- Let’s explore the structure of the project.
deployment/ --> includes necessary manifests and scripts for the deployment of the project
functions/ --> includes templates and the requiredlabel function itself
Dockerfile --> includes instructions to build an image of the project
build --> automated way to build and push an image of the project
4. Deploy ValidatingAdmissionWebhook
- We will generate the TLS certificates required for the ValidatingAdmissionWebhook using the following: Kubernetes TLS Certificates Management
$ cd deployment
$ sh webhook-create-signed-cert.sh
- Get the Certificate Authority (CA) from the local cluster
$ export CA_BUNDLE=$(kubectl config view --minify --flatten -o json | jq -r '.clusters[] | select(.name == "'$(kubectl config current-context)'") | .cluster."certificate-authority-data"')
$ sed -e "s|\${CA_BUNDLE}|${CA_BUNDLE}|g" validatingwebhook.yaml | kubectl apply -f -
$ cd ..
- Build the project
$ export DOCKER_USER="docker-hub-username"
$ ./build
Now edit deployment.yaml
and set ‘DOCKER_USER’ to the above.
- Deploy it project
$ cd deployment
$ kubectl apply -f rbac.yaml,service.yaml,deployment.yaml
5. Build and Deploy OpenFaaS Function (Optional)
- Pull the golang-middleware template from OpenFaaS Official Template Store
$ faas-cli template store list # check available templates in store
$ faas-cli template store describe golang-middleware # describe the specific template
$ faas-cli template store pull golang-middleware
- Create the function
$ export OPENFAAS_PREFIX=$DOCKER_USER
$ faas-cli new requiredlabel --lang go-middleware
$ cd requiredlabel
$ go mod init requiredlabel
$ # fill the handler.go with the corresponding code
$ go get
- Deploy the function
$ cd functions
$ faas-cli up -f requiredlabel.yml --build-arg GO111MODULE=on # (build-push-deploy) make sure you are using your docker hub username. i.e: devopps
- Verify the functions that are working in
openfaas-fn
namespace
$ kubectl get pods --namespace openfaas-fn
6. Test the whole workflow
- The purpose of this PoC is that to validate that pods has required
labels
. Which means you must have that labels:
app.kubernetes.io/name: sleep
app.kubernetes.io/instance: sleep
app.kubernetes.io/version: "0.1"
app.kubernetes.io/component: dummy
app.kubernetes.io/part-of: admission-webhook-example
app.kubernetes.io/managed-by: kubernetes
- Any Pod who have above labels is valid for us.
`deployment/sleep.yaml` -> Incorrect, not-valid (We should deny this creation request.)
`deployment/sleep-no-validation.yaml` -> Skip-validation (Based on `admission-webhook-example.qikqiak.com/validate: "false"` annotation, we skipped validation.)
`deployment/sleep-with-labels.yaml` -> Correct, valid (We should accept this creation request.)
7. A way of extend the operation event and function
In this demo, we only consider the Pod create request by specifying operations at the ValidatingWebhookConfiguration’s matching request rules section in the deployment/validatingwebhook.yaml file.
If we want to extend the operations, we can add a new operation for the Pods like DELETE, UPDATE, CONNECT etc. By specifying a new operation, now apiserver being started to send a new event for this operation additional to create request.
Now we can specify more than one serverless function for the operation types by checking request operation type. For example:
switch req.Kind.Kind {
case "Pod":
switch req.Operation {
case v1beta1.Create:
// do something for create operation
case v1beta1.Delete:
// do something for delete operation
}
}
Also you can specify function name and namespace that you want to use while deploying the webhook server using these environment variables:
env:
- name: FUNCTION_NAME
value: requiredlabel
- name: FUNCTION_NAMESPACE
value: openfaas-fn
Taking it further
- Mutating webhooks
You could take this example and convert it from validating webhooks to mutating webhooks. This is useful when a user wants to upgrade or modify objects that are created, such as adding which user created them, or adding a compulsory memory limit.
- Adding more functions
In the example I used a single function, however, you could register more than one function, so that you can then have a function for validating memory limits, and a separate one for checking that a minimum set of labels are present
Join the community
Do you have questions, comments or suggestions? Tweet to @openfaas.
Want to support our work? You can become a sponsor as an individual or a business via GitHub Sponsors with tiers to suit every budget and benefits for you in return. Check out our GitHub Sponsors Page
Acknowledgements
- Special Thanks to Alex Ellis for all guidance and for merging changes into OpenFaaS to better support this workflow.
- Special Thanks to Furkan Türkal for all the support.