loading...

Kubernetes – Multitenancy

Kubernetes also has an additional construct for isolation at the cluster level. In most cases, you can run Kubernetes and never worry about namespaces; everything will run in the default namespace if not specified. However, in cases where you run multitenancy communities or want broad-scale segregation and isolation of the cluster resources, namespaces can be used to this end. True, end-to-end multitenancy is not yet feature complete in Kubernetes, but you can get very close using RBAC, container permissions, ingress rules, and clear network policing. If you’re interested in enterprise-strength multitenancy right now, Red Hat’s Openshift Origin (OO) would be a good place to learn.

You can check out OO at https://github.com/openshift/origin.

To start, Kubernetes has two namespaces—default and kube-system. The kube-system namespace is used for all the system-level containers we saw in Chapter 1, Introduction to Kubernetes, in the Services running on the minions section. UI, logging, DNS, and so on are all run in kube-system. Everything else the user creates runs in the default namespace. However, our resource definition files can optionally specify a custom namespace. For the sake of experimenting, let’s take a look at how to build a new namespace.

First, we’ll need to create a namespace definition file test-ns.yaml like the one in the following lines of code:

apiVersion: v1 
kind: Namespace 
metadata: 
  name: test

We can go ahead and create this file with our handy create command:

$ kubectl create -f test-ns.yaml

Now, we can create resources that use the test namespace. The following listing, ns-pod.yaml, is an example of a pod using this new namespace:

apiVersion: v1 
kind: Pod 
metadata: 
  name: utility 
  namespace: test 
spec: 
  containers: 
  - image: debian:latest 
    command: 
      - sleep 
      - "3600" 
    name: utility 

While the pod can still access services in other namespaces, it will need to use the long DNS form of <service-name>.<namespace-name>.cluster.local. For example, if you were to run a command from inside the container in listing ns-pod.yaml, you could use node-js.default.cluster.local to access the Node.js example from Chapter 2, Pods, Services, Replication Controllers, and Labels.

Here is a note about resource utilization. At some point in this book, you may run out of space on your cluster to create new Kubernetes resources. The timing will vary based on cluster size, but it’s good to keep this in mind and do some cleanup from time to time. Use the following commands to remove old examples:
 $ kubectl delete pod <pod name>
 $ kubectl delete svc <service name>
 $ kubectl delete rc <replication controller name>
 $ kubectl delete rs <replicaset name>
.

Limits

Let’s inspect our new namespace a bit more. Run the describe command as follows:

$ kubectl describe namespace/test

The following screenshot is the result of the preceding command:

The describe namespace

Kubernetes allows you to both limit the resources used by individual pods or containers and the resources used by the overall namespace using quotas. You’ll note that there are no resource limits or quotas currently set on the test namespace.

Suppose we want to limit the footprint of this new namespace; we can set quotas as shown in the following listing quota.yaml:

apiVersion: v1 
kind: ResourceQuota 
metadata: 
  name: test-quotas 
  namespace: test 
spec: 
  hard:  
    pods: 3 
    services: 1 
    replicationcontrollers: 1 

 

In reality, namespaces would be for larger application communities and would probably never have quotas this low. I am using this for ease of illustration of the capability in this example.

Here, we will create a quota of 3 pods, 1 RC, and 1 service for the test namespace. As you have probably guessed, this is executed once again by our trusty create command, as follows:

$ kubectl create -f quota.yaml

Now that we have that in place, let’s use describe on the namespace, as follows:

$ kubectl describe namespace/test

The following screenshot is the result of the preceding command:

The describe namespace after the quota is set

You’ll note that we now have some values listed in the quota section, and that the limits section is still blank. We also have a Used column, which lets us know how close to the limits we are at the moment. Let’s try to spin up a few pods using the following definition busybox-ns.yaml:

apiVersion: v1 
kind: ReplicationController 
metadata: 
  name: busybox-ns 
  namespace: test 
  labels: 
    name: busybox-ns 
spec: 
  replicas: 4 
  selector: 
    name: busybox-ns 
  template: 
    metadata: 
      labels: 
        name: busybox-ns 
    spec: 
      containers: 
      - name: busybox-ns 
        image: busybox 
        command: 
          - sleep 
          - "3600" 

You’ll note that we are creating four replicas of this basic pod. After using create to build this RC, run the describe command on the test namespace once more. You’ll notice that the Used values for pods and RCs are at their max. However, we asked for four replicas and can only see three pods in use.

Let’s see what’s happening with our RC. You might attempt to do that with the following command:

kubectl describe rc/busybox-ns

However, if you try, you’ll be discouraged by being met with a not found message from the server. This is because we created this RC in a new namespace and kubectl assumes the default namespace if not specified. This means that we need to specify --namepsace=test with every command when we wish to access resources in the test namespace.

We can also set the current namespace by working with the context settings. First, we need to find our current context, which is found with the following command:
$ kubectl config view | grep current-context
Next, we can take that context and set the namespace variable like in the following code:
$ kubectl config set-context <Current Context> --namespace=test
Now, you can run the kubectl command without the need to specify the namespace. Just remember to switch back when you want to look at the resources running in your default namespace.

Run the command with the namespace specified as shown in the following command. If you’ve set your current namespace as demonstrated in the tip box, you can leave off the --namespace argument:

$ kubectl describe rc/busybox-ns --namespace=test

The following screenshot is the result of the preceding command:

Namespace quotas

As you can see in the preceding image, the first three pods were successfully created, but our final one fails with a Limited to 3 pods error.

This is an easy way to set limits for resources partitioned out at a community scale. It’s worth noting that you can also set quotas for CPU, memory, persistent volumes, and secrets. Additionally, limits work in a similar way to quota, but they set the limit for each pod or container within the namespace.

Comments are closed.

loading...