Hello and welcome to this lecture.
In this lecture, we look at deploying
multiple schedulers in a Kubernetes cluster.
Now, we have seen how the default scheduler works
in Kubernetes in the previous lectures.
It has an algorithm that distributes parts across nodes evenly,
as well as takes into consideration various conditions
we specify through taints and tolerations and node affinity, et cetera,
but what if none of these satisfies your needs?
Say you have a specific application that requires its components
to be placed on nodes after performing some additional checks.
You decide to have your own scheduling algorithm to place
parts on nodes so that you can add your own custom conditions
and checks in it.
Kubernetes is highly extensible.
You can write your own Kubernetes scheduler program,
package it and deploy it as the default scheduler
or as an additional scheduler in the Kubernetes cluster.
That way, all of the other applications can go through the default scheduler.
However, some specific applications
that you may choose can use your own custom scheduler.
Your Kubernetes cluster can have multiple schedulers at a time.
When creating a pod or a deployment,
you can instruct Kubernetes to have the pod
scheduled by a specific scheduler, so let's see how that's done.
Now, when there are multiple schedulers, they must have different names
so that we can identify them as separate schedulers.
The default scheduler is named default-scheduler,
and this name is configured
in a kube-scheduler configuration file that looks like this.
Now, the default scheduler doesn't really need one
because if you don't specify a name,
it sets the name to default-scheduler,
but this is how it would look if you were to create one.
For the other schedulers, we could create a separate configuration file
and set the scheduler name like this.
Let's start with the most simplest way
of deploying an additional scheduler.
Now, earlier we saw how to deploy the Kubernetes kube-scheduler.
We download the kube-scheduler binary
and run it as a service with a set of options.
Now, to deploy an additional scheduler,
you may use the same kube-scheduler binary
or use one that you might have built for yourself,
which is what you would do if you needed
the scheduler to work differently.
In this case, we are going to use the same binary
to deploy the additional scheduler,
and this time we point the configuration
to the custom configuration file that we created.
Each scheduler uses a separate configuration file,
and with each file having its own scheduler name.
Note that there are other options to be passed in such
as the kube config file to authenticate into the Kubernetes API,
but I'm just keeping that for now, just to keep it super simple.
This is not how you would deploy a custom scheduler
99% of the time today because with kubeadm deployment,
all the control plane components run as a pod
or a deployment within the Kubernetes cluster.
Let's look at another way.
Let's look at how it works
if you were to deploy the scheduler as a pod.
So we create a pod definition file and specify the kube config property,
which is the path to the scheduler con file
that has the authentication information to connect to the Kubernetes API server.
We then pass in our custom kube-scheduler
configuration file as a config option to the scheduler.
Note that we have the scheduler name specified in the file,
so that's how the name gets picked up by the scheduler.
Now, another important option to look here is the leader elect option,
and this goes into the kube-scheduler configuration.
The leader elect option is used
when you have multiple copies of the scheduler
running on different master nodes,
as in a high availability setup where you have multiple master nodes
with the kube-scheduler process running on both of them.
If multiple copies of the same scheduler are running on different nodes,
only one can be active at a time,
and that's where the leader elect option helps in choosing a leader
who will lead the scheduling activities.
We will discuss more about HA setup in another section.
In case you do have multiple masters, just remember that you can pass
in this additional parameter to set a log object name,
and this is to differentiate the new custom
scheduler from the default election process.
Now let's take a look at how to deploy
the additional scheduler as a deployment, and for this,
I'm going to go into the Kubernetes documentation pages
and for the one for configuring multiple schedulers.
If you look here first of all, it shows you how to,
if you were to build your own scheduler,
how you could clone the Kubernetes repo
and then make changes to the kube-scheduler and build it
and package it into a docker image.
Then here you can see the config file
to create a scheduler as a deployment.
You can ignore all of this to begin with.
This is what it is really.
Here you have the deployment, and this deployment is similar to the pod,
but it's just that it's a deployment,
the pod that we just looked at but it's just that it's a deployment.
Here you have the image which is the custom kube-scheduler image,
and this is the config file that we just talked about
which has my-scheduler config file.
The binary is the kube-scheduler binary.
Now for this to work there are some additional prerequisites.
Some of these are like the service account and clusteral bindings.
These are basically for authentication.
We discuss about clusteral bindings and cluster roles
and service accounts in the odd section.
If you haven't gone through that yet, just hold off on this.
You can basically ignore this for now.
The only other thing is how this file is passed into the deployment.
You can either create this file locally
and then pass it in as a volume mount
as it's done for the other pods usually,
or another thing you could do that's done here.
The approach that's used here is basically creating a config map.
This is the kube-scheduler configuration that we just talked about with
the leader elect option set to false
because I think the replicas is just one.
Then you have the scheduler name here.
This is the config file.
Then this config file is passed in as a volume here as a config file.
Basically whatever is the content in that particular config
map is then mapped to a volume mount to a volume here, this specific location.
Then from this location,
you have this YAML file which basically has this content.
This is how that maps.
Again, if you haven't gone through the volume
and volume mount sections of this course
then just hold off on this for now,
then you'll understand that when we talk about these.
Then again, liveness probes, readiness probes,
resources and security contexts,
all of these are sections that we talk about later.
Again, you can ignore these for now,
but what we need to understand here is just that this section
on how this file is passed to this custom schedule.
That's about that.
Just proceeding with our lecture.
When you run the get pods command in the kube system
namespace you can then see the new custom scheduler running.
This is if you ran it as a pod, and if you ran it as a deployment,
then you'll probably see a slightly different naming convention,
but you'll be able to see the pod there.
Just make sure you're checking the right namespace.
Now, once we have deployed that custom scheduler,
the next step is to configure a pod
or a deployment to use this new scheduler.
How do you use our custom scheduler?
Here we have a pod definition file.
What we need to do is add a new field called schedulerName
and specify the name of the new scheduler.
That's basically it.
This way, when the pod is created the right scheduler gets picked up
and the scheduling process works.
We now create the pod using the kubectl create command.
If the scheduler was not configured correctly,
the pod will continue to remain in a pending state.
If everything is good then the pod will be in a running state.
If the pod is in a pending state,
then you can look at the logs under the pod describe command,
the kubectl describe command.
You'll mostly notice that the scheduler isn't configured correctly.
Now how do you know which scheduler picked it up?
We have multiple schedulers.
How do you know which scheduler picked up scheduling a particular pod?
Now we can view this in the events using the kubectl
get events command with the -O wide option.
This will list all the events in the current name space,
and look for the scheduled events.
As you can see the source of the event
is the custom scheduler that we created.
That's the name that we gave to the custom scheduler.
The message says that successfully assigned the image.
That indicates that it's working.
You could also view the logs
of the scheduler in case you run into issues.
For that, view the logs using the kubectl logs command
and provide the scheduler name,
either the pod name or the deployment name, and then the right namespace.
That's it for this lecture.
I'll see you in the next one.