Let's go through the lab on cluster maintenance.
Let us first explore the environment.
How many nodes do you see in the cluster?
We will set the alias to kubectl.
Let's do get nodes.
We see that there are two nodes, the controlplane and node01.
That's two.
Now, how many applications do you see hosted on the cluster?
Check the number of deployments.
Let's do get deploy and we see that we have one deployment.
That's just one.
Now, which nodes are the applications hosted on?
Let's do get pods and we see there are three pods.
Let's use the wide option to see what nodes they're on.
We see that they're on node01.
All of them are on node01.
That's the answer to this.
Now we need to take node01 out for maintenance.
Empty the node of all applications and mark it unscheduleable.
We learned that the command to take a node down for maintenance
is drain so we're going to do a drain node01.
When we run that, it says it's not drained
because you cannot delete DaemonSet-managed pods.
There are DaemonSet-managed pods on this node.
We could use the ignore-daemondet option.
Let's do that.
On the same command with the ignore DaemonSet option,
and it's going to evict the pods for the blue deployment.
That's done.
Not fully.
Let's wait for the third pod to be evicted.
Okay, so that's done.
Now what nodes are the apps on now?
Let's take a look at the pods and see where they are.
Now you can see that they're all on the controlplane node.
If you look at the status of the node,
we see that node01 is in a scheduling disabled state.
To answer to this question, all of them are on controlplane.
Applying patches on node01, hold on.
The maintenance tasks have been completed.
We now need to configure the node01 to be scheduled again.
We saw that it's in a scheduling disabled state.
To bring a node out of the scheduling disabled state
and back into the cluster,
we do uncordon and then we give this node name.
The node is uncordoned.
When you do get nodes, it's now back to its normal original state.
The next question is how many pods are scheduled on node01 now?
Let's run the get pods command again with the wide option,
and we see that they're all still
on the controlplane node.
There are no pods currently on node01.
That's 0.
Why are there no pods on node01?
Is it because node01 is faulty? No.
Is it cordoned? No, we just uncordoned it.
Then is it because it did not upgrade successfully? No.
It's because only when new pods are created, they'll be scheduled.
We have not deployed any new application after we uncordoned
the node so we don't expect anything to be on that node.
The pods that were moved to controlplane
are not going to move back automatically to node01.
Now, why are the pods placed on the controlplane node?
Check the controlplane node for details.
Usually, we don't expect applications to be deployed on the controlplane
because it usually has a taint of some sort,
but let's look at the node controlplane
and see if it has any taints.
We see that there are no taints.
That's one of the reason that the pods
were scheduled on the controlplane node.
It's not because it's faulty or can never run pods on a master node.
It does not have any taints.
It does not have any taints so that's the correct answer.
Usually, when you have a multi-node cluster,
the master node or the controlplane node would have taints
that would prevent pods from being placed on it.
In this case, it doesn't, and that's why I have it.
Now we're going to time travel to the next maintenance window.
It's time to carry out the maintenance activity on node01, again,
try draining the node again, using the same command.
This is the same kubectl drain node command
with the ignore demand says option.
Let's run that again, but we see that there is an error.
It says that cannot delete pods not managed by ReplicationController,
ReplicaSet, Job, DaemonSet, or StatefulSet so use --force to override.
Did that work, this command?
No, it did not.
Why did the drain command fail on node01?
It worked the first time.
There seems to be a change.
Here it says there's a default hr-app.
Let's take a look at that.
We see that, indeed, there is a hr-app,
which seems to be a new addition and which is a pod that has been added
or an application that is deployed as a pod
since the first maintenance window.
Let's see where that is scheduled.
We see that that one is scheduled on node01.
If you look at the error message here,
you cannot delete pods not managed by any of these,
and when you do a drain.
When you do a drain,
what happens is it first cordons node and marks it unscheduleable,
and then it tries to evict all the pods.
Now it is easy to delete the pods that are managed
by either one of these by a ReplicaSet, or Job,
or ReplicationController because when you delete the pod,
they take care of recreating the pod on another node.
That's less risky, so as part of the draining process,
you can just go ahead and kill the pods.
Then the ReplicaSet or whatever it is that's managing the pod would take
care of recreating them on the other node.
In this case, it's a pod, just a pod.
It's not one that is created as part
of a deployment or ReplicaSet or any of this.
If it deleted this pod as part of this training process,
that pod and anything that part is created, any data that is created
or stored locally on that pod is going to be lost forever.
That's why you have this error message here.
That's why it did not kill or terminate this pod.
The answer to this question
is that this is because there's a pod on node01,
which is not part of a ReplicaSet
or any of these.
What is the name of the pod hosted on node01
that is not part of the ReplicaSet?
It's hr-app.
Now what would happen to hr-app if node01
is drained forcefully?
There is an option to drain the node, as you can see here,
using the force option.
Let's try that out, --force.
What's going to happen is that this pod is going to be deleted
and it's going to be gone forever.
There's nothing here that's going to move
or reschedule this pod on any other worker node.
That's mainly the reason why it's preventing us from doing that.
The answer to this question is that hr-app will be lost forever.
Now we did not want to do that because hr-app
is a critical application that should not be destroyed.
We have now reverted back to the previous state
and redeploy the hr-app as a deployment.
Now, you look at the deployments,
we see that the hr-app is now deployed as a deployment.
If you look at the pods,
we see that the hr-app is indeed deployed as a deployment.
This is the ReplicationController, and then you have the deployment here.
This is also scheduled on node01.
Now hr-app is a critical app and we don't want it to be removed
and we also do not want to schedule any more parts on node01.
Currently, the hr-app is on node01,
and we don't want to schedule anything more
on node01 because apparently, it's a critical app.
We need to mark node01 as unscheduleable
so that no new pods are scheduled on the node,
but we also have to make sure that the hr-app is not affected.
What we want to do is--
Basically, what we're saying is we have to currently the node,
if a new application is deployed,
it's going to be scheduled here because it's in a ready state
and we want to prevent that from happening.
At the same time, we also don't want to kill this app by doing a drain.
The other option is to just cordon.
We're going to do kubectl cordon node01.
All that's going to do is mark the node as SchedulingDisabled
but at the same time,
the hr-application is just going to be fine.
Okay, so that's the end of this lab.