by 
For the demo purpose, we are going to use below three pods that are currently running in our two-node Kubernetes cluster.
[node1 ~]$ kubectl get pods NAME READY STATUS RESTARTS AGE my-nginx-66b6c48dd5-dhb8p 1/1 Running 0 2m34s my-nginx-66b6c48dd5-l6qxz 1/1 Running 0 2m34s my-nginx-66b6c48dd5-wg75z 1/1 Running 0 2m34s
It is always easy to check the logs of a single pod by just using kubectl logs <pod> syntax. For example, here we are checking logs of my-nginx-66b6c48dd5-dhb8p pod by using kubectl logs my-nginx-66b6c48dd5-dhb8p the command as shown below.
[node1 ~]$ kubectl logs my-nginx-66b6c48dd5-dhb8p
But the real challenge comes when you are having some issues and you don’t know the pod on which you need to check the error. In a real scenario, you might have more than 5 replica pods running for an application stream. With the priority issue in hand, usually, you won’t have much time to check the logs of each of the pods one by one. So it is always a better idea to run some quick command or a small script to check the logs of multiple pods at once. This will be beneficial in multiple ways:-
- First, you will be able to figure out the problematic pod where the error is happening.
- Second, you will be able to take the error output and save it in a file for reference purposes.
- Third, it will save your precious time of running single command every time for each of the pods.
- And, finally, you can run almost the same command every time to find multiple errors.
I always use the one-liner command to get the error I would like to check on multiple pods. But before that, you need first to get the list of all the pods on which the error needs to be checked. For example, here we are checking the logs of all the pods that start with the name my-nginx using the below command.
[node1 ~]$ kubectl get pods | grep my-nginx | awk {'print $1'}
my-nginx-66b6c48dd5-dhb8p
my-nginx-66b6c48dd5-l6qxz
my-nginx-66b6c48dd5-wg75z
Then you can simply run a for loop on the above command to print the entire logs of all the pods that start with the name my-nginx.
[node1 ~]$ for i in $(kubectl get pods | grep my-nginx | awk {'print $1'});do echo "******$i******\n";kubectl logs $i;done
******my-nginx-66b6c48dd5-dhb8p******
******my-nginx-66b6c48dd5-l6qxz******
******my-nginx-66b6c48dd5-wg75z******
If you are looking for some specific error from the logs then you can grep the keyword like below. For example, here I am looking for any error from the pods by just grepping the "error" keyword.
[node1 ~]$ for i in $(kubectl get pods | grep my-nginx | awk {'print $1'});do echo "******$i******\n";kubectl logs $i | grep -i error;done
******my-nginx-66b6c48dd5-dhb8p******
******my-nginx-66b6c48dd5-l6qxz******
******my-nginx-66b6c48dd5-wg75z******
If you want to save all the above output logs to a file then you can use the redirection operator(>>) and append all the output to a file say error_log as shown below. After saving the output, you can verify the contents of the file by using cat error_log the command shown below.
[node1 ~]$ for i in $(kubectl get pods | grep my-nginx | awk {'print $1'});do echo "******$i******\n";kubectl logs $i | grep -i error;done >> error_log
[node1 ~]$ cat error_log
******my-nginx-66b6c48dd5-dhb8p******
******my-nginx-66b6c48dd5-l6qxz******
******my-nginx-66b6c48dd5-wg75z*****
Also, read Running Jenkins in Kubernetes Cluster with Persistent Volume
