7 Practical Kubernetes Troubleshooting Use Cases Every DevOps Engineer Should Know in 2026

Itcharlie Mj

11 min read

·

Oct 15, 2026

--

Listen

Share

Press enter or click to view image in full size

🚀 7 Practical Kubernetes Troubleshooting Scenarios Every DevOps Engineer Should Know in 2026!

Kubernetes is one of the most important skills in today’s DevOps and Cloud-Native ecosystem. But knowing how to deploy applications is not enough. A good DevOps engineer must also know how to troubleshoot problems when things go wrong.

From unhealthy nodes and disk pressure to container startup failures and memory issues, understanding Kubernetes troubleshooting is essential for managing reliable production environments.

In this article, I cover 7 real-world Kubernetes troubleshooting use cases, including:

🔹 Kubernetes Node NotReady — Diagnose node health, kubelet, and networking issues.

🔹 Node DiskPressure — Identify disk space and inode problems.

🔹 OOMKilled — Troubleshoot container memory limits and resource consumption.

🔹 CreateContainerError — Investigate container creation and runtime failures.

🔹 CreateContainerConfigError — Fix missing ConfigMaps, Secrets, and configuration issues.

🔹 ImagePullBackOff — Resolve image registry, authentication, and image-pulling problems.

🔹 RunContainerError — Troubleshoot startup commands, file permissions, and security contexts.

💡 My recommendation: Don’t just memorize Kubernetes commands. Build your own Kubernetes lab, intentionally reproduce these errors, investigate logs and events, identify root causes, and practice fixing them.

Real troubleshooting experience is what helps you move from learning Kubernetes to becoming a confident DevOps engineer.

Kubernetes skills remain among the most valuable skills in the DevOps and Cloud-Native ecosystem. However, understanding Kubernetes commands is only one part of becoming a skilled Kubernetes engineer.

The real challenge begins when applications fail, containers refuse to start, nodes become unhealthy, or workloads suddenly get terminated.

Understanding how to identify, investigate, and resolve these problems is an essential skill for anyone working with Kubernetes.

Here are 7 practical Kubernetes troubleshooting scenarios worth learning, practicing, and saving for your next Kubernetes lab.

1. Kubernetes Node NotReady — How to Diagnose and Fix It

What is Node NotReady?

A Kubernetes node can become NotReady when the control plane determines that the node is not healthy or is no longer reporting its status correctly.

The kubelet running on each node regularly communicates with the Kubernetes control plane and reports information about the node’s health.

Common reasons include:

• Kubelet service failure.
• Network connectivity problems.
• Container runtime issues.
• Container Network Interface (CNI) problems.
• Insufficient CPU or memory.
• Disk or PID pressure.
• Unhealthy Pod Lifecycle Event Generator (PLEG).

Step 1: Identify the Problem

Start by checking the status of your cluster nodes.

kubectl get nodes

Example output:

NAME              STATUS     ROLES    AGE   VERSION
master            Ready      control-plane  51m  v1.31.0
node-worker-1     NotReady   worker         49m  v1.31.0
node-worker-2     Ready      worker         47m  v1.31.0

Here, node-worker-1 is reporting a NotReady status.

Step 2: Investigate Node Conditions

Use the following command:

kubectl describe node node-worker-1

Pay particular attention to these conditions:

• MemoryPressure: The node is experiencing memory pressure.
• DiskPressure: The node has insufficient available disk space or inodes.
• PIDPressure: The node is running low on available process IDs.
• Ready: Indicates whether the node is healthy and ready to accept workloads.

For example:

Ready: False
Reason: KubeletNotReady
Message: PLEG is not healthy

This indicates that the kubelet is reporting an unhealthy Pod Lifecycle Event Generator.

Step 3: Check Network Connectivity

Test connectivity between the control plane and the affected worker node.

ping <node-IP>

You can also investigate network routes:

traceroute <node-IP>

Check firewall rules, routing, DNS, and the network connectivity required by your Kubernetes control plane and nodes.

Remember that a failed ping alone does not prove a network failure because ICMP traffic may be blocked.

Step 4: Check Kubelet Status

Connect to the affected worker node and execute:

sudo systemctl status kubelet

If the service is unhealthy, inspect its logs:

sudo journalctl -u kubelet -n 100 --no-pager

If appropriate, restart the service:

sudo systemctl restart kubelet

Step 5: Investigate Container Runtime and Networking

Check whether containerd or your configured container runtime is running:

sudo systemctl status containerd

Inspect the system pods:

kubectl get pods -n kube-system -o wide

Check your CNI plugin and kube-proxy where applicable.

How to Fix Node NotReady

Depending on the root cause:

• Restore network connectivity between nodes.
• Resolve kubelet service failures.
• Investigate container runtime errors.
• Repair CNI configuration or networking issues.
• Free up memory, disk space, or process capacity.
• Verify that required Kubernetes ports are accessible.

Important: Do not restart services or delete system pods blindly. First identify the underlying failure.

2. Kubernetes Node DiskPressure — How to Identify and Resolve It

What is DiskPressure?

Kubernetes Node DiskPressure occurs when a node runs low on available filesystem space or inodes.

When this happens, Kubernetes identifies the node as experiencing disk pressure and may evict pods to protect the node and its critical workloads.

A typical warning looks like:

Warning  NodePressure
Node status is now: NodeHasDiskPressure

Common Causes

• Accumulation of unused container images.
• Large container logs.
• Temporary files consuming disk space.
• Application-generated files.
• Insufficient node storage capacity.
• Exhausted filesystem inodes.

Step 1: Check Node Conditions

kubectl describe node <node-name>

Look for:

DiskPressure: True

You can also check all nodes:

kubectl get nodes

Step 2: Investigate Disk Usage

Connect to the affected node.

Check filesystem capacity:

df -h

Check inode availability:

df -i

Identify large directories:

sudo du -sh /var/lib/containerd
sudo du -sh /var/log

The exact directories depend on your container runtime and system configuration.

Step 3: Check Kubernetes Workloads

Identify pods running on the affected node:

kubectl get pods -A -o wide

Check recent events:

kubectl get events -A --sort-by=.metadata.creationTimestamp

How to Fix Node DiskPressure

1. Clean Up Unused Container Images

For containerd-based environments, inspect images using:

sudo crictl images

Remove unused images carefully using the appropriate container runtime tools.

2. Investigate Large Logs

Check log directories and configure log rotation to prevent unlimited log growth.

3. Increase Disk Capacity

For cloud-based Kubernetes environments, consider increasing the attached disk capacity.

For example, on AWS, you can expand an EBS volume and then extend the filesystem when required.

4. Separate Application Data

Move high-volume application data and logs to appropriately configured storage.

5. Configure Monitoring

Use Prometheus and Grafana to monitor node filesystem utilization and inode consumption.

You can also configure alerts before disk usage reaches Kubernetes eviction thresholds.

3. Kubernetes OOMKilled — How to Troubleshoot Memory Issues

What is OOMKilled?

OOMKilled stands for Out Of Memory Killed.

It occurs when a container is terminated by the Linux out-of-memory mechanism because it exceeds its available memory limit or when the system experiences memory exhaustion.

A common indication is exit code 137.

Example:

NAME           READY   STATUS      RESTARTS   AGE
web-app-pod    0/1     OOMKilled   1          4m

OOMKilled is commonly associated with:

• Insufficient container memory limits.
• Application memory leaks.
• Unexpected traffic spikes.
• Memory-intensive operations.
• Incorrect resource allocation.

Step 1: Identify the OOMKilled Container

kubectl get pods

Step 2: Inspect Pod Details

kubectl describe pod <pod-name>

Look for the container’s last termination state:

Last State:     Terminated
Reason:         OOMKilled
Exit Code:      137

Step 3: Investigate Memory Consumption

Check current resource usage:

kubectl top nodes

Check pod memory consumption:

kubectl top pods -A

For individual containers:

kubectl top pod <pod-name> --containers

Step 4: Review Resource Requests and Limits

Example configuration:

resources:
  requests:
    memory: "256Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Adjust the values according to the application’s actual memory requirements and available node capacity.

Step 5: Check Pod QoS Class

kubectl get pod <pod-name> \
-o jsonpath='{.status.qosClass}'

Kubernetes provides three Quality of Service classes:

• Guaranteed
• Burstable
• BestEffort

These classes influence how Kubernetes prioritizes pods during node-pressure eviction.

Additional Linux Memory Investigation

You can inspect a running container’s OOM score:

cat /proc/<PID>/oom_score

And its OOM score adjustment:

cat /proc/<PID>/oom_score_adj

A higher effective OOM score generally means a process is more likely to be selected by the Linux OOM killer.

How to Fix OOMKilled

• Increase memory limits when justified.
• Investigate application memory leaks.
• Optimize memory-intensive operations.
• Monitor memory consumption over time.
• Configure realistic resource requests and limits.
• Avoid excessive memory overcommitment.

Important: Exit code 137 alone does not prove an OOM event. Always inspect the container’s termination reason and Kubernetes events.

4. Kubernetes CreateContainerError — Understanding Container Creation Failures

What is CreateContainerError?

CreateContainerError occurs when Kubernetes has progressed to the container creation stage, but the container runtime fails to create the container.

This can happen because of:

• Incorrect volume configuration.
• Invalid container runtime settings.
• Permission problems.
• Invalid commands or executable paths.
• Runtime-level errors.
• Incorrect mount configurations.

Example:

NAME           READY   STATUS                 RESTARTS   AGE
techops-app    0/1     CreateContainerError    0          5m

Step 1: Inspect Pod Details

kubectl describe pod techops-app

Check the container state, reason, and Events section.

Step 2: Check Container Logs

kubectl logs techops-app

If the container has not been created successfully, logs may not be available.

In that situation, focus on pod events and kubelet logs.

Step 3: Investigate Pod Events

kubectl get events \
--field-selector involvedObject.name=techops-app

Look for messages describing the container creation failure.

Step 4: Verify Resource Availability

kubectl top nodes

Also inspect the pod’s resource requests and limits.

Step 5: Validate Volume Configuration

Check whether the required PersistentVolumeClaim exists:

kubectl get pvc

Inspect the relevant claim:

kubectl describe pvc <pvc-name>

Example volume configuration:

volumes:
  - name: data-volume
    persistentVolumeClaim:
      claimName: data-pvc

Make sure the PVC exists in the same namespace as the pod.

How to Fix CreateContainerError

• Correct invalid volume mounts.
• Verify container commands and executable paths.
• Check file and directory permissions.
• Investigate container runtime errors.
• Validate PVC and storage configurations.
• Review kubelet logs.

For node-level investigation:

sudo journalctl -u kubelet -n 100 --no-pager

5. Kubernetes CreateContainerConfigError — Fix Missing ConfigMaps and Secrets

What is CreateContainerConfigError?

CreateContainerConfigError occurs when Kubernetes cannot generate the required container configuration before creating the container.

It usually indicates a problem with the resources or configuration referenced by the pod.

Common causes include:

• Missing ConfigMaps.
• Missing Secrets.
• Incorrect resource names.
• Incorrect namespace references.
• Invalid environment variable references.
• Incorrect configuration settings.

Example:

NAME       READY   STATUS                      RESTARTS   AGE
web-pod    0/1     CreateContainerConfigError  0          11m

Step 1: Inspect the Pod

kubectl describe pod web-pod

Review the Events section and container waiting reason.

Step 2: Check ConfigMaps

List ConfigMaps:

kubectl get configmaps

Inspect the required ConfigMap:

kubectl describe configmap web-config

If it does not exist, create it:

kubectl create configmap web-config \
--from-file=config.yaml

Step 3: Check Secrets

List available Secrets:

kubectl get secrets

Inspect a specific Secret:

kubectl describe secret db-secret

If the Secret is missing, create it:

kubectl create secret generic db-secret \
--from-literal=password='your-password'

Never store real credentials in publicly accessible manifests or repositories.

Step 4: Verify Namespace

Remember that ConfigMaps and Secrets are namespace-scoped resources.

Check the pod namespace:

kubectl get pod web-pod -o jsonpath='{.metadata.namespace}'

Check the resources in the same namespace:

kubectl get configmaps,secrets -n staging

Step 5: Validate Pod Configuration

Review the ConfigMap and Secret references in your deployment YAML.

Example:

env:
  - name: DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-secret
        key: password

How to Fix CreateContainerConfigError

• Create missing ConfigMaps.
• Create missing Secrets.
• Correct resource names.
• Verify namespace configuration.
• Check environment variable references.
• Validate the pod’s YAML configuration.

Key takeaway: CreateContainerConfigError is often a configuration or dependency problem, not an application runtime problem.

6. Kubernetes ImagePullBackOff — How to Fix Image Pull Failures

What is ImagePullBackOff?

ImagePullBackOff occurs when Kubernetes cannot successfully pull the container image from the configured registry.

Kubernetes initially attempts to retrieve the image. When the attempt fails, it retries with an increasing delay between attempts.

Example:

NAME       READY   STATUS             RESTARTS   AGE
api-pod    0/1     ImagePullBackOff   0          1m

Common Causes

• Incorrect image name.
• Non-existent image tag.
• Private registry authentication failure.
• Network connectivity problems.
• Registry availability issues.
• Incorrect imagePullSecrets configuration.

Step 1: Check Pod Status

kubectl get pods

You might initially see:

ErrImagePull

After unsuccessful retries, the status can become:

ImagePullBackOff

Step 2: Inspect Pod Events

kubectl describe pod api-pod

Example event:

Warning Failed
Failed to pull image:
repository does not exist or may require authentication

This message helps identify whether the problem is related to the image or registry authentication.

Step 3: Verify the Image Name and Tag

Check your deployment YAML:

containers:
  - name: api-container
    image: nginx:1.27

Make sure the repository and tag exist.

You can also test pulling the image manually using your container runtime.

Step 4: Configure Private Registry Authentication

Create a registry Secret:

kubectl create secret docker-registry reg-secret \
  --docker-server=private-repo.com \
  --docker-username=myuser \
  --docker-password='your-password'

Reference the Secret in your pod specification:

spec:
  imagePullSecrets:
    - name: reg-secret

  containers:
    - name: private-api
      image: private-repo.com/internal-app:v3.0

How to Fix ImagePullBackOff

• Verify the image repository.
• Correct image tags.
• Confirm registry availability.
• Configure valid registry credentials.
• Check imagePullSecrets.
• Investigate DNS and network connectivity.
• Verify the service account’s image pull configuration.

Remember: ImagePullBackOff is a symptom. Always investigate the Events section to identify the actual reason behind the failed image pull.

7. Kubernetes RunContainerError — Troubleshooting Container Startup Failures

What is RunContainerError?

RunContainerError occurs when the container runtime encounters an error while attempting to start a container.

Unlike ImagePullBackOff, the image may already have been successfully downloaded.

However, the container cannot start because of an issue with its execution environment.

Common causes include:

• Incorrect startup commands.
• Missing executable files.
• File permission problems.
• Invalid volume mounts.
• Read-only filesystem restrictions.
• Security context misconfiguration.
• Container runtime problems.

Example:

NAME           READY   STATUS             RESTARTS   AGE
techops-app    0/1     RunContainerError  0          6m

Step 1: Inspect Pod Events

kubectl describe pod techops-app

Look for messages such as:

Error:
failed to start container

exec: "/app/start.sh":
permission denied

This indicates a problem with the executable file’s permissions.

Step 2: Check Container Logs

If the container started briefly before failing:

kubectl logs techops-app

For previous container instances:

kubectl logs techops-app --previous

Logs may not be available if the container never successfully started.

Step 3: Validate Commands and Entrypoints

Review the command and args fields in your YAML.

Example:

containers:
  - name: techops-container
    image: techops-image:v1.2.3
    command: ["/app/start.sh"]
    args: ["--env", "production"]

Verify that the specified executable exists inside the image and is executable.

Step 4: Check File Permissions

If the error mentions permission denied, investigate the relevant file permissions.

For example:

ls -l /app/start.sh

An executable script should have the appropriate execute permission.

You can configure permissions in your Dockerfile:

RUN chmod +x /app/start.sh

Step 5: Inspect Security Context

Check whether the configured user and group have permission to execute the application.

Example:

securityContext:
  runAsUser: 1000
  runAsGroup: 3000

Also investigate volume permissions, read-only mounts, and security restrictions.

How to Fix RunContainerError

• Correct invalid startup commands.
• Verify executable paths.
• Fix file permissions.
• Check volume mount configurations.
• Review security contexts.
• Validate container image configuration.
• Investigate container runtime and kubelet logs.

Key takeaway: A container image can be pulled successfully and still fail during startup. Understanding the difference between image-pulling and container-execution errors is important.

Bonus: Essential Kubernetes Troubleshooting Commands

Beyond these seven scenarios, every Kubernetes engineer should become comfortable with the following commands.

Cluster and Node Inspection

kubectl cluster-info
kubectl get nodes -o wide
kubectl describe node <node-name>
kubectl top nodes

Pod Troubleshooting

kubectl get pods -A
kubectl get pods -o wide
kubectl describe pod <pod-name>
kubectl logs <pod-name>
kubectl logs <pod-name> --previous

Events and Resource Inspection

kubectl get events -A
kubectl get events -A --sort-by=.metadata.creationTimestamp
kubectl top pods -A
kubectl get deployments -A

Kubernetes System Components

kubectl get pods -n kube-system
kubectl get daemonsets -n kube-system
kubectl get services -n kube-system

Node-Level Troubleshooting

sudo systemctl status kubelet
sudo journalctl -u kubelet -f
sudo systemctl status containerd
df -h
df -i
free -h

Final Thoughts: Learn Kubernetes by Troubleshooting Real Problems

Learning Kubernetes is not just about deploying applications, creating pods, or writing YAML manifests.

It is also about understanding what happens when things go wrong.

These seven troubleshooting scenarios provide a practical starting point for developing real-world Kubernetes administration and DevOps skills.

My recommendation is to build a small Kubernetes lab and intentionally reproduce these errors.

For example:

• Stop the kubelet and investigate a node failure.
• Fill up a test filesystem and observe DiskPressure.
• Deploy a container with an insufficient memory limit.
• Reference a missing ConfigMap.
• Use an incorrect container image.
• Configure an invalid startup command.
• Introduce a volume permission problem.

Then investigate the symptoms, read the events, analyze the logs, identify the root cause, and apply the appropriate fix.

Don’t just memorize Kubernetes commands. Understand why problems happen and how to solve them.

That is how you move from knowing Kubernetes to becoming confident in troubleshooting Kubernetes environments.

What other Kubernetes troubleshooting scenarios would you add to this list?

Share your experience in the comments. Your real-world examples could help someone else on their Kubernetes learning journey.