Skip to content

Kubernetes Level 01 – Day 11: Resolve Pod Deployment Issue

This document explains how to troubleshoot and fix a Pod that is failing due to an incorrect container image tag. The issue prevented the webserver Pod from reaching the Running state.





Objective

Fix the Pod named webserver so that both containers run successfully.

  • Pod Name: webserver
  • Container 1: nginx-container (failing)
  • Container 2: sidecar-container (ubuntu:latest) (running)
  • Target State: 2/2 Running



Understanding the Problem

The Pod is partially running. One container starts correctly, but the other fails during image pull.

When you check Pod status:

kubectl get pod webserver

You may see:

ImagePullBackOff

This indicates that Kubernetes cannot download the container image from the registry.



Step 1: Identify the Exact Error

Describe the Pod to inspect events:

kubectl describe pod webserver

In the Events section, you may see something similar to:

Failed to pull image "nginx:latests": not found

This shows the image tag is incorrect.

The correct tag is nginx:latest, not nginx:latests.



Step 2: Export Current Pod Definition

Extract the YAML configuration:

kubectl get pod webserver -o yaml > webserver.yaml



Step 3: Fix the Image Tag

Open the file:

vi webserver.yaml

Locate the container definition for nginx-container.

Incorrect configuration:

- image: nginx:latests
  imagePullPolicy: IfNotPresent
  name: nginx-container

Correct it to:

- image: nginx:latest
  imagePullPolicy: IfNotPresent
  name: nginx-container

Remove unnecessary runtime-generated fields if needed (such as resourceVersion, uid, and status) before reapplying.



Step 4: Replace the Pod

Since Pods are immutable regarding container image fields, recreate it.

Option 1:

kubectl replace --force -f webserver.yaml

Option 2:

kubectl delete pod webserver
kubectl apply -f webserver.yaml

This deletes the broken Pod and creates a corrected version.



Step 5: Verify

Check status:

kubectl get pod webserver

Expected output:

webserver   2/2   Running   0   5s

Both containers should now be operational.



Deep Dive: What is ImagePullBackOff?

When Kubernetes attempts to pull a container image and fails, it retries with increasing delay intervals. This retry mechanism is called exponential backoff.

ImagePullBackOff means:

  • Image pull failed
  • Kubernetes is waiting before retrying

It does not mean the Pod will never recover; it retries automatically.



Common Causes of ImagePullBackOff

  1. Typographical error in image name or tag
  2. Image does not exist in registry
  3. Authentication failure for private registry
  4. Network connectivity problems

In this case, the issue was a simple typo in the tag (latests).



Internal Flow During Image Pull

  1. Kubelet receives Pod spec
  2. It checks if image exists locally
  3. If not, it contacts container registry
  4. If image not found → Pull fails
  5. Kubelet enters backoff retry mode

Correcting the image tag resolves the issue immediately.



Key Outcome

The webserver Pod has been corrected and is now running both containers successfully. The issue was identified through event inspection and resolved by fixing an invalid image tag and recreating the Pod.