Introduction

When using Helm for managing Kubernetes applications, you may encounter a situation where an upgrade operation is stuck in the STATUS: pending-upgrade. This typically occurs when a previous operation was interrupted or not completed successfully.

Common Causes

The pending-upgrade status can arise from several scenarios:

  • Interrupted Operations: The upgrade process was canceled or the terminal session was closed unexpectedly.
  • Failed Hooks: Pre- or post-upgrade hooks may hang or fail, preventing the upgrade from completing.
  • Resource Conflicts: Kubernetes resources may be in an unstable state, blocking the upgrade.
  • Corrupted Release Data: The Helm release secret may contain incorrect or malformed status information.

Troubleshooting Steps

To resolve the issue, follow these steps:

Step 1: Check the Release Status

First, confirm that the release is indeed stuck:

helm list --namespace <your-namespace>

This command will show the current status of your releases.

Step 2: Inspect Release History

You can inspect the release history to identify any issues:

helm history <release-name> --namespace <your-namespace>

Step 3: Decode the Helm Release Secret

If the release is stuck, you may need to decode the Helm release secret to gather more information:

kubectl get secrets --namespace <your-namespace> | grep sh.helm.release.v1

Then, decode the relevant secret:

kubectl get secret <secret-name> --namespace <your-namespace> -o jsonpath='{.data.release}' | base64 --decode | gunzip

Step 4: Resolve the Stuck Operation

If you find that an operation is indeed stuck, you can attempt to delete the Helm release secret to reset the state:

kubectl delete secret sh.helm.release.v1.<release-name>.v<revision-number> --namespace <your-namespace>

After deleting the secret, you should be able to retry the upgrade command:

helm upgrade --namespace <your-namespace> --install --force --atomic --wait --version <version> --values ./<your-values-file>.yaml <release-name> <chart-name>

Conclusion

By following these steps, you should be able to resolve the pending-upgrade issue and successfully upgrade your Helm release. Always ensure to check the status and history of your releases to prevent such issues in the future.