Opscraft
Kubernetes troubleshooting · Lesson 10 of 11 · 3 min read

GitOps (Flux) troubleshooting

When the cluster doesn't match Git, check the Flux chain from source to workload:

Enlarge diagramGitRepositorycan Flux fetch it?Kustomizationbuilds andapplies?HelmReleaseinstalls orupgrades?Workloadshealthy?flux get sources git -A · flux get kustomizations -A · flux logs --level=error
Figure 1. Troubleshoot Flux along its chain: if a link is not Ready, everything after it is stale.
Swipe sideways to see all of it, or tap to open it full screen.
flux check                                   # controllers healthy, versions
flux get sources git -A                      # can Flux fetch the repo? (auth, branch, deploy key)
flux get kustomizations -A                   # Ready? which revision applied? error message
flux get helmreleases -A                     # chart install/upgrade failures
flux logs --level=error --all-namespaces
kubectl describe kustomization apps -n flux-system
flux reconcile kustomization apps --with-source     # force a sync now
flux diff kustomization apps --path ./clusters/prod # what would change
flux suspend kustomization apps / flux resume kustomization apps

Typical failures:

  • Source not ready — Git auth (deploy key rotated), wrong branch/path, SOPS decryption key missing.
  • Kustomization failed — invalid YAML/Kustomize build error, a CRD not installed yet (use dependsOn so CRDs and controllers apply first), validation webhook rejection, immutable field changes (e.g. a Deployment selector or Job spec — needs force: true or delete/recreate).
  • Health checks timing out — Flux waits for workloads to become Ready; a crashing Deployment shows up here first.
  • HelmRelease failed — install/upgrade errors, values errors; check remediation/rollback settings.
  • Drift — someone kubectl edited a resource: Flux reverts it on the next reconcile; changes must go through Git. In emergencies, flux suspend, fix, then revert in Git and resume.
  • Image automation not updating tags — ImageRepository can't list ECR (IRSA), policy (semver/regex) doesn't match tags, push to Git failing.