Kubernetes Integration Lifecycle: From CI Artefact to Recovery

Kubernetes integration is the complete lifecycle that turns an application artefact into an operable workload: build, configuration, deployment, identity, networking, secrets, storage, telemetry, upgrades, and recovery. Connecting a CI job to kubectl is only one step. The integration is complete when a team can change, observe, roll back or roll forward, restore, and retire the workload predictably.

Define the application contract before writing manifests

Record listening ports, protocols, startup and shutdown behaviour, resource shape, configuration, secrets, writable paths, persistence, dependencies, and identity. Decide whether the workload can run more than one replica and what happens when two instances process the same event. Containerisation does not make local files durable, a singleton concurrent, or a non-idempotent worker safe to retry.

Build one immutable image per source revision, test it before deployment, identify it by digest, and promote the same artefact between environments. Keep environment configuration outside the image. Define ownership and a retirement path for base images so patched application releases are produced when the operating-system layer changes.

Separate CI from desired-state delivery

Continuous integration should compile, test, scan, create provenance and a software bill of materials where required, publish the artefact, and propose a version change. Deployment credentials do not need to live in every build runner. A desired-state repository should show which reviewed version and configuration belongs in each environment.

GitOps controllers can pull and reconcile that desired state. The OpenGitOps principles describe state as declarative, versioned and immutable, pulled automatically, and continuously reconciled. Decide how emergency changes are recorded: an untracked kubectl edit may be reverted by the controller or become invisible drift. Define repository ownership, promotion, secret references, reconciliation alerts, and the break-glass path before production.

Package configuration without hiding behaviour

Helm can package related resources as a versioned release; the Helm chart best-practices guide covers values, templates, dependencies, labels, custom resources, and RBAC. Keep values as a deliberate public interface. Deeply nested templating, environment conditionals, and arbitrary manifest injection make rendered state difficult to review and test.

Render and validate changes in CI, enforce policy on the rendered resources, and compare them with the live API version. Sequence namespaces, custom-resource definitions, controllers, and custom resources explicitly. A Helm rollback restores Kubernetes resources from release history; it does not undo an incompatible database migration or restore deleted external data.

Integrate platform dependencies in a known order

Establish cluster and workload identity, registry access, DNS, ingress or Gateway routing, certificates, network policy, storage classes, backup targets, secret sources, and observability. Assign an owner and service objective to each external controller. Two controllers must not reconcile the same field, address, certificate, or DNS record.

Kubernetes Secret values are base64-encoded, not encrypted by that encoding, and are stored unencrypted in etcd by default unless encryption at rest is configured. Follow the official Secrets good practices: restrict get, list, and pod-creation permissions, encrypt at rest, limit each container's access, rotate credentials, and consider an external store. Avoid committing plaintext secret manifests to the desired-state repository.

Design health signals around traffic and recovery

A readiness probe controls whether a Pod receives Service traffic; liveness can restart a container; a startup probe protects slow initialisation from premature liveness checks. The Kubernetes probe guidance warns that incorrect liveness probes can cause cascading failures. Do not make liveness depend on every remote service: restarting all replicas during a database outage increases load without repairing the database.

Set resource requests from measured normal and peak behaviour, limits from failure and tenancy needs, and autoscaling bounds from dependency capacity. Expose user-centred service indicators alongside saturation, restarts, pending Pods, reconciliation lag, queue age, and storage health. Alerts need an action and owner; controller log volume is not an operating model.

Roll out application and schema changes independently

Use progressive delivery and verify application invariants, not only Pod readiness. For a database change, expand the schema compatibly, deploy code that tolerates old and new forms, migrate data with checkpoints, switch reads, and remove the old form later. Jobs need idempotency, deadlines, bounded retries, and visible completion. A failed hook must not leave the release permanently unchangeable.

Exercise a cancelled rollout, unavailable registry, invalid secret, failing admission rule, exhausted quota, unavailable zone, and stalled reconciliation. Document when to roll back, when to roll forward, and when to stop automation. Preserve evidence of the deployed image, configuration revision, and database state for incident diagnosis.

Operate upgrades, restoration, and retirement

Inventory cluster APIs, controllers, custom resources, charts, storage and network plugins, and their compatibility before an upgrade. Upgrade a representative non-production environment, exercise critical workloads, then roll through bounded production cohorts. Monitor deprecated APIs before they block the next control-plane version.

Back up application data and the configuration needed to recreate it, then restore into an isolated environment and measure recovery. Finally, make deletion a workflow: drain traffic, stop writers, retain or export required data, remove external load balancers and DNS, revoke identities and secrets, delete persistent resources intentionally, and confirm cost has stopped. Lifecycle integration ends with verified removal, not an abandoned namespace.