Platform secrets

The Secrets page lists all platform secrets managed by Sealed Secrets. Platform administrators can create new secrets and update existing ones. Secret values are write-only and are encrypted immediately on save and are never displayed in plaintext.

Platform secrets are encrypted using Bitnami Sealed Secrets and distributed to applications via the External Secrets Operator (ESO). No secrets are stored in plaintext on disk or in Git.

How platform secrets work

  1. All sensitive configuration values (passwords, API tokens, TLS keys) are encrypted into SealedSecret manifests and stored in the values Git repository.
  2. The sealed-secrets controller running in the cluster decrypts them into standard Kubernetes Secrets in the apl-secrets namespace.
  3. ESO reads from apl-secrets via a ClusterSecretStore and distributes the secrets to each application's namespace using ExternalSecret resources.

This means the values Git repository contains only encrypted manifests, never plaintext secrets.

Secret lifecycle

On installation

During the bootstrap phase the platform:

  1. Generates an RSA-4096 key pair for the sealed-secrets controller.
  2. Encrypts all sensitive configuration values into SealedSecret manifests.
  3. Writes the encrypted manifests to the values repository.
  4. Deploys the sealed-secrets controller, which decrypts the manifests into Kubernetes Secrets in the apl-secrets namespace.
  5. Deploys ESO and the ClusterSecretStore, which distributes secrets to application namespaces via ExternalSecret resources.

On runtime

The operator polls the values Git repository for changes. When a change is detected:

  1. Helmfile applies updated ExternalSecret resources.
  2. ESO syncs the corresponding Kubernetes Secrets to each application's namespace.
  3. ESO refreshes all secrets on a hardcoded one-hour interval. Secret updates take up to one hour to propagate to application namespaces.

The sealed secrets key pair

The RSA key pair used to encrypt and decrypt all platform secrets is stored as a Kubernetes TLS secret named sealed-secrets-key in the sealed-secrets namespace. The sealed-secrets controller uses this key to decrypt SealedSecret manifests on the cluster.

📘

If this key pair is lost and cannot be recovered, all SealedSecret manifests become undecryptable. Back up the key pair immediately after installation and store it securely outside the cluster.

Back up the key pair

Use the following command to copy key pair from the kubernetes cluster:

kubectl get secret -n sealed-secrets -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > keys.yaml

Git credentials storage and access

Platform components (apl-operator and otomi-api) access Git credentials through a dedicated Kubernetes Secret: apl-secrets/apl-git-config. Unlike other secrets managed by Sealed Secrets, Git credentials require a special storage pattern because they are a prerequisite to accessing the GitOps values repository and cannot be stored within it.

Why direct secret access?

Git credentials follow a different pattern from application secrets:

  1. Not rendered into environment variables: Credentials are read directly from the Secret via the Kubernetes API, avoiding the need to render them in Helm chart templates
  2. Runtime credential reload: Components read the Secret on each reconcile cycle, allowing credentials to be updated at runtime without pod restarts
  3. Simplified rotation: When Git credentials change, updates take effect on the next reconcile operation — no need to restart pods or redeploy manifests

Accessing Git credentials

To view the Git clone command with embedded credentials:

kubectl get secret -n apl-secrets apl-git-config -ojsonpath='{.data.gitCloneCmd}' | base64 --decode

The apl-git-config Secret contains:

  • gitCloneCmd: Pre-formatted git clone command with embedded credentials
  • username: Git username
  • password: Git personal access token or password
  • Additional fields used by components for Git operations

Updating Git credentials

When Git credentials need to be rotated or changed use one of the following:

  • PUT /v2/git endpoint
  • GitOps in the platform setting of the platform web interface.