Disaster recovery
This page covers a few potential scenarios in which a complete or partial restore of the platform is required.
Prerequisites
To ensure you are able to perform the necessary disaster recovery steps when needed, regularly review the prerequisites and limitations listed below.
-
The following items should be backed up regularly by the platform administrator:
-
The Kubernetes secret ending in "-wildcard-cert" in namespace "istio-system" (if installed via the Linode cloud console, or using your own certificate).
-
The
sealed-secrets-keyKubernetes secret in namespacesealed-secrets. This contains the RSA key pair required to decrypt all platform secrets. Export the sealed-secrets key pair before decommissioning the cluster:kubectl get secrets -n sealed-secrets \ -l sealedsecrets.bitnami.com/sealed-secrets-key=active \ -o yaml > sealed-secrets-key.yaml -
Optionally manual backups of databases, as covered in this guide for the CloudNative PostgreSQL Operator, should be taken.
-
-
Object storage needs to be set up for all backup types referred to. Credentials should be added to Platform Settings -> Object Storage.
-
All backup types should be activated in the Platform Settings -> Backup.
-
This guide does not cover the partial or complete loss of attached object storage. For production environments, it is advised to set up additional object storage in a different region. Then, mirror all contents of your platform's object storage so that it can be retrieved in the event of accidental deletion, data center availability issues, or other issues. The process for mirroring your data to a secondary remote storage location is not covered in this guide.
-
Workloads may store data in local storage, object storage, different types of databases, message queues, or in other ways. The backup and recovery strategy of workload storage is not covered here.
-
Currently it is not supported to reinstall a cluster in-place that has been provisioned directly using the Linode API or Console. Such an LKE cluster must instead be reprovisioned with the application platform through a Helm install. However, since the cluster ID changes, the domain suffix will also change. Adjustments need to be made to the values file before the restore. Also, you will need a domain name set up with a DNS provider supported by App Platform and the credentials should be added to the values file.
-
All instructions assume you are familiar with essential Kubernetes tools such as
kubectland have access to the Kubernetes API. Usage of TUI applications such ask9sfrom the administration terminal is strongly recommended.
Guides
- Basic disaster recovery scenario: Restoring the platform declared state from an existing git repository.
- Gitea repositories and database: Restoring the platform's Gitea database and repositories from the application backup.
- Reinstalling the platform: Restoring the complete platform, including settings and data.
- Restoring platform databases: Backup and restore of the CNPG databases.
Updated 19 days ago
