Kubernetes backups with Velero
Talk about backup in a Kubernetes cluster may sound weird and you may thing is not necessary as you can recreate at any time and in a very quick way any of your deployments or resources simply aplying a yaml file… but in some cases a backup of your resources can be very useful and can be our life guard.
Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.
That said, let’s enumerate in which situations Velero can help us:
- Backup stateful applications
- Backup applications installed in a non-declarative way
- Backup PVC information
- Cluster migrations
- Replicate cluster configurations (for example, from production to testing or development clusters)
After the boring introduction we will start with the funny part. 🙂
Install Velero on your kubernetes cluster
For this example we will use AWS EKS as our cloud provider.
First of all, you must install the Velero client on your workstation. To to do that, simply go here and download the latest tarball according to your workstation operating system and extract the tarball:
tar -xvf RELEASE-TARBALL-NAME.tar.gz
and move the Velero binary to somewhere in your $PATH. On MacOS simply execute this:
brew install velero
Once we have the Velero client installed on our system we can proceed to install Velero on our kubernetes cluster. Just execute this:
velero install \ --provider aws \ --bucket where-to-store-velero-backups \ --use-restic \ --secret-file ~/.aws/credentials \ --use-volume-snapshots=false \ --plugins=velero/velero-plugin-for-aws:v1.0.0 \ --backup-location-config region=eu-west-1
By default Velero is installed in the velero namespace but you can change it adding the –namespace flag. Velero uses restic to store the data of attached volumes. When the installation finishes you should see something like this in your k8s cluster:
$ kubectl get all -n velero NAME READY STATUS RESTARTS AGE pod/restic-4xr6v 1/1 Running 0 157d pod/restic-bg58w 1/1 Running 0 157d pod/restic-hf2fm 1/1 Running 0 157d pod/restic-w4rvh 1/1 Running 0 157d pod/velero-57cd659988-bd6nd 1/1 Running 0 73d NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/restic 4 4 4 4 4 157d NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/velero 1/1 1 1 157d NAME DESIRED CURRENT READY AGE replicaset.apps/velero-57cd659988 1 1 1 157d
In our case, we have 4 worker nodes and restic is installed on every node we have (configures a daemonset).
By default, Velero will not include the volumes in the backup. To backup them you have to add an annotation. You can add it like this:
$ kubectl -n annotate pod/your_pod backup.velero.io/backup-volumes=volume_1,volume_2
(add volumes comma separated) or simply add it in your deployment definition, for example:
apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deployment spec: replicas: 1 selector: matchLabels: app: myapp template: metadata: labels: app: myapp annotations: backup.velero.io/backup-volumes: myapp-logs spec: volumes: - name: myapp-logs persistentVolumeClaim: claimName: myapp-logs containers: - image: myapp:latest name: myapp ports: - containerPort: 80 volumeMounts: - mountPath: "/var/log/myapp" name: myapp-logs readOnly: false
Note: hostPath volumes are not supported, but the new local volume type is supported.
And finally, some utilization examples to create, restore and show backups:
One time backup:
# Backup ALL resources in the cluster (the whole cluster) $ velero backup create my-backup-20200515 # Backup a namespace $ velero backup create my-backup-20200515 --include-namespaces namespace_to_backup # Backup ALL namespaces except ones specified $ velero backup create my-backup-20200515 --exclude-namespaces namespace_1_to_exclude,namespace_2_to_exclude
$ velero create schedule myapp-backup-daily --schedule="0 7 * * *" --include-namespaces namespace_to_backup
Velero will backup all resources included in your selection (pods, deployments, services, …)
The default backup retention is 30 days. If you want to change it add the –ttl flag. This flag allows you to specify the backup retention period with the value specified in hours, minutes and seconds in the form –ttl 24h0m0s. If not specified, a default TTL value of 30 days will be applied.
Restore a backup:
$ velero restore create --from-backup backup_name
Some other useful commands:
# To show all stored backups list (name, status, creation and expiration date) $ velero get backups # To show one specific backup details $ velero describe backup backup_name
Velero is quite easy to install and run and it can save your life in some circumstances. 😉