Kubernetes backups with Velero

Talk about backup in a Kubernetes cluster may sound weird and you may thing is not necessary as you can recreate at any time and in a very quick way any of your deployments or resources simply aplying a yaml file… but in some cases a backup of your resources can be very useful and can be our life guard.

Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.

That said, let’s enumerate in which situations Velero can help us:

• Backup stateful applications
• Backup applications installed in a non-declarative way
• Backup PVC information
• Cluster migrations
• Replicate cluster configurations (for example, from production to testing or development clusters)

After the boring introduction we will start with the funny part. 🙂

Install Velero on your kubernetes cluster

For this example we will use AWS EKS as our cloud provider.

First of all, you must install the Velero client on your workstation. To to do that, simply go here and download the latest tarball according to your workstation operating system and extract the tarball:

tar -xvf RELEASE-TARBALL-NAME.tar.gz

and move the Velero binary to somewhere in your $PATH. On MacOS simply execute this:

brew install velero

Once we have the Velero client installed on our system we can proceed to install Velero on our kubernetes cluster. Just execute this:

velero install 
    --provider aws 
    --bucket where-to-store-velero-backups 
    --use-restic 
    --secret-file ~/.aws/credentials 
    --use-volume-snapshots=false 
    --plugins=velero/velero-plugin-for-aws:v1.0.0 
    --backup-location-config region=eu-west-1

By default Velero is installed in the velero namespace but you can change it adding the –namespace flag. Velero uses restic to store the data of attached volumes. When the installation finishes you should see something like this in your k8s cluster:

$ kubectl get all -n velero
NAME                          READY   STATUS    RESTARTS   AGE
pod/restic-4xr6v              1/1     Running   0          157d
pod/restic-bg58w              1/1     Running   0          157d
pod/restic-hf2fm              1/1     Running   0          157d
pod/restic-w4rvh              1/1     Running   0          157d
pod/velero-57cd659988-bd6nd   1/1     Running   0          73d

NAME                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/restic   4         4         4       4            4                     157d

NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/velero   1/1     1            1           157d

NAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/velero-57cd659988   1         1         1       157d

In our case, we have 4 worker nodes and restic is installed on every node we have (configures a daemonset).

By default, Velero will not include the volumes in the backup. To backup them you have to add an annotation. You can add it like this:

$ kubectl -n annotate pod/your_pod backup.velero.io/backup-volumes=volume_1,volume_2

(add volumes comma separated) or simply add it in your deployment definition, for example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
      annotations:
        backup.velero.io/backup-volumes: myapp-logs
    spec:
      volumes:
        - name: myapp-logs
          persistentVolumeClaim:
           claimName: myapp-logs
      containers:
      - image: myapp:latest
        name: myapp
        ports:
        - containerPort: 80
        volumeMounts:
          - mountPath: "/var/log/myapp"
            name: myapp-logs
            readOnly: false

 Note: hostPath volumes are not supported, but the new local volume type is supported.

And finally, some utilization examples to create, restore and show backups:

One time backup:

# Backup ALL resources in the cluster (the whole cluster)
$ velero backup create my-backup-20200515

# Backup a namespace
$ velero backup create my-backup-20200515 --include-namespaces namespace_to_backup

# Backup ALL namespaces except ones specified
$ velero backup create my-backup-20200515 --exclude-namespaces namespace_1_to_exclude,namespace_2_to_exclude

Scheduled backup:

$ velero create schedule myapp-backup-daily --schedule="0 7 * * *" --include-namespaces namespace_to_backup

Velero will backup all resources included in your selection (pods, deployments, services, …)

The default backup retention is 30 days. If you want to change it add the –ttl flag. This flag allows you to specify the backup retention period with the value specified in hours, minutes and seconds in the form –ttl 24h0m0s. If not specified, a default TTL value of 30 days will be applied.

Restore a backup:

$ velero restore create --from-backup backup_name

Some other useful commands:

# To show all stored backups list (name, status, creation and expiration date)
$ velero get backups

# To show one specific backup details
$ velero describe backup backup_name

Velero is quite easy to install and run and it can save your life in some circumstances. 😉

I hope you’ve enjoyed this post and I encourage you to check our blog for other posts that you might find helpful. Do not hesitate to contact us if you would like us to help you with your projects.

See you in the next post!