OK, you read from my previous post that I’ve tooled up some things in public cloud (specifically Google GCP & GKE). Now, I’m sure they have a strong track record of backup/restore/disaster recovery. But what if… something goes wrong. Maybe I make a mistake and delete the project, my credit card gets stolen and they lock me out, whatever. How would I keep a disaster recovery copy of my data?
I mulled over various approaches, looked at some of the things which use e.g. AWS/EBS to ‘push’.
So here is what I came up with. Its hybrid Cloud Native (Kubernetes) and ‘Old School’ (rsync). And it works quite well.
So what I did is create 1 (or more) ‘backup’ PersistentVolumes. And then each application (Git, Taiga, …) does a backup to this (they mount a subPath, so e.g. /var/backups/git, /var/backups/taiga, etc). They do this in their native way (psql dump, tar of repo, etc) so its not strictly a disk copy (postgresql doesn’t work well if you just tar it up).
And, I’ve created a container that mounts this read-only, and in turn exposes a restricted rsync via ssh. I launch this like so (below). I add port 2222 into my tcp: configmap on my ingress.
Now I can rsync (via ssh) to port 2222 and efficiently mirror this backup volume offline. That runs as a cron job on the vault that lives in a secure location not to be confused with my basement.
That container (you can see my source at the link) creates a user with a authorized_keys file as:
command="/usr/bin/rrsync -ro /sync/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-ed25519 XXX...
What do you think? Yay or Nay?
apiVersion: apps/v1beta2 kind: Deployment metadata: name: corp-backup labels: app: corp-backup spec: replicas: 1 selector: matchLabels: app: corp-backup strategy: type: Recreate template: metadata: labels: app: corp-backup spec: imagePullSecrets: - name: regcred containers: - name: backup image: cr.agilicus.com/corp-tools/rsync-container imagePullPolicy: Always env: - name: SSH_PUBKEY value: "ssh-ed25519 XXXmy-ed25519-pubkey" - name: SSHD_PORT value: "2222" ports: - name: ssh containerPort: 2222 volumeMounts: - name: sync mountPath: /sync readOnly: true volumes: - name: sync persistentVolumeClaim: claimName: pv-backup-claim --- apiVersion: v1 kind: Service metadata: name: corp-backup labels: app: corp-backup spec: ports: - port: 2222 targetPort: 2222 name: ssh selector: app: corp-backup