Centralized Rsync backup monitoring with Go-lang and VueJS

Centralized Rsync backup monitoring with Go-lang and VueJS
Image copyright: Marco Griep (CC BY-NC-ND) - Written from: Marco Griep

Zentrale Rsync-Backup-Überwachung mit Go-lang- und VueJS

Rsync is a robust and reliable solution for Unix systems to copy data quickly and reliably to other devices (e.g. NAS systems). I’ve been using Rsync for years for my data backup from my Ubuntu server to a NAS system from QNAP, which in turn replicates my data to an external hard drive that ends up offline in the closet. Rsync does this reliably every day with no problems and starts with a cronjob. The log files log everything down to the last detail - so far, no explicit backup software has been necessary for me.


Rsync Logfile Entry

The problem with multiple devices

Over the last few months I have done a massive rebuild of my home network. A Docker host and 3 Raspberry Pi’s have been added. Both the Docker host and one of the three Rasperries want to be backed up. This can be easily done with Rsync, of course. However, it is cumbersome to monitor if the backup is working reliably. I could periodically check for devices with SSH connections and log files. Alternatively, I could include the log files in a shared order and check. However, I would prefer central monitoring of backups without having to read log files.

Current structure of my data backup

DIY data backup with Rsync

True to the motto “why make it simple when it gets complicated”, I added centralized monitoring to my backup process. My thought was to program a simple web API using Go, to which the most important information of the backup is sent as soon as the backup is completed. Analyzing the log files makes me a program that I wrote and compiled with Go. The Go program sends the hostname, job name (passed via parameters), backup timestamp, and number of files and new data to the web API. A simple one-page web application (programmed with VueJS) then shows me my backup information. I integrate this web interface as an external page in Nextcloud and therefore have everything at a glance.

First draft

Rsync Backup Monitor

Install RSync Monitor

The full code of my solution is open source and available on GitLab. The easiest way is to use the ready Docker images to install the application yourself. How to configure the application correctly is in the readme file in the repository.

Instructions in English can be found in the readme file on my GitLab repository

Information

This software thrives on community support. I think this is helpful or if you want me to improve this application, please tag this application on github or better yet, become a sponsor on Patreon or PayPal