Backups program is FUBAR

Hi there,

The Zorin OS 15 Ultimate Backups program does not work. It fails trying to use "duplicity".

The backup starts when I press "Back Up Now". It scans files from the target directory, then
asks for an encryption password (subsequent attempts do NOT remember the password) or not,
then says "Creating backup, this may take a while", lists the files then fails. This happens whether
I use encryption password or not:

Code:

Failed with an unknown error.
  File "/usr/bin/duplicity", line 1555, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1541, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1393, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1511, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 572, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 454, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 453, in <lambda>
    vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 342, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)

The duplicity prog is installed and runs from the command line, so what gives?

I'm not enamored of this backups program, seems like it should work right out of the "Ultimate" box.

I'd prefer one that uses rsync and hard-links. What do others use?

-Kevin

Swarfendor recommends "Redobackup."
I do not use a back up. I just save a few files to PCloud on occasion. Settings that are saved in .local, .config, pictures, music, pdf's (A metric ton of pdfs...). Anything else can pretty much get lost.
Given fifteen minutes after a re-install, it's all pretty much back to the way it was before I went nuclear. Maybe a bit cleaner...
I made a list on a notepad which also gets saved on Pcloud of all the programs I need to remember to install.

I have always been dubious about deja dup and never sure how it worked. Basically, if I am doing a fresh install I backup my /home folder to an external drive whilst NOT connected to the internet - cybercriminals now have the capacity to identify external drives and lock it for ransomware - you have been warned! If I were to do a full system backup then I would use redobackup but there are other free alternatives such as Easeus and another one - all in the Zorin 15 Unofficial Manual. In Windoze I always used Acronis - not had much luck with it for GNU/Linux - tried to restore an old Ultimate Edition 2.9 install from a while back from a different rig and got nowhere - surprising as Acronis uses GNU/Linux coding for its software - how do I know? Well in the earlier versions of Acronis the registry keys would suddently get a new resident - cygwin - this is the equivalent (of sorts) of Linux for Windows (reverse of WINE). Redobackup is simple - not complex, can't shrink to a smaller size (it does not do proportional Partition sizing to a smaller drive - has to match same size or be bigger) but it does what it says on the tin - no development beyond 2012 but it still works.

Thanks for the suggestion, I'll try out "redobackup". I am trying out "luckybackup" since
it is available via "synaptic". Once I was able to run it from root prompt by first correctly
setting the DISPLAY env var:

Code:

export DISPLAY=:1


the rsync errors disappeared and all ownership permissions are now correct and remote
backups are working.

When I need to restore a file/dir, I like being able to see the files, the permissions and
have multiple daily copies. Now I can add the luckybackup command to root's crontab
and we're done.

BTW, this is how I set up new servers or a new OS.

1. Install OS without setting up wired/wireless network.
2. Immediately run: "ufw enable" disabling access by anyone.
3. Setup and start network.
4. Install openssh-server and openssh-client if not already done.
5. Run: ssh-keygen -t rsa -b 4096 as the user with which you connect.
6. Use ssh to login to whatever remote server you are using. The password
is in the clear, but inside the SSH tunnel, so it is not visible to snoopers.
7. Copy the remote server's id_rsa.pub to the local ~/.ssh/authorized_keys
and you can now login back and forth without using passwords.
8. Edit /etc/ssh/sshd_config and change:

Code:

#PasswordAuthentication yes
to
PasswordAuthentication no


This takes out the brute-force password guessing capability
by bad actors (and even those not named "Nicolas Cage").

9. Now run apt-get update && apt-get upgrade
10. Copy /etc/ufw/user.rules from remote to local, modifying for your
new system. If you didn't already have a good copy of user.rules, then
ya gotta make one. I included part of mine as an attachment.

If there's a hole here, I'd love to hear what it might be.

-Kevin

Nice! and thanks for sharing. Would greatly appreciate your contribution to the How-to section on this.

I've PM"d Team Zorin suggesting lucky backup should replace deja dup in future updates.

Hope that's helpful, I'd be glad to create a How-to if it is needed.

As for LuckyBackup, when you hit the button to "Cron-It", it creates a similar line in crontab to this:

Code:

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ luckybackup entries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0 2 * * *   env DISPLAY=:0 /usr/bin/luckybackup --no-questions --skip-critical /root/.luckyBackup/profiles/krieger.local.profile > /root/.luckyBackup/logs/krieger.local-LastCronLog.log 2>&1
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ end of luckybackup entries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

which won't work, cuz DISPLAY is incorrect. I couldn't even find a way to use DISPLAY=:1 . But you can turn on the "Console"
setting and it uses the -c switch instead and all then works swimmingly. I am using ssh and id_rsa private file.

Code:

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ luckybackup entries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0 2 * * *   /usr/bin/luckybackup -c --no-questions --skip-critical /root/.luckyBackup/profiles/krieger.local.profile > /root/.luckyBackup/logs/krieger.local-LastCronLog.log 2>&1
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ end of luckybackup entries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for letting us know about the error with the Backups app (Deja Dup) and duplicity.

After investigating the issue, it appears that it has been fixed upstream in duplicity and in Ubuntu 19.04, but Ubuntu haven't published the new duplicity package in Ubuntu 18.04. We have built a backported package of the new version for testing, which you can install by following these steps:
Install the latest updates from the Software Updater
Run the following command in the Terminal to install the new dependencies: sudo apt install librsync1 python-fasteners python-lockfile python-monotonic
Download this .deb package and double-click to install it: https://launchpad.net/~zorinos/+archive ... _amd64.deb
Reboot your computer to make sure all changer have been applied

We would be grateful if you could test to see if this works now and if there were any regressions with the new package. If everything appears fine, we'll publish the update to every Zorin OS 15 user.

Well it works. The initial problem is that pressing "Backups" icon runs as user, not root. This should absolutely ask
for permission to run as root or whatever so that it will backup ALL files and permissions. Since it won't, one has
to figure out that it needs to run as "sudo deja-dup" and that isn't obvious.

Then when one runs "sudo deja-dup", it takes 40 seconds for the "Backups" display to appear. No status in the meantime.

OK, onward.

Pressing the "Backup Now" button with /etc as the "File to save" works, but it displays this error in the
terminal:

Code:

** (org.gnome.DejaDup:4240): CRITICAL **: 09:45:24.934: string_contains: assertion 'self != NULL' failed

When the backup finishes, it produces this message:

Code:

(org.gnome.DejaDup:4453): GLib-GIO-WARNING **: 09:49:06.622: unable to send notifications through org.freedesktop.Notifications: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.Notifications was not provided by any .service files


I don't know how serious these warning/errors are, but they are certainly disconcerting.

Now, when I use "Restore", it does let me restore the files, but only the entire backup. When I choose "Restore to a Specific Folder" and choose a location, it restores the entire backup directory and along-side, it creates a "root" directory with a ".cache" sub-directory. I should think that would be created or used within root's home directory instead of creating another directory.

Hope that helps,

I've actually gone ahead and written a Perl script to use rsync and cp -al to create daily folders using hardlinks similarly to rsnapshot, just not as bloody confusing with 24 "hourly" directories. I had been using rsnapshot until I needed some backups and they were not complete. My script allows me to peruse the backups without using gunzip and tar.

-Kevin