Backups

[Flohack74]

While thinking what to do for a better backup (my magic-device-tool-backup is not the best option so far) I thought that maybe the installer could be extended into a multi-purpose-tool for install, backup and update.

We see that Android for example has much better update support for the firmware, since most vendors have kind of a notifier app to remind users of updates, and then they also package firmware updates with normal OS updates. If we fall behind with firmware, our users might face issues that could easily be solved, especially in the radio part. Its also the only way to flash a firmware update for users that are afraid to use fastboot or cannot do it due to their skills limitation.

The same goes for the backup. I would throw in my ideas about the backup reqirements, but unfortunately I am no good node dev, so I cant implement it ;)

[NeoTheThird]

Yep, these are features that I wanted to work on as well, but not before some of the refactoring in #58 is done. The code complexity has to decrease significantly before we should think about adding new features.

[NeoTheThird]

So, i thought a little about how to best do this. The simplest way is probably to tar the home folder on the device and pull it using adb. That file can be stored either in the installer cache dir, or saved in an arbitrary location on the pc. To restore, push the file to the device and untar it over the home folder. Here's the adb commands:

# Backup
adb shell rm /tmp/backup.tar.gz || echo ""
adb shell tar -cvpzf /tmp/backup.tar.gz --exclude=/userdata/user-data/phablet/.cache/upstart --exclude=/userdata/user-data/phablet/.cache/*/qmlcache --exclude=/userdata/user-data/phablet/.cache/*/qml_cache /userdata/user-data/phablet
adb pull /tmp/backup.tar.gz ./backup.tar.gz
# Restore
adb push ./backup.tar.gz /tmp/backup.tar.gz
adb shell tar -xvpzf /tmp/backup.tar.gz -C / --numeric-owner

[NeoTheThird]

And, for the record, firmware upgrades are already happening.

[Flohack74]

I agree except that it could run low on diskspace if we are packing on the device, maybe we can redirct the gzip stream directly to the installer?

[NeoTheThird]

I agree except that it could run low on diskspace if we are packing on the device, maybe we can redirct the gzip stream directly to the installer?

I'd rather not do that. Those kinds of tasks over adb aren't very reliable and can create corrupted files. You wouldn't want your backup fail silently... If someone has very little free storage, they'll just have to exclude some more files. There's always some things you don't need to back up, like music for example.

[NeoTheThird]

Update: Someone started working on this on the forums.

[Flohack74]

This is still missing the system-data equivalent, which might be not even readable to the phablet user. As a user I want my WiFi and other settings also backed up. the home directory does not contain this.

[NeoTheThird]

This is still missing the system-data equivalent, which might be not even readable to the phablet user. As a user I want my WiFi and other settings also backed up. the home directory does not contain this.

Are you sure, did you test it? Pretty sure that is in /userdata/user-data/phablet (not in /home/phablet, mind you!) as well. But we need to confirm that.

[Flohack74]

Ok will take a look...

[Alain94W]

any results ?

[Flohack74]

• /userdata/user-data/phablet is the usual home-dir
• /userdata/system-data is indeed the system-data folder. So for backup in the running system you can take those 2 (/userdata/user-data without phablet in case in the future there would be more than 1 user)
  But: This is readonly - For a restore you still would need recovery mode, and pushing via ADB atm. Or, we find a way to have a recovery with ssh+rsync e.g.

[Alain94W]

Hi Guys, so what do we do ? I nearly finished the backup and restore feature with remote file transfer (let's call the stream through adb to the computer like that).

If the device get disconnected during the backup, the backup will fail and display an error message to the screen. I have not yet tested the disconnection during restore but we can say that it will leave the device in an inconsistent state and the user will have to reinstall the OS, then try again the restore. I check the device free space before to start the remote restore and if the device doesn't have enough space, the restore will not be possible.
I did it because for example, I have a Pro5 of 64Go and another one at 32Go of internal storage, It's safer to check if we have enough space before starting the restore possess.

If we go with the backup made on the internal flash then fetched via a adb pull, I have to change the code for both backup and restore functions and, also the graphical UI to allow the user to select what he would like to include in he's backup and restore (pictures, music, documents).

[NeoTheThird]

@Alain94W It's fine, you can go forward with it. We'll test it properly and put a note before it saying it's still experimental. If you push your latest changes, i'll review them and we can move forward :)

[Alain94W]

OK thanks ! Am I already banned from the telegram group ?

[NeoTheThird]

?

[Alain94W]

Question again, Is it possible to restore some system data and user data from a old OTA version to a new one ? I mean for example restoring a backup made on OTA-5 to OTA-7 ? is there a risk ? if yes I hace to check this before to restore

[Flohack74]

I dont think there is a risk: It is mostly system settings that could be influenced by that, and so we must make sure that an OTA has migrations of settings where needed. But until now we did not have this function, so I cannot 100% be sure. I would slice this out still. Do it without check, do not try to solve all issues with the first iteration of the function.

[NeoTheThird]

I don't think it'll be a problem. As of today, you can migrate from canonical UT to Ubuntu Touch OTA-11 and keep your data without any issues. In any case, Florian is right, don't worry about it now.

[Alain94W]

Ok, I sarted again to work on it today, I'll push the sources as soon as possible.

[Alain94W]

I am stuck on one point, I don't know how to monitor with a progress bar when I restore the backup. I could not stat the folder as it is already existing with nearly the same used space as the backup space.

Any idea on how to monitor that ?