Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added backup functions, device status (recovery,device,fastboot), fol… #28

Closed
wants to merge 4 commits into from

Conversation

Alain94W
Copy link
Contributor

@Alain94W Alain94W commented Dec 8, 2019

Added backup function, devicestatus (recovery,device,bootloader) info, floder or partition size info, customecommand execution (not only adb for the backup). Started to write the restore function but not tested yet, working on the GUI first, do not use the restore function now.

Copy link
Member

@NeoTheThird NeoTheThird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution! First things first, please use the fs module instead of calling stat. Then, run npm run lint-fix to enforce the code style. Then we'll see about the rest :)

src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
@NeoTheThird
Copy link
Member

You still need to push your changes, afaics

@Alain94W
Copy link
Contributor Author

Alain94W commented Dec 9, 2019

You still need to push your changes, afaics

Yes know ! I would like to change some things regarding the backup function first !

@Alain94W
Copy link
Contributor Author

Alain94W commented Dec 9, 2019

Thank you for your contribution! First things first, please use the fs module instead of calling stat. Then, run npm run lint-fix to enforce the code style. Then we'll see about the rest :)

I have issue with lint-fiw :

`> eslint tests/unit-tests/.js src/.js "--fix"

sh: 1: eslint: not found
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! [email protected] lint: eslint tests/unit-tests/*.js src/*.js "--fix"
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the [email protected] lint script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?

npm ERR! A complete log of this run can be found in:
npm ERR! /home/alain/.npm/_logs/2019-12-09T16_44_05_540Z-debug.log
`

Do you have a hint for me ?

@Alain94W
Copy link
Contributor Author

Alain94W commented Dec 9, 2019

Forget it, Some packages needs to be insalled, I fixed it

@Alain94W
Copy link
Contributor Author

Alain94W commented Dec 9, 2019

@NeoTheThird Can you see the new commit ?

Copy link
Member

@NeoTheThird NeoTheThird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some proposed tweaks and fixes.

Before merging, we also need to add some unit-tests, but let's think about this one first: Are you sure you want to go with the pipes approach? We had previously thought to tar on the device itself to get around buffer limitations and general adb flakyness. That way you could be a lot more certain that your backup would not be corrupted. What do you think?

src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
src/adb.js Outdated Show resolved Hide resolved
@Flohack74
Copy link
Member

Just my 2 cents, tar on the device puts quite a load on the flash. How is Android doing that? We do not want to wear out the sd card by doing too many backup cycles on it.

@Flohack74
Copy link
Member

In the end, we could also have a recovery with ssh or rsync, who says it must be adb only.

@Alain94W
Copy link
Contributor Author

Alain94W commented Dec 9, 2019

In the end, we could also have a recovery with ssh or rsync, who says it must be adb only.

Yes it can be also done via ssh,rsync but this mean that we need to enable the developers mode on the device right ?

In fact I planned to write 2 backup function, and 2 restore functions, one "insitue" where the backup will take place on the device, the other one is the one I already wrote.

@Flohack74
Copy link
Member

Let it be in recovery for the time being. The folders are not accessible in an easy way during normal boot mode.

@NeoTheThird
Copy link
Member

Just my 2 cents, tar on the device puts quite a load on the flash. How is Android doing that? We do not want to wear out the sd card by doing too many backup cycles on it.

That's not really that big of a problem. A tradeoff you might be willing to accept for the certainty that your backup does not fail silently.

Let it be in recovery for the time being. The folders are not accessible in an easy way during normal boot mode.

No? The method described in ubports/ubports-installer#141 would work during normal boot as well.

@Alain94W
Copy link
Contributor Author

Just my 2 cents, tar on the device puts quite a load on the flash. How is Android doing that? We do not want to wear out the sd card by doing too many backup cycles on it.

That's not really that big of a problem. A tradeoff you might be willing to accept for the certainty that your backup does not fail silently.

Let it be in recovery for the time being. The folders are not accessible in an easy way during normal boot mode.

No? The method described in ubports/ubports-installer#141 would work during normal boot as well.

If we did it on normal boot, what will happens if we receive a SMS or a telegram during the backup task (even if we copy locally on the phone) ? I have nothing against the fact of doing it like you want, just one things is what will happens on the OS if it run out of space because of the tar operation ?

@NeoTheThird
Copy link
Member

If we did it on normal boot, what will happens if we receive a SMS or a telegram during the backup task (even if we copy locally on the phone) ?

It's not uncommon to back up linux desktops using tar, and they don't have a recovery. If you miss one sms, it usually doesn't hurt.

I have nothing against the fact of doing it like you want, just one things is what will happens on the OS if it run out of space because of the tar operation ?

Yep, that's the limitation of that approach. If you don't have enough free space, you can't do it like that.

@Alain94W
Copy link
Contributor Author

If we did it on normal boot, what will happens if we receive a SMS or a telegram during the backup task (even if we copy locally on the phone) ?

It's not uncommon to back up linux desktops using tar, and they don't have a recovery. If you miss one sms, it usually doesn't hurt.

I have nothing against the fact of doing it like you want, just one things is what will happens on the OS if it run out of space because of the tar operation ?

Yep, that's the limitation of that approach. If you don't have enough free space, you can't do it like that.

And what about if we put the 2 possibilities and let the final user to choose.
And if we implement a tar verification at the end of the backup ? like send the tar decompression to dev/null and check the error code returned ?

@Flohack74
Copy link
Member

Could we discuss this further on Telegram or so, and stay here ontopic for the PR ;) - We got still the installer group there eg.g.

@Alain94W
Copy link
Contributor Author

Could we discuss this further on Telegram or so, and stay here ontopic for the PR ;) - We got still the installer group there eg.g.

What is the telegram group name please ?

@Flohack74
Copy link
Member

@Alain94W please send me your telegram name, I will invite you...

@Alain94W
Copy link
Contributor Author

@Alain94W please send me your telegram name, I will invite you...

This is : @Weiszaaaaa

Copy link
Member

@NeoTheThird NeoTheThird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly very nice, mainly just styling concerns. Only one slightly larger thing: Any chance you could add some tests for the new functions? Should be fairly straightforward if you base it on the existing tests.

var _this = this;
return new Promise(function(resolve, reject) {
_this
.shell(["cat", "/etc/system-image/channel.ini", "|grep tag="])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What information do you want here, exactly? The tag is usually in the same line with the rest of the version-detail.

Maybe we could rename this function to avoid confusion with getOs. Maybe getUbuntuTouchVersionDetail.

It would also be better to parse the response in javascript, rather than piping through grep.

});
} else if (stdout == "device") {
// Device
return this.shell("du -shk " + file + " |tail -n1") // + " --output=used|tail -n1")//df -hBK
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment still needed?

srcfile,
" 2>/backup.pipe' | dd of=" + destfile
]);
//_this.execCommand(["exec-out 'tar -cvp ",srcfile," 2>/backup.pipe' | dd of="+destfile,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still needed?

common.stdoutFilter("%]")
])
.then((error, stdout, stderr) => {
//process.platform == "win32" ? ' | findstr /v "%]"' : ' | grep -v "%]"']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Comment on lines +665 to +681
resolve(); // Everything's Fine !
})
.catch(e => {
reject(e + ", Unable to delete the pipe ");
}); // Pipe Deletion
})
.catch(e => {
reject(e + ", Unable to backuping the device ");
}); // Backup start
})
.catch(e => {
reject(e + ", Pipe creation failed ");
}); // Pipe Creation
})
.catch(e => {
reject(e + ", Unable to get the partition's filesize ");
}); // Get file size
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the comments here don't seem very useful

Comment on lines +704 to +725
/*
var lastSize = 0;
_this.fileSize = bckSize;
var progressInterval = setInterval(() => {
_this
.shell([
"stat",
"-t",
common.quotepath("/restore.pipe")
])
.then(stat => {

lastSize = eval(stat.split(" ")[1]);
progress ((lastSize / _this.fileSize)*100);
//lastSize = fileSizeInBytes/1024;
})
.catch(e => {
clearInterval(progressInterval);
_this.log("failed to stat: " + e);
});
}, 1000);
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

*/

// Start the restoration
_this.log("Starting Restore..."); //adb push user.tar /resto.pipe & adb shell 'cd /; cat /resto.pipe | tar -xv'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment still needed?

])
.then((error, stdout, stderr) => {
_this.log("Restore Ended");
//clearInterval(progressInterval);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Comment on lines +745 to +757
resolve(); // Everything's Fine !
})
.catch(e => {
reject(e);
}); // Pipe Deletion
})
.catch(e => {
reject(e);
}); // Backup start
})
.catch(e => {
reject(e);
}); // Pipe Creation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe include comments in error messages instead

@NeoTheThird NeoTheThird mentioned this pull request Oct 22, 2020
@NeoTheThird
Copy link
Member

Work continues in #39. Let's keep this one open until it's resolved for reference.

@NeoTheThird
Copy link
Member

Superseded by #39. Thanks again for your original work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants