039 Automated backup system in JCB

Jump to bottom Edit New page

Amigo edited this page Jul 31, 2019 · 21 revisions

AUTOMATED BACKUP SYSTEM IN JCB

Extension API - Backup Feature

This is a demonstration of the new automated backup feature that has been added to JCB. It is part of an extension which is called the API which gives the option of querying JCB via a URL, to perform certain functions. First would be to generate a backup. We are not exactly sure what kind of features should be added to this API. A discussion about this had been opened on GitHub.00:00:42 For more info: My email address is behind this link(See video). Check the features that we want to add to the API. Currently the first one is the backup feature.

Button Called Backup

JCB already has in its component area an export component feature, which we extended to automate. It has been left as a button called 'Backup'. It is the same feature except it can manually be triggered. It can either be triggered by a 'CronJob' or being triggered by Backup button.00:01:36 What the backup feature does, it takes all your components and export them, encrypt them and store them on a local folder, and then emails the key to a trusted email address which you have set up. 00:01:58 The components that are in trash will not be part of this Backup. Only components that are published or unpublished or archived, will be in the Backup, everything else will be ignored. Basically when the Backup runs either manually or automated via a 'CronJob', 00:02:26 it will take those components, use the specific keys as it would with the exporting of that components, then encrypt and store them in that folder.

New Features - Mail Configuration, DKIM, Encryption Settings, CronJob Tabs Added

There are some things that need to be set up for all this to function as expected. First of these are in the options area of JCB. There are some new features. Open that. There is a 'Mail Configuration', 'DKIM' and a 'CronJob' tab added and also this field called API user.

API User

The API user will primarily be used for the ID that is used in the permission structure. When you are in the components area and trigger this feature manually, it uses this current 'Login user', to determine whether the components has been encrypted and backed up, whether he has the permission to do that. 00:03:44 As the permission structure of JCB is expanded, you might end up having components, that certain individuals who are in certain groups, may not have access to. There might be components and fields and views that they may have access to or some that may not have access to. We are laying some of the foundation to make sure that there is not some loophole where either by triggering a batch update or an export of components or backup. 00:04:17 They can not extrude certain components which they by default do not have access to. This is just to explain why an API user is still needed. A user is still necessary, which; if it has automated the backup, that this user ID allows him to make the backup. So that means the API user should be a user which has the permission to all the components fields and everything in the JCB component. 00:04:52 It is secure because the component gets compiled and stored locally. We might consider adding the option to push the component back up to an FTP server, but that part is not fully functional yet.00:05:14 Proposals have been made to do that, so that the backup should not be on the same system, but that it is secured on another system in case of a collapse. 00:05:41 A FTP option will be added as well, which will work similar as your components, when a component is compiled it gets send off to a sale server. That is the same features that we will make use of.

CronJob Tab

In the 'CronJob' tab, is some basic instructions for those who are familiar with CronJobs and if not, Google around, find some tutorials, read up, make sure that you are able to set up a CronJob. 00:06:40 There are various ways to trigger this URL. The system currently detects whether your system can run 'curl' request and if not, it will fall back on the 'weget' option. Assuming that JCB is installed in an environment that allows these two functions to work and if not, simply take the URL, and run it in a CronJob in a manner that is applicable to your system. 00:07:16 This URL triggers the backup to start. When the backup is finished, it gives the same message that you will get if you run the backup by clicking 'Backup' (See video). There is a green message that pops up, indicates: 'Yes the backup was done and the email was sent'. 00:07:42 Those are the only messages that gets returned on this URL. Currently that response gets thrown away. It may be added to a log, depending on how regularly this 'CronJobs' is running. Currently there is not any date. 00:08:06 It might make sense for us to add a date. In case you run into a firewall, Joomla website has a firewall installed like RS firewall, then it is necessary to adapt the 'curl' request to behave like a real browser. This post on https://stackoverflow.com may be of some help.

Cronjob Backup Folder Path

Currently the 'Cronjob Backup Folder Path' is set here(See video). It currently backups to a local folder. As mentioned previously an FTP option is going to be add here. Joomla does not by default encrypt fields in the Global Configuration of the component.

Email (Backup Key)

Currently it stores it into a local folder, and it Emails the backup key to the email address that have been set in here(See Video). Make sure that email address is secure and that it is safe because the keys are being sent to it.

Package Name Placeholders

Here is a naming structure for your file by default. It adds only up till the day, so it will make backups all through the day, but overwrite the file every time unless you add an hour, then it will only per hour overwrite it. If a minute is added , it will never overwrite it, because it will be a different minute every time. That is to rename the package and to see how much of the backups you want to keep.

Mail Configuration Area

In the mail configuration area, it may be seen that it is currently set to use the Global Mail Configuration which is the Joomla default. NB. Use a very secure method of sending the emails. Either SMTP overwriting the SMTP settings. If the Global ones are used, then overwrite or insure that the Global Email settings in Joomla is also using SMTP that runs through an SSL, and that it is secure.

DKIM

The DKIM is also a added feature which can increase trust and security of your emails.

Company Details

Then there is the Company details, which at this stage is important to add, since it will also become part of the backup package.<<<<<<<<<<

We will look at that again when we restore a backup to show you where this information comes up. 00:11:28 That is some of the settings you need to first set within your Global Configuration. Now once you've set those settings, Company settings, your CronJob settings, your local path, the email, the name, and your API user, you can save and close this area and we can start looking at a component to 00:11:54 see this feature in action. Now since this is a default JCB install, there's only one component. It's the demo component.

Set An Export Key

Now usually the demo component doesn't have an encryption key. I'll open it, and 00:12:18 I've set it. You come and set an export key in at least one of the components that will be part of the backup. If any one of the components that are being backup have an export key, it will encrypt all the components with that export key. If multiple one's have an export key, it will combine those keys 00:12:47 to use as the encryption key. It hashes those keys so the actual key is not what is actually being used. The actual key that's being used is the one that will be emailed out to that trusted email address. We'll save and close the demo component.

Backup Manually

We will just run a manual backup, which I explained before, is similar to the CronJob except that it's triggered by clicking the Backup button. 00:13:18 It's one component, it's not a very big process, it should be finished quite quickly. It will say: The backup has been done successfully. 00:13:36 The owner details was set. If for some reason you forgot to set the owner details in the options area, it will tell you that here. The email with the new key was sent. To make sure that your backup works, you should go check the folder in which you set the backup should be placed. See whether the backup is there, and also 00:14:06 check the email address and see whether that they receive the key. I see I have a backup here in the folder as expected. The backups been done, I can double click into the backup, and make sure that it has all the expected files. At this stage it looks like it does. If you have a lot of custom files 00:14:38 that you've added into the component, there will also be an extra custom folder in the Zip Document and not only in image one. We will test this backup by importing it. But first let's see if we received the email. We can see that it send the email as expected, with the corresponding key. We'll be using this key to test the backup. 00:15:08 Be sure like I said that it sends the key to a secure Website Email Address, and that the email that sends the Email from Joomla is using a secure SSL by SMPT.

Testing Backup

Now let's test this backup. 00:15:35 A backup will only makes sense if we lost our data. I'm going to take the demo component, and throw that in trash. Then go to trash and empty the trash. There is no components. 00:16:10 Click on import components. Now we need to browse to the Directory, where that component was backed up. We'll click on the Directory tab. There we have the components path to the backup package. We click on 'Get File'. Now we will take that key that was sent to us via the email, and we will force the update, and will add that key. 00:16:45 We'll click continue.

Package Owner Details

I just step back for a moment because I realized I didn't explain what you'd seeing here. Like I said before your Package Data of the owneris desplayed here, and the Package Details is being displayed here. If you have more than one component backed up, all of those components will show here on the 'Components Being Imported'. Here you will see the the Package Owner 00:17:17 Details, and get the key from this link here(Testing Company).

I will again do this import. Let's click continue. Wonderful so we have the component back. It's all being restored. That is what the purposes of the backup is, is when something goes wrong 00:17:40 you can come back to where it was when the backup was made. We can test with a backup if it is been successfully restored by going to the compiler, and then selecting the component. Clicking on compile. As you can see is been completely successfully built. We can now click on install the demo component, and then we can go to the demo component, and see that everything is working. 00:18:18 Adding a new look. There is all the various fields as it usually are available in the demo component. That is automated settings for JCB, and setting them up and you making use of them.