039 Automated backup system in JCB

AUTOMATED BACKUP SYSTEM IN JCB

Extension API - Backup Feature

00:00:00

This is a demonstration of the new automated backup feature that has been added to JCB. It is part of an extension which is called the API which gives the option of querying JCB via a URL, to perform certain functions. First would be to generate a backup. We are not exactly sure what kind of features should be added to this API. A discussion about this had been opened on GitHub.00:00:42 For more info: My email address is behind this link(See video). Check the features that we want to add to the API. Currently the first one is the backup feature.

Button Called Backup

00:01:07

JCB already has in its component area an export component feature, which we extended to automate. It has been left as a button called 'Backup'. It is the same feature except it can manually be triggered. It can either be triggered by a 'CronJob' or being triggered by Backup button.00:01:36 What the backup feature does, it takes all your components and export them, encrypt them and store them on a local folder, and then emails the key to a trusted email address which you have set up. 00:01:58 The components that are in trash will not be part of this Backup. Only components that are published or unpublished or archived, will be in the Backup, everything else will be ignored. Basically when the Backup runs either manually or automated via a 'CronJob', 00:02:26 it will take those components, use the specific keys as it would with the exporting of that components, then encrypt and store them in that folder. <<<<<

New Features - Mail Configuration, DKIM, Encryption Settings, CronJob Tabs Added

00:02:40

There are some things that need to be set up for all this to function as expected. First of these are in the options area of JCB. There are some new features. Open that. There is a 'Mail Configuration', 'DKIM' and a 'CronJob' tab added and also this field called API user. <<<<<

API User

Let me start with this field. The API user primarily be used for the id that is used 00:03:20 in the permission structure. When you are in the components area and you trigger this feature manually, it uses this current login user, to determine whether the components being encrypted and backed up, whether he has the permission to do that. 00:03:44 It is a way as we increase the permission structure of JCB. You might end up having components, that certain individuals who are in certain groups, may not have access to them. There might be components and fields and views that they may have access to or some that may not have access to. We're laying some of the foundation to make sure that there isn't some loophole where either by triggering a batch 00:04:17 update or an export of components or backup. They can't extrude certain components which they by default don't have access to. I know that's a long explanation, but it's to give you background why we still need an API user. We still need a user, which if it's automated the backup, that this user ID allows him to make the backup. So that means the API user should be a user 00:04:52 which has the permission to all the components fields and everything in the JCB component. It is secure because like I said the component gets compiled and stored locally. We might 00:05:14 consider adding the option to push the component back up to an FTP server, but that part is not fully functional yet. We have had the proposals made to do that, that the backup is not even on the same system, but that it is secure on another system, so if this system collapses you are still safe 00:05:41 and still got your data out. That sounds to me as a good idea and we will be adding FTP option as well. Which will work similar as your components, when you compile a component and it sends it off to a sale server. That is the same features will make use of. That's just talking about selecting an API user, 00:06:13 and why that is important.

CronJob Tab

When we go to the CronJob tab, we will see that there is some basic instructions for those of you who are familiar with CronJobs. I suppose everything is self explanatory. Those who are not I would suggest you Google around, find some tutorials, read up, make sure what you're doing, 00:06:40 and that you're comfortable by setting up a CronJob. There are various ways for you to trigger this URL. The system currently detects whether your system can run curl request. If it can't it will fall back on the weget option. Assuming that JCB is installed in an environment that allows these two functions to work. If not you can simply take the URL, and run it in a CronJob in a manner that is 00:07:16 applicable to your system. And know that this URL triggers the backup to start. When the backup is finished, it tells you the same message that you will get if you run the backup by clicking backup here. You will see there is a green message that pops up, saying yes the backup was done, the email was sent. 00:07:42 Those are the only messages that also gets returned upon this URL. We currently throwing that responds away. But you could add it to a log, depending on how regularly this CronJobs is running. There isn't the date currently. I think 00:08:06 it might make sense for us to add a date. But at this stage there isn't one. If you're running in a firewall, your Joomla website has a firewall installed like RS firewall, then you might need to adapt your curl request to behave like a real browser. Here is a post on https://stackoverflow.com that could help you understand more about that.

Cronjob Backup Folder Path

Currently the 00:08:39 Cronjob Backup Folder path is set here. It currently backups to a local folder. As I said we will add an FTP option here. We will still work out how we can do that and still have it secure. Joomla doesn't by default encrypt fields in the Global Configuration of your component. That's might be part of what makes it difficult for us to implement this 00:09:11 here.

Email (Backup Key)

The point is currently it stores it into a local folder, and it Emails the backup key to the email address that you set in here. Now make sure that email address is secure and that it is safe because the keys are being sent to it.

Package Name Placeholders

Now here is a naming structure for your file by default. It adds only up till the day, so it will 00:09:45 make backups all through the day, but overwrite the file every time unless you add an hour, then it'll only per hour overwrite it. If you add a minute, it will never overwrite it, because it will be a different minute every time. That is to rename your package and to see how much of the backups you want to keep.

Mail Configuration Area

If we go to the mail configuration area, you will see that it is currently set to use the Global Mail Configuration which is your Joomla default. 00:10:23 Now I would advise that you use a very secure method of sending the emails. Either SMTP overwriting the SMTP settings. If you use the Global ones then overwrite or insured that your Global Email settings in Joomla is also using SMTP that runs through an SSL and is secure.

DKIM

The DKIM is also a added feature which can increase 00:11:00 trust and security of your emails.

Company Details

Then there is the Company details, which at this stage is important to add, since it will also become part of the backup package. We will look at that again when we restore a backup to show you where this information comes up. 00:11:28 That is some of the settings you need to first set within your Global Configuration. Now once you've set those settings, Company settings, your CronJob settings, your local path, the email, the name, and your API user, you can save and close this area and we can start looking at a component to 00:11:54 see this feature in action. Now since this is a default JCB install, there's only one component. It's the demo component.

Set An Export Key

Now usually the demo component doesn't have an encryption key. I'll open it, and 00:12:18 I've set it. You come and set an export key in at least one of the components that will be part of the backup. If any one of the components that are being backup have an export key, it will encrypt all the components with that export key. If multiple one's have an export key, it will combine those keys 00:12:47 to use as the encryption key. It hashes those keys so the actual key is not what is actually being used. The actual key that's being used is the one that will be emailed out to that trusted email address. We'll save and close the demo component.

Backup Manually

We will just run a manual backup, which I explained before, is similar to the CronJob except that it's triggered by clicking the Backup button. 00:13:18 It's one component, it's not a very big process, it should be finished quite quickly. It will say: The backup has been done successfully. 00:13:36 The owner details was set. If for some reason you forgot to set the owner details in the options area, it will tell you that here. The email with the new key was sent. To make sure that your backup works, you should go check the folder in which you set the backup should be placed. See whether the backup is there, and also 00:14:06 check the email address and see whether that they receive the key. I see I have a backup here in the folder as expected. The backups been done, I can double click into the backup, and make sure that it has all the expected files. At this stage it looks like it does. If you have a lot of custom files 00:14:38 that you've added into the component, there will also be an extra custom folder in the Zip Document and not only in image one. We will test this backup by importing it. But first let's see if we received the email. We can see that it send the email as expected, with the corresponding key. We'll be using this key to test the backup. 00:15:08 Be sure like I said that it sends the key to a secure Website Email Address, and that the email that sends the Email from Joomla is using a secure SSL by SMPT.

Testing Backup

Now let's test this backup. 00:15:35 A backup will only makes sense if we lost our data. I'm going to take the demo component, and throw that in trash. Then go to trash and empty the trash. There is no components. 00:16:10 Click on import components. Now we need to browse to the Directory, where that component was backed up. We'll click on the Directory tab. There we have the components path to the backup package. We click on 'Get File'. Now we will take that key that was sent to us via the email, and we will force the update, and will add that key. 00:16:45 We'll click continue.

Package Owner Details

I just step back for a moment because I realized I didn't explain what you'd seeing here. Like I said before your Package Data of the owneris desplayed here, and the Package Details is being displayed here. If you have more than one component backed up, all of those components will show here on the 'Components Being Imported'. Here you will see the the Package Owner 00:17:17 Details, and get the key from this link here(Testing Company).

I will again do this import. Let's click continue. Wonderful so we have the component back. It's all being restored. That is what the purposes of the backup is, is when something goes wrong 00:17:40 you can come back to where it was when the backup was made. We can test with a backup if it is been successfully restored by going to the compiler, and then selecting the component. Clicking on compile. As you can see is been completely successfully built. We can now click on install the demo component, and then we can go to the demo component, and see that everything is working. 00:18:18 Adding a new look. There is all the various fields as it usually are available in the demo component. That is automated settings for JCB, and setting them up and you making use of them.