afterValidate operation hook

[drish]

Hello guys.

The docs say the current version does not support afterValidate hook, but what about the case where I need to manipulate data after it is validated ?
For example, after all my fields are validated I'd like to encrypt some of them and persist into datastore.

Common hook model.afterValidate is deprecated.

Thanks in advance.

[superkhau]

Use operation hooks. See http://docs.strongloop.com/display/LB/Operation+hooks. For future reference, please post questions to https://groups.google.com/forum/#!forum/loopbackjs. See https://github.com/strongloop/loopback/wiki/Reporting-issues#question for more details.

[fabien]

@drish @superkhau see loopbackio/loopback-datasource-juggler#545

[superkhau]

@bajtos What should we do with this issue? Are validation hooks deprecated in favour of operation hooks?

[bajtos]

See also loopbackio/loopback-datasource-juggler#433 and loopbackio/loopback-datasource-juggler#482

[superkhau]

I see. I will assign this issue to you for further processing as it seems you are already working on it.

[bajtos]

I need to manipulate data after it is validated ? For example, after all my fields are validated I'd like to encrypt some of them and persist into datastore.

@drish Can you please elaborate a bit more please? Are you trying to store the property value in an encrypted form and decrypt it on load?

As I said in loopbackio/loopback-datasource-juggler#441 (comment), hooks are not the right tool for encrypted storage.

The hooks are not the right tool for transparent encryption of model properties (data), because the instance passed to "before save" hooks is then returned in the server response. The HTTP client will thus receive encrypted data, unless you implement also an "after save" hook that would decrypt them.

See #1260 for a discussion about the best way how to implement encrypted properties.

[drish]

@drish Can you please elaborate a bit more please? Are you trying to store the property value in an encrypted form and decrypt it on load?

@bajtos , gladly.

This is an example, on how I think we would need to have afterValidate, back (cuz it's deprecated).

On customer.json

{
   "name": "string"
}

On validation.js

customer.validateLengthOf("name", { min: 4 });

If I use something like:

customer.observe("before save", function(c, next) {
  encrypt(c); // suppose this function manipulates and encrypt some data
  next();
});

The validation on validation.js will fail, because in this case,
it will receive the instance already encrypted.

The right way, would be something like:

customer.observe("after validate", function(c, next) {
  // all validations passed
  encrypt(c);
  next()
});

Consequently, we would need to have the observe("loaded", cb) hook, in order to properly manipulate the data after it loaded from database, as is described at #441

Thanks for the efforts.

[bajtos]

@drish thank you for describing your use case.

So what if we implemented #1260 and allowed you to provide encrypt/decrypt functions as a part of your property definition, so that you don't need to deal with hooks at all? Would that be a better solution for your problem?

{
  "name": "MyModel",
  "properties": {
    "name": { "type": "string", "encrypted": true }
  }
}

module.exports = function(MyModel) {
  MyModel._encryptPropertyValue = function(propertyName, value, cb) {
    // your encryption algorithm
    cb(null, encryptedPropertyValue);
  };

  MyModel._decryptPropertyValue = function(propertyName, encryptedValue, cb) {
    // your encryption algorithm
    cb(null, value);
  };
}

[fabien]

@bajtos @drish I would propose a lower-level approach using hooks first - over time you could move towards something like #1260 which could be built on top of afterValidate (loopbackio/loopback-datasource-juggler#545) and loaded hooks. With #1260 you cover one specific use-case, I would like to see a more generalized solution.

[fabien]

In fact, I would kindly suggest to finally implement start implementing such functionality as mixins ... so you can do:

{
  "name": "MyModel",
  "properties": {
    "name": { "type": "string" }
  },
  "mixins": {
    "Encrypt": { "properties": ["name"] }
  }
}

[bajtos]

@fabien ok, I see your point. The Encrypt mixin can even read property definitions to see which of them should be encrypted, so that the user does not have to enumerate property names in mixin options.

{
  "name": "MyModel",
  "properties": {
    "name": { "type": "string", "encrypted": true }
  },
  "mixins": {
    "Encrypt": {}
  }
}

How about "loading" and "saving" hooks then? Note that you need to encrypt the property value even when performing updateAll which does not trigger validation, thus "after validate" is not a good hook for this.

[fabien]

@bajtos yes, of course - it's easy to do.

What if we had the following hooks as a final solution (hopefully):

• load: whenever an instances is loaded from the datasource
• save: in-between before save and saving through the DS, just after validation is performed
• error: whenever an error occurs along the way ({ ctx.error: ..., ctx.type: ... } will be set to differentiate)

If you have some bandwidth available, I'd like you to implement this. In turn, I can create the Encrypt mixin as a seperate npm package, and resolve #1260. What do you think?

[bajtos]

• load: whenever an instances is loaded from the datasource
• save: in-between before save and saving through the DS, just after validation is performed

Yes, this is exactly what I had in mind. I feel save may be confusing in the context of before save and after save. How about the name persist?

• error: whenever an error occurs along the way ({ ctx.error: ..., ctx.type: ... } will be set to differentiate)

+1 for ctx.error. What value do you expect in ctx.type? Are you proposing to trigger the same error hook for all operations, including find and delete?

If you have some bandwidth available, I'd like you to implement this.

I started to look into the save failed error hook. The implementation requires changes almost everywhere in dao.js so it takes longer than I expected.

/cc @raymondfeng

[drish]

Yes, this is exactly what I had in mind. I feel save may be confusing in the context of before save and after save. How about the name persist?

I agree with @bajtos, persist is more reasonable in this case.

@fabien , A mixins solution totaly LGTM, that would be nice to have that.

[bajtos]

• load: whenever an instances is loaded from the datasource
• save: in-between before save and saving through the DS, just after validation is performed

Yes, this is exactly what I had in mind. I feel save may be confusing in the context of before save and after save. How about the name persist?

Let's move the discussion here: loopbackio/loopback-datasource-juggler#559

error: whenever an error occurs along the way ({ ctx.error: ..., ctx.type: ... } will be set to differentiate)

+1 for ctx.error. What value do you expect in ctx.type? Are you proposing to trigger the same error hook for all operations, including find and delete?

Let's move the discussion to a standalone issue: loopbackio/loopback-datasource-juggler#560

---

@drish I am closing this issue in favour of loopbackio/loopback-datasource-juggler#441 and loopbackio/loopback-datasource-juggler#559. Feel free to reopen if you disagree.