I found out the json-rules-engine is emiting events twice.

[hitesh-c]

What is the purpose of emitting event from rule.js when handled in engine.js?

@CacheControl @brianphillips @bjacobso @feenst

[hitesh-c]

Engine.js

Rule.js

[chris-pardy]

@hitesh-c conceptually the rule could be run in multiple engines, or the engine may emit events from many rules, and not just a specific rule you care about.

Regardless you shouldn't be seeing 2 events emitted from the engine. You should be seeing a rule emit 1 event per-evaluation. The engine should emit either a success of failure event for each rule in each evaluation. And if there is a success event it should also emit the specific event type. The purpose of emitting the specific event type on success is it lets you subscribe to a much smaller set of events if you only care about specific types of success.

[chris-pardy]

I'll take a look shortly

[hitesh-c]

Thanks, Will wait. It will be helpful if you can tell something about caching, If I use this package in real-time will facts keeps on cache?
How can I control Cache?

[chris-pardy]

Ok so the problem is here: https://github.com/hitesh-c/json-rules-engine-demo/blob/76eac5902c3a5d4505f13ea014c19b889249da07/example.js#L18-L20

Because you also added the rules when you initialized the engine (Line 10) you actually have 2 copies of every rule. Hence 2 copies of every event.

In terms of controlling the cache and re-using between runs there's 2 approaches.

1. Cache outside of the engine, for instance in a client for an external service.
2. Create your own Almanac and re-use it between different executions. This can be done by passing the almanac to the second argument to run

[hitesh-c]

Can you please give some advice on integrating Redis for controlling cache. I need some information on scaling this repository.

[chris-pardy]

you can create an instance of a fact that gets the data from Redis, look at the custom fact examples to see what that could look like.

[hitesh-c]

Please advice me if I can use json-rules-engine in real-time? I want to integrate json-rules-engine with Kafka, what challenges I can face?

[chris-pardy]

@hitesh-c the biggest real-time issue is that while an engine is running it's not safe to start another execution.

What I'd recommend is one of the following approaches:

• New Engine for each message - This is simple, allows parallel processing of messages, but you're going to have setup overhead.
• Process each message fully before proceeding - This is simple but it means that you can have a backlog of messages that could take a while to resolve.
• Use a pool of engines to process messages - Sort of a mix of the other 2 options where you pay your startup cost for multiple engines upfront and then have each one process one message at-a-time. However this is also the most complicated option to maintain and understand the performance trade-offs of.

[hitesh-c]

I am thinking to use RxJs and add more capabilities like time-based rule, window aggregations. like Drools. Can you advice if this is possible?

[chris-pardy]

Should be fine. Just remember that when you call run the engine returns a promise and until that promise resolves it's not safe to call run again.

[hitesh-c]

Thanks for the information. You mean to run engine in for loop with await instead of promise.all?.

I also thought to initialize Engine class for every rule in rules i.e. to have 1 engine for 1 rule. I think there is no cache logic for combined conditions of all rules? Please tell if you have any best practice to run this engine in real-time data processing.

[chris-pardy]

If you're talking about true "real-time" then you shouldn't be using javascript / node as it's not a real-time system and is garbage collected. If you're talking about streaming processing in a reasonable timeframe then the biggest question is if you have any facts that have to call out to external systems. Since Javascript is single threaded you're only going to get to run code in parallel if you have I/O. Assuming you do need to get data from outside your system then what I'd suggest is using a pool of workers each with an engine.

class Worker {
    constructor(rules, engineOptions) {
         this.#engine = new Engine(rules, engineOptions);
    }
    async run(iterator) {
         while(true) {
              const { value, done } = await iterator.next();
              if (done) {
                  break;
              }
              const results = await engine.run(value);
              // do something with results
         }
    }
}

// get the data as an async iterator
const iterator = getDataFromQueue()[Symbol.asyncIterator]();

const promises = [];
// create a fixed number of workers.
const worker_count = 4;
for (let i = 0; i < worker_count; i++) {
   promises.push(new Worker().run(iterator));
}
// wait for everything to be done.
await Promise.allSettled(promiese);