Async processing

[advancedlogic]

Is there a way to process collected data during the harvesting time (process as you get) instead of waiting for the end of the script? Maybe using goroutines and channels?

[ziflex]

Hi @advancedlogic,

No, there is no such functionality, but you can register a custom function which would do it for you.

compiler. RegisterFunction('consume', func(ctx context.Context, args ...core.Value) (core.Value, error) {
    go func () {
       // do what you need to do
       // also, you can pass a control flow component with the context, that could help to orchestrate the async execution
    } ()
}) 

[advancedlogic]

Here the code if you want to implement the command:

ch := make(chan []byte) 
defer close(ch)
comp := compiler.New()
comp.RegisterFunction("consume", func(ctx context.Context, args ...core.Value) (core.Value, error) {
    err := core.ValidateArgs(args, 1, 1)
    if err != nil {
        return values.None, nil
     }
     j, _ := values.ToArray(ctx, args[0]).MarshalJSON()
     ch <- j
     return values.None, nil
})