filter_modify: Add data type condition checker to Modify filter.

[zihengCat]

Summary

It would be good if Fluent Bit supports strong data type condition check when applied in structured log stream. I implemented a data type condition checker for Modify filter.

Check data type using lua filter isn't clear enough for end-user and costs more performance overhead.

Usage

Modify filter supports two new conditions. See modify_filter_example.conf below.

Condition	Parameter 1	Parameter 2	Description
Key_Value_Type_Matches	STRING:KEY	STRING:VALUE	Is true if KEY exists and its value data type is VALUE.
Key_Value_Type_Does_Not_Match	STRING:KEY	STRING:VALUE	Is true if KEY exists and its value data type is not VALUE.

Parameter 2 data type string literal could be:

• null
• bool
• number
• int
• float
• str
• array
• map

[INPUT]
    Name dummy
    Tag  test
    dummy {"nullType": null, "boolType": true, "intType": 123, "floatType": 3.14159, "strType": "awesome!", "arrayType": [1, 2, 3], "mapType": {"k": "v"}}

[OUTPUT]
    Name  stdout
    Match *

[FILTER]
    Name  modify
    Match *

    Condition Key_Value_Type_Matches             nullType          null
    Condition Key_Value_Type_Matches             boolType          bool
    Condition Key_Value_Type_Matches             intType           number
    Condition Key_Value_Type_Matches             intType           int
    Condition Key_Value_Type_Matches             floatType         float
    Condition Key_Value_Type_Matches             strType           str
    Condition Key_Value_Type_Matches             arrayType         array
    Condition Key_Value_Type_Matches             mapType           map

    Condition Key_Value_Type_Does_Not_Match      mapType           array

    Rename nullType nullTypeMatches
    Rename mapType mapTypeDoesNotMatch

modify_filter_example.conf

Unit Tests

Here comes unit testing results.

13/31 Test #13: flb-rt-filter_kubernetes .........   Passed  193.00 sec
      Start 14: flb-rt-filter_parser
14/31 Test #14: flb-rt-filter_parser .............   Passed   20.00 sec
      Start 15: flb-rt-filter_modify
15/31 Test #15: flb-rt-filter_modify .............   Passed   55.00 sec
      Start 16: flb-rt-core_engine
16/31 Test #16: flb-rt-core_engine ...............   Passed   20.00 sec

Addresses

#2530

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

• Documentation required for this feature

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[edsiper]

thanks for this PR.

• is it nil a configuration value ?, if so, can it be null ? , that's more representative for end-users
• does number covers int and float ?

[zihengCat]

@edsiper
Fine.

• Change nil to null.
• number covers both int and float, corresponding to JSON number data type.

[edsiper]

public interfaces must be prefixed with flb_, e.g: flb_hash_gen_hash or flb_hash_generate

[zihengCat]

@edsiper Done!

• Rename gen_hash to flb_hash_generate.

[philomory]

Is there any chance this might eventually be merged?