out_syslog: Add nested key support. #2516
Conversation
Signed-off-by: Pascal Rivard <[email protected]>
pascapascapascapasca
requested review from
edsiper,
fujimotos and
koleini
as code owners
…_keys Conflicts: plugins/out_syslog/syslog.c plugins/out_syslog/syslog_conf.c plugins/out_syslog/syslog_conf.h
…_keys Signed-off-by: Pascal Rivard <[email protected]>
|
Hi, I was just wondering if there a problem with this Pull Request? I can't see any error, except for Travis CI that failed for something not due to the changes, and I haven't got any news from the reviewers. Any feedback would be appreciated! |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This pull request should not be closed. It is frequent to have to work with nested keys when dealing with data that is in the ECS format. In the case where we want to send logs to multiple destinations, both Elasticsearch and syslog for backwards compatibility reason, we want to have this nested key support. Is it possible to move a head with the review of this pull request? |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
It is frequent to have to work with nested keys when dealing with data that is in the ECS format. In the case where we want to send logs to multiple destinations, both Elasticsearch and syslog for backwards compatibility reason, we want to have this nested key support. Is it possible to move a head with the review of this pull request? |
|
Hi, |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
still no news ? |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR is still relevant, @edsiper is there any chance to get a bit of cycles to look at it? |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR is still relevant, although after nearly 16 months I am not sure there is any hope this will ever be reviewed or merged... |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
The PR is still relevant: we needed to write our own custom syslog plugin to circumvent the limitation but would like to be able to use the standard plugin to handle structures with nested keys. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
The PR is still relevant: we needed to write our own custom syslog plugin to circumvent the limitation but would like to be able to use the standard plugin to handle structures with nested keys. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
The PR is still relevant: we needed to write our own custom syslog plugin to circumvent the limitation but would like to be able to use the standard plugin to handle structures with nested keys. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
The PR is still relevant: we needed to write our own custom syslog plugin to circumvent the limitation but would prefer to be able to use the standard plugin to handle structures with nested keys. |
|
any plans to merge this? |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
The PR is still relevant: we needed to write our own custom syslog plugin to circumvent the limitation but would prefer to be able to use the standard plugin to handle structures with nested keys. |
|
I think that maybe a better approach would be improving the current version to use the record accessor. That way we wouldn't be adding task specific "duplicated" code but instead we'd be reusing the existing infrastructure. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
Signed-off-by: Pascal Rivard [email protected]
Add nested key support in the out_syslog plugin. When a nested key separator is provided in the config, recursively iterate through map values inside the key-values provided by fluent-bit. Build a list of expanded keys with associated values, using the defined separator. Iterate through that list to find the defined syslog keys.
Addresses #2168
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
Documentation
valgrind-out.txt
fluent-bit.log
logs.zip
] Documentation required for this feature
NOTE: Valgrind shows a leak, but it seems to have to do with this line, which was not modified in this PR. From what I see, this string is never freed.
syslog_conf.c: 167
ctx->sd_key[ctx->nsd] = flb_sds_create(prop->val);
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.