-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
in_exec: Can't handle non-ASCII characters output #4460
Labels
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
in_exec
can not handle non-ASCII characters output.It is because of the specification of
child_process_execute
:external_encoding
:ascii-8bit
internal_encoding
:utf-8
encoding_options
:invalid: :replace, undef: :replace
This always breaks none non-ASCII characters.
fluentd/lib/fluent/plugin_helper/child_process.rb
Lines 65 to 72 in 1a2759c
fluentd/lib/fluent/plugin_helper/child_process.rb
Lines 247 to 253 in 1a2759c
fluentd/lib/fluent/plugin_helper/child_process.rb
Line 279 in 1a2759c
We can easily confirm the io behavior by irb:
I'm wondering if we should fix the implementation of
in_exec
as follows:By specifying
internal_encoding: nil
, we can stop the automatic encoding conversion inchild_process_execute
.This allows
in_exec
to handle non-ASCII characters.Does the current automatic encoding conversion make any sense?
One possible cause could be that the encoding of the data must be
utf-8
.Even if so, I believe it would be wrong to always convert the actual encode to
utf-8
of the result of the command inin_exec
.To Reproduce
Run the following sample config.
Expected behavior
in_exec
can handle non-ASCII characters output as well.Your Environment
Your Configuration
Your Error Log
Additional context
No response
The text was updated successfully, but these errors were encountered: