filter_kubernetes: Support config of endpoint path in Kube_URL.

[jimwrightcz]

The motivation is to make it possible to test config against a rancher cluster
before deployment.

Also fix a bug that the default port is only correct for the default protocol
i.e. HTTPS but also used incorrectly when Kube_URL is configured with http: and
no port. RFC-1738 dictates that the port is 80 in that case.

On the other hand, for backward compatibility the same default path is applied
if none is specified. If you really want none then specify the root /. In other
cases a trailing / is ignored and always inserted when the URL is used later.

The default path is extracted to FLB_KUBE_API_PATH. Similar constants are now
only defined in kube_conf.h, Unused duplicates in kube_meta.h have been removed
but the names were better so now used kube_conf.h.

Signed-off-by: Jim Wright [email protected]

• plugin code changes described above
• extension of unit test - not much can be automatically checked but test cases that exercise the code are there and that helps during development

None AFAIK but it is possible someone reported the port issue as a bug.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

• Documentation required for this feature

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[jimwrightcz]

Example config

[FILTER]
    Name               kubernetes
    Match              pod.*
    Kube_Tag_Prefix    pod.
    Regex_Parser       pod_tag
    Kube_URL           http://127.0.0.1/k8s/clusters/c-45qm6/api/v1/

[PARSER]
    Name        pod_tag
    Format      regex
    Regex       (?<namespace_name>[^.]+)\.(?<pod_name>[^.]+)

e.g. if you are using:
kubectl proxy --port=80

[jimwrightcz]

Debug and valgrind output:

==26978== Memcheck, a memory error detector
==26978== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==26978== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==26978== Command: /home/local/MGM/jwright/git/fluent-bit/build/bin/fluent-bit --config=/tmp/test.sh-26951-conf
==26978== 
�[1mFluent Bit v1.5.0�[0m
* �[1m�[93mCopyright (C) 2019-2020 The Fluent Bit Authors�[0m
* �[1m�[93mCopyright (C) 2015-2018 Treasure Data�[0m
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2020/06/15 15:31:35] [ info] Configuration:
[2020/06/15 15:31:35] [ info]  flush time     | 1.000000 seconds
[2020/06/15 15:31:35] [ info]  grace          | 5 seconds
[2020/06/15 15:31:35] [ info]  daemon         | 0
[2020/06/15 15:31:35] [ info] ___________
[2020/06/15 15:31:35] [ info]  inputs:
[2020/06/15 15:31:35] [ info]      forward
[2020/06/15 15:31:35] [ info] ___________
[2020/06/15 15:31:35] [ info]  filters:
[2020/06/15 15:31:35] [ info]      kubernetes.0
[2020/06/15 15:31:35] [ info] ___________
[2020/06/15 15:31:35] [ info]  outputs:
[2020/06/15 15:31:35] [ info]      stdout.0
[2020/06/15 15:31:35] [ info] ___________
[2020/06/15 15:31:35] [ info]  collectors:
[2020/06/15 15:31:36] [debug] [storage] [cio stream] new stream registered: forward.0
[2020/06/15 15:31:36] [ info] [storage] version=1.0.4, initializing...
[2020/06/15 15:31:36] [ info] [storage] in-memory
[2020/06/15 15:31:36] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/06/15 15:31:36] [ info] [engine] started (pid=26978)
[2020/06/15 15:31:36] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2020/06/15 15:31:36] [debug] [in_fw] Listen='0.0.0.0' TCP_Port=24224
[2020/06/15 15:31:36] [ info] [input:forward:forward.0] listening on 0.0.0.0:24224
[2020/06/15 15:31:36] [debug] [filter:kubernetes:kubernetes.0] Kube API endpoint http://127.0.0.1:80/k8s/clusters/c-45qm6/api/v1/
[2020/06/15 15:31:36] [ info] [filter:kubernetes:kubernetes.0] https=0 host=127.0.0.1 port=80
[2020/06/15 15:31:36] [ warn] [filter:kubernetes:kubernetes.0] cannot open /var/run/secrets/kubernetes.io/serviceaccount/namespace
[2020/06/15 15:31:36] [ info] [filter:kubernetes:kubernetes.0] Fluent Bit not running in a POD
[2020/06/15 15:31:36] [debug] [router] match rule forward.0:stdout.0
[2020/06/15 15:31:36] [ info] [sp] stream processor started
==26978== Warning: client switching stacks?  SP change: 0xffeffff78 --> 0x5be91f0
==26978==          to suppress, use: --max-stackframe=68606324104 or greater
==26978== Warning: client switching stacks?  SP change: 0x5be9168 --> 0xffeffff78
==26978==          to suppress, use: --max-stackframe=68606324240 or greater
==26978== Warning: client switching stacks?  SP change: 0xffeffff78 --> 0x5be9168
==26978==          to suppress, use: --max-stackframe=68606324240 or greater
==26978==          further instances of this message will not be shown.
[2020/06/15 15:31:36] [debug] [http_client] server 127.0.0.1:80 will close connection #21
[2020/06/15 15:31:36] [debug] [filter:kubernetes:kubernetes.0] API Server (ns=some_namespace, pod=tomcat-5c97498576-9bp8j) http_do=0, HTTP Status: 200
[2020/06/15 15:31:36] [debug] [task] created task=0x5be2db0 id=0 OK
[2020/06/15 15:31:36] [debug] [task] destroy task=0x5be2db0 (task_id=0)
[engine] caught signal (SIGTERM)
[2020/06/15 15:31:41] [ warn] [engine] service will stop in 5 seconds
[2020/06/15 15:31:45] [ info] [engine] service stopped
==26978== 
==26978== HEAP SUMMARY:
==26978==     in use at exit: 0 bytes in 0 blocks
==26978==   total heap usage: 619 allocs, 619 frees, 957,799 bytes allocated
==26978== 
==26978== All heap blocks were freed -- no leaks are possible
==26978== 
==26978== For counts of detected and suppressed errors, rerun with: -v
==26978== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.