customRuntimeOptions in "Creates a new Pulsar Source in cluster mode" API call in source connector doesn't add nodeselectors & tolerations

[Affanmir]

Describe the bug
The API call "Creates a new Pulsar Source in cluster mode" LINK Adding node selectors and toleration under customRuntimeOptions flag doesn't seem to add those selectors & toleration to the created pulsar functions source pods. For reference I'm using apache pulsar installed using helm on k8

To Reproduce
Steps to reproduce the behavior:

1. Create an "Creates a new Pulsar Source in cluster mode" with "customRuntimeOptions": json.dumps(
   "nodeSelectorLabels": {
   "customLabel": "value"
   },
   "tolerations": [
   {
   "key": "custom-key",
   "value": "value",
   "effect": "NoSchedule"
   }
   ]) As mentioned here

2, the created source connector doesn't have the node selector / toleration

Expected behavior
The passed nodeselectors and tolerations from customRuntimeOptions should be inherited onto the pods that get created

Desktop (please complete the following information):

• OS: Mac OS 13.5.2 (22G91) Apple M1 Pro

Additional context
Add any other context about the problem here.

[lhotari]

Related to #382

[kunj-bosamia]

Related to #382

In this issue , you suggested to configure things by passing values to broker's config map. which i did

PF_runtimeCustomizerClassName: "org.apache.pulsar.functions.runtime.kubernetes.BasicKubernetesManifestCustomizer"
PF_runtimeCustomizerConfig_nodeSelectorLabels_test: value

based on this configuration , this will add nodeSelectors test:value to all the functions which will be deployed. Which is correct its the default nodeSelector. But I want different nodeSelector for different functions which I deploy. So I should pass ": "customRuntimeOptions": "{\"nodeSelectorLabels\":{\"customLabel\":\"value\"}" Passing it as string because type of customRuntimeOptions is string which is mentioned here.
I am getting this error when I am paassing customRuntimeOptions when creating a pulsar function ->

{"reason":"Function my-test-function cannot be admitted:- java.io.EOFException: End of input at line 1 column 40 path $.nodeSelectorLabels"}

If I don't pass customRuntimeOptions while creating the function , it works fine and take the default node selector test:value

[lhotari]

But I want different nodeSelector for different functions which I deploy.

@kunjbosamiaFalkonry One possible approach would be to create your own customizer implementation. Currently there's not an easy way to get your custom classes into use. You need a custom overlay docker image with the jar file in /pulsar/lib . There's a feature request to add init container support that could copy the jar file to /pulsar/lib when the pod starts. That's #491 / #240 . This is an open source project and contributions are more than welcome to improve Apache Pulsar Helm chart.