[enhancement] Windows support

[jkroepke]

Describe the enhancement you'd like

I have some cluster with Windows nodes enabled. I would like to ask if I can add windows support or do you think it out of context here?

Unlike kubernetes-mixin, which have separate dashboard, I would like to add the Windows queries into the existing one. Thats possible by using queries with OR, e.g.:

sum(container_memory_working_set_bytes{cluster="$cluster",namespace=~"$namespace", image!="", pod=~"${created_by}.*"}) by (pod)
OR
<WINDOWS Query>

Additional context

Since I'm running multiple OS hybrid clusters, I would like to add PRs for windows pods here. I'm not expecting that the maintainers here provide support for Windows. Before start to work here, I would like to know if its getting accepted?

[dotdc]

Hi @jkroepke,

I'm not totally against it, but I have a few doubts...

Here are a few questions to start the discussion:

• If I understand correctly, you have clusters with both GNU/Linux and Windows node pools.
  Could you share your use-case? (just curious)
• Which dashboard(s) are you planning to add Windows support for?
• Why do you think having the queries in here would be better than having them in separate dashboards?
• Will there be any OS specific panels? And how do you plan to manage them (if any)?
• After the initial work, would you be able to help Windows users if they have issues related to your change?

If anyone is also using Windows hosts, feel free to jump in the discussion with your thoughts!

[jkroepke]

If I understand correctly, you have clusters with both GNU/Linux and Windows node pools.
Could you share your use-case? (just curious)

Sure! The Kubernetes Control Plane does not support Windows. I have to run Linux Node (e.g. for CoreDNS, ArgoCD) and Windows Node Pools for Application (Customer runs dotNET Application)

On Azure, a Kubernetes managed Services requires at-least one Linux Node and you can add additional Windows nodes. We are also quite common, that customers wants to use Standard infrastructure components like Redis, Solr or elasticsearch which are not running on Windows node. In general, we avoid Windows nodes as much as possible. Hybrid OS clusters are common, because mono-Windows cluster can't be exists.

Which dashboard(s) are you planning to add Windows support for?

Cluster, Namespace, Node, Pod

Why do you think having the queries in here would be better than having them in separate dashboards?

We have namespace which contains windows and linux containers. My personal opinion is that one dashboard provides a better user expericence.

Since kube-state-metrics provides request/limits for pods on any OS, Windows Pods are already included Pod CPU/Memory Request/Limits panels, but excluded on the usage panel. For separate dashboards, Requests/Limits from Windows Pods should be excluded then.

Example Panel: Real (linux only), Requests/Limits (any OS)

Will there be any OS specific panels?

No, except Windows is not providing metrics, some panels will be Linux specific. But I do not plan to add new panels.

After the initial work, would you be able to help Windows users if they have issues related to your change?

Yes, because out company has an upstream first culture. If other users have a bug, we may also have the bug, too. Of course, I will have.

The good thing is that the dashboard are providing only basic metrics. I don't plan to engineer the queries on my own. kubernetes-mixin provides some rules for windows which I will use as base. for the panel. I won't add a dependency against the kubernetes-mixin recording rules.

---

Since I expect some large changes with #15, I plan to start after the cluster variable work is finished.

[dotdc]

Hi @jkroepke,
I'm fine with the idea, we can try to add Windows support, as soon as it doesn't impact the experience on Linux.
Will just finish the work on the other issues before looking further into this.

[jkroepke]

Will just finish the work on the other issues before looking further into this.

I would prefer to split the Windows integration into different, smaller PRs. At minimum a PR for each dashboard. Let me know, when I can start with the cluster dashboard.

[dotdc]

@jkroepke The other issues/PRs are done, so you can start working on this whenever you want.

[jkroepke]

Hey @dotdc ,

In #103, I tired to split Linux and Windows queries when ever it was possible. Did you prefer this kind of solution or should we prefer more complex query as seen on the CPU by namespace queries?

[dotdc]

Hi @jkroepke,
I think I prefer the split version, but if it's not possible to have that everywhere, maybe go for the other one to have consistency.
What do you think ?

[jkroepke]

What do you think ?

The combind solution (one query for Windows and OS) results into very complex and un-understandable queries. Due complexity, external users may avoid to contribute improvement or they may destroy the Windows support by accident

[dotdc]

Agree.
Do you think it's possible to split the combined queries like "CPU by namespace" ?

[jkroepke]

Maybe, no idea yet. I may have to ask at Grafana slack.

[jkroepke]

Do you think it's possible to split the combined queries like "CPU by namespace" ?

In theory yes, but the make it complex at Grafana side:

[dotdc]

Can we manage all the cases using combined queries?

If it's the case, it might be better to go for the combined queries with comments to have consistency.
Here's an example :

What do you think ?

[dotdc]

Also, could be nice to add Windows support knowledge in the docs (README.md) for the other users.