Windows pods do not start with flannel host-gw

[nwelch-wsp]

We are building a new k8 cluster that needs to run windows pods on windows nodes. We are using host-gw and win-bridge. Everything in the cluster seems to be running without errors (all of the kube-system pods, including flanneld and kube-proxy), except for a sample windows pod which is stuck in ContainerCreating with an error (the win-webserver example here).

Expected Behavior

The sample windows webserver pod should start successfully and run in a stable condition as described by the kubernetes docs.

Current Behavior

When we apply the sample yaml for the win-webserver pod, it goes into ContainerCreating state and gets stuck there. If I describe the win-webserver pod, it has the following error:

Warning FailedCreatePodSandBox 3m48s (x84 over 21m) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "87f12bfcf389b899cb8a0dd6213ca0d87889169715e0da6a11b6c2e21d1d93c8": plugin type="flannel" name="cbr0" failed (add): failed to delegate add: error while adding HNSEndpoint: failed to find HNSEndpoint 87f12bfcf389b899cb8a0dd6213ca0d87889169715e0da6a11b6c2e21d1d93c8_cbr0: Endpoint 87f12bfcf389b899cb8a0dd6213ca0d87889169715e0da6a11b6c2e21d1d93c8_cbr0 not found

Possible Solution

I'm not sure if this is a bug or a setup issue. If it's a setup issue, some additional documentation would be helpful.

Steps to Reproduce (for bugs)

1. Build a k8 cluster on 1.28, using k8 documentation starting here
2. Build a windows 2019 machine, install containerd, and join to the k8 cluster, using documentation here
3. Install flannel using guide here
4. Run a basic sample webserver pod on the windows node, using example yaml here
5. The win-webserver pod is stuck in ContainerCreating state with the error message above shown when you describe the pod.

Context

Our end goal is to have a k8 cluster that will run a mix of linux pods and windows pods. The windows pods need to run a website with IIS, and they must connect to a domain to access domain resources.

We are currently running this setup in a few other clusters that are running kubernetes v1.22.2 and v1.23.1. We are just trying to build a new cluster on the latest version of kubernetes that will do the same thing as the older clusters. The other difference is the old clusters are running docker, whereas we're using containerd on the new one as seems to be the recommendation from kubernetes.

Your Environment

• Flannel version: v0.21.5
• Backend used (e.g. vxlan or udp): host-gw
• Etcd version: ?
• Kubernetes version (if used): v1.28.1
• Operating System and version: ubuntu 22.04, windows server 2019

[rbrtbnfgl]

Hi sorry for the late responds. Do you have more logs from Flannel windows agent? I am not so in depth of the microsoft internal I could try to help you.

[danijam]

@nwelch-wsp Hey did you ever have any luck with your goal? We are trying to achieve the same.

Our end goal is to have a k8 cluster that will run a mix of linux pods and windows pods.

We are facing lots of deadends!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.