Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kind cluster: context deadline exceeded #1401

Open
1 task
JCzz opened this issue Apr 16, 2024 · 7 comments
Open
1 task

kind cluster: context deadline exceeded #1401

JCzz opened this issue Apr 16, 2024 · 7 comments
Assignees
Labels
bug Something isn't working terratest

Comments

@JCzz
Copy link

JCzz commented Apr 16, 2024

Describe the bug
I am running k8s kind cluster on macos arm64. When running:
go test -v -tags kubernetes -run TestKubernetesHelloWorldExample
It do apply the pods, but when testing for http://172.18.0.1:5000 it runs into problems, see terminal output below.
Do you use client-go to access http://172.18.0.1:5000 or do I need to create an ingress to access from the host?

Nice to have

  • Terminal output
    TestKubernetesHelloWorldExample 2024-04-16T19:26:01+02:00 retry.go:103: HTTP GET to URL http://172.18.0.1:5000 returned an error: Get "http://172.18.0.1:5000": context deadline exceeded (Client.Timeout exceeded while awaiting headers). Sleeping for 3s and will try again.

Versions
Terratest version: v0.46.13
MacOS
go version
go version go1.21.7 darwin/arm64

Same On Windows 11 WSL
go version go1.21.3 linux/amd64

Additional context
Add any other context about the problem here.

@JCzz JCzz added the bug Something isn't working label Apr 16, 2024
@denis256
Copy link
Member

Hi,
to access Kubernetes is used go client, it may be that the request reaches the cluster but hangs for some reason(API problem, network policies) - I would say to try to use kubectl cli to check if the cluster is reachable and can create pods.

@JCzz
Copy link
Author

JCzz commented Apr 17, 2024

Hi @denis256

Thanks, I can create pods and use kubectl for usual stuff. I can also see:
kubectl get pods

NAME                                     READY   STATUS    RESTARTS       AGE
hello-world-deployment-944cbd46d-2jpzz   1/1     Running   0              18h

I can also run my own client-go up against the cluster.

@denis256
Copy link
Member

Hm, it may be a problem with accessing exposed services - it is responding when accessing through services?

@JCzz
Copy link
Author

JCzz commented Apr 17, 2024

That is the problem, I think. The service is running in a kind cluster, using docker on macos. I can see the services, but unable to "curl 172.18.0.1:5000" to the service from host machine:

kubectl get svc
NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hello-world-service   LoadBalancer   10.96.31.44     172.18.0.1    5000:31173/TCP   25h
kubernetes            ClusterIP      10.96.0.1       <none>        443/TCP          27d

curl 172.18.0.1:5000
curl: (7) Failed to connect to 172.18.0.1 port 5000 after 8 ms: Couldn't connect to server

This is known issue running on non linux environment, but using client-go should be able to to access the services from within the same network, I think?

	// website::tag::4::Check if NGINX service was deployed successfully.
	// This will get the service resource and verify that it exists and was retrieved successfully. This function will
	// fail the test if the there is an error retrieving the service resource from Kubernetes.
	service := k8s.GetService(t, options, "nginx-service")
	require.Equal(t, service.Name, "nginx-service")

@denis256
Copy link
Member

denis256 commented Apr 18, 2024

On macOS and Windows, docker does not expose the docker network to the host. Because of this limitation, containers (including kind nodes) are only reachable from the host via port-forwards

https://kind.sigs.k8s.io/docs/user/loadbalancer/

@JCzz JCzz closed this as completed Apr 19, 2024
@JCzz JCzz reopened this Apr 19, 2024
@JCzz
Copy link
Author

JCzz commented Apr 20, 2024

Maybe a suggestion could be to use https://github.com/curl/curl-docker

@jk1m
Copy link

jk1m commented May 29, 2024

Note: Running on Apple MBP M1

I was getting the same error but managed to figure it out after looking closer. It's actually an issue with the image, training/webapp:latest. execing into the pod, there were no Flask processes running and when trying to manually run it, I received Segmentation fault (core dumped). Possible reasons? Trying to pull the image yields:

"Suggest the author of docker.io/training/webapp:latest to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2."

Changing the image and corresponding port (in the test, too) will get the test to pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working terratest
Projects
None yet
Development

No branches or pull requests

4 participants