[Tencent:Cluster] I cannot access k8s cluster with kubeconfig

[sykim-etri]

텐센트 TKE를 통해 생성한 쿠버네티스에 대해 GetCluster()를 호출하여 확보한 .AccessInfo.Endpoint와 .AccessInfo.Kubeconfig를 활용하여 해당 쿠버네티스 클러스터에 접근할 수 없는 상황입니다.

Endpoint와 Kubeconfig의 데이터 양식이 변경되어 그런 것으로 추정되며, 추가적인 확인이 필요합니다.

    "AccessInfo": {
      "Endpoint": "lb-fa8wewdk-86wdzr4px2e3b51u.clb.hk-tencentclb.com:443",
      "Kubeconfig": "apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWT ...
    },  

[sykim-etri]

Tencent Console에서 제공하는 Kubeconfig를 다운로드받아 접근시에도 동일한 증상이 발생하고 있습니다.

E0103 08:50:18.037003  278078 memcache.go:265] couldn't get current server API group list: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:54602->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:54586->124.156.123.236:443: read: connection reset by peer
E0103 08:50:29.332838  278078 memcache.go:265] couldn't get current server API group list: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:35416->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:35408->124.156.123.236:443: read: connection reset by peer
E0103 08:50:40.637683  278078 memcache.go:265] couldn't get current server API group list: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:40736->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:40724->124.156.123.236:443: read: connection reset by peer
E0103 08:50:51.963073  278078 memcache.go:265] couldn't get current server API group list: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:41368->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:41366->124.156.123.236:443: read: connection reset by peer
E0103 08:51:03.299669  278078 memcache.go:265] couldn't get current server API group list: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:43748->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:43738->124.156.123.236:443: read: connection reset by peer
error: Get "https://lb-8k3ka4g2-9xu4zy82q6tmlgp7.clb.hk-tencentclb.com:443/api?timeout=32s": read tcp 211.175.140.145:43748->124.156.123.236:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:43738->124.156.123.236:443: read: connection reset by peer

[sykim-etri]

로드밸런서는 정상적으로 생성되어 있으며, 리스너의 443 포트가 열려 있고, SecurityGroup도 크게 이슈가 없는 것으로 보인다.

• 로드밸런서 현황
  

• 리스너 현황
  

• 연관 SecurityGroup 항목
  

• 연관 SecurityGroup 항목의 Rule
  

[sykim-etri]

@powerkimhub @seokho-son
ap-seoul, na-siliconvalley 리전 등에 대해 클러스터 생성 및 접근에는 문제가 없는 것으로 확인되었습니다.
ap-hongkong 리전만의 이슈인 것으로 판단되며, 현 시점에서는 CB-TB에서 해당 리전을 접근할 수 없도록 임시 조치할 예정입니다.

[sykim-etri]

중국 지역 리전(ap-hongkong, ap-beijing, ap-chengdu, ap-chongqing, ap-guangzhou, ap-nanjing, ap-shanghai)의 클러스터에 대해, devstack 내 VM과 원내 VM에서 접근 시도시 아래와 같이 접속 실패하는 현상이 지속되는 상황이며, 이외의 리전에서는 정상적으로 접속이 가능한 것으로 확인되었습니다.
더불어 ap-nanjing 내 VM에서 상기 리전의 클러스터에 대해서는 모두 정상적으로 접속이 가능하였습니다.

중국 이외 지역에서 중국 지역 리전에 대한 접근 문제는 네트워크 이슈가 존재하는 것으로 추정해볼 수 있겠습니다.

$ kubectl --kubeconfig k8scluster-01.kubeconfig get node --v=8
I0107 09:54:36.124829 2652098 loader.go:395] Config loaded from file:  k8scluster-01.kubeconfig
I0107 09:54:36.126408 2652098 round_trippers.go:463] GET https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
I0107 09:54:36.126434 2652098 round_trippers.go:469] Request Headers:
I0107 09:54:36.126447 2652098 round_trippers.go:473]     Accept: 
...
I0107 09:54:51.903495 2652098 round_trippers.go:463] GET https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
I0107 09:54:51.903519 2652098 round_trippers.go:469] Request Headers:
I0107 09:54:51.903548 2652098 round_trippers.go:473]     Accept: application/json;g=apidiscovery.k8s.io;v=v2;as=APIGroupDiscoveryList,application/json;g=apidiscovery.k8s.io;v=v2beta1;as=APIGroupDiscoveryList,application/json
I0107 09:54:51.903573 2652098 round_trippers.go:473]     User-Agent: kubectl/v1.30.2 (linux/amd64) kubernetes/3968350
I0107 09:54:52.127873 2652098 round_trippers.go:574] Response Status:  in 224 milliseconds
I0107 09:54:52.127937 2652098 round_trippers.go:577] Response Headers:
I0107 09:54:53.128394 2652098 with_retry.go:234] Got a Retry-After 1s response for attempt 4 to https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
...
E0107 09:55:12.763669 2652098 memcache.go:265] couldn't get current server API group list: Get "https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s": read tcp 211.175.140.145:38652->118.24.227.207:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:38640->118.24.227.207:443: read: connection reset by peer
I0107 09:55:12.763764 2652098 cached_discovery.go:120] skipped caching discovery info due to Get "https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s": read tcp 211.175.140.145:38652->118.24.227.207:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:38640->118.24.227.207:443: read: connection reset by peer
I0107 09:55:12.764038 2652098 round_trippers.go:463] GET https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
I0107 09:55:12.764116 2652098 round_trippers.go:469] Request Headers:
I0107 09:55:12.764142 2652098 round_trippers.go:473]     Accept: application/json;g=apidiscovery.k8s.io;v=v2;as=APIGroupDiscoveryList,application/json;g=apidiscovery.k8s.io;v=v2beta1;as=APIGroupDiscoveryList,application/json
I0107 09:55:12.764164 2652098 round_trippers.go:473]     User-Agent: kubectl/v1.30.2 (linux/amd64) kubernetes/3968350
I0107 09:55:12.993760 2652098 round_trippers.go:574] Response Status:  in 229 milliseconds
I0107 09:55:12.993827 2652098 round_trippers.go:577] Response Headers:
I0107 09:55:13.994186 2652098 with_retry.go:234] Got a Retry-After 1s response for attempt 1 to https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
...
I0107 09:55:37.008509 2652098 round_trippers.go:463] GET https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s
I0107 09:55:37.008531 2652098 round_trippers.go:469] Request Headers:
I0107 09:55:37.008563 2652098 round_trippers.go:473]     Accept: application/json;g=apidiscovery.k8s.io;v=v2;as=APIGroupDiscoveryList,application/json;g=apidiscovery.k8s.io;v=v2beta1;as=APIGroupDiscoveryList,application/json
I0107 09:55:37.008582 2652098 round_trippers.go:473]     User-Agent: kubectl/v1.30.2 (linux/amd64) kubernetes/3968350
I0107 09:55:37.184763 2652098 round_trippers.go:574] Response Status:  in 176 milliseconds
I0107 09:55:37.184805 2652098 round_trippers.go:577] Response Headers:
E0107 09:55:37.184946 2652098 memcache.go:265] couldn't get current server API group list: Get "https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s": read tcp 211.175.140.145:41656->118.24.227.207:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:41640->118.24.227.207:443: read: connection reset by peer
I0107 09:55:37.184972 2652098 cached_discovery.go:120] skipped caching discovery info due to Get "https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s": read tcp 211.175.140.145:41656->118.24.227.207:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:41640->118.24.227.207:443: read: connection reset by peer
error: Get "https://lb-88i9czq2-gfue7cahikx8l2om.clb.cq-tencentclb.com/api?timeout=32s": read tcp 211.175.140.145:41656->118.24.227.207:443: read: connection reset by peer - error from a previous attempt: read tcp 211.175.140.145:41640->118.24.227.207:443: read: connection reset by peer