현재 클러스터에서, 타 클러스터 접속 후에 다시 현재 클러스터 정보 확인 시 인증서 x509 에러 출력. /etc/kubernetes/admin.conf 파일이 변경되어 있는 것을 확인. kubelet.conf 에 타 클러스터 정보를 병합해야 하는 부분 없어서 발생한 것으로 추정되어, 이후 admin.conf 파일 관련해서 인증서 전체 업데이트 진행.
[1]. kubectl로 노드 정보 확인시 에러 확인
$ kubectl get node
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
[2]. 쿠버네티스 인증서 확인
kubeadm certs-check-expiration 로 확인시에, admin.conf 가 <invalid> 로 확인
# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Jun 17, 2023 06:37 UTC <invalid> ca no
apiserver Mar 26, 2024 06:41 UTC 112d ca no
apiserver-etcd-client Mar 26, 2024 06:41 UTC 112d etcd-ca no
apiserver-kubelet-client Mar 26, 2024 06:41 UTC 112d ca no
controller-manager.conf Mar 26, 2024 06:41 UTC 112d ca no
etcd-healthcheck-client Mar 26, 2024 06:41 UTC 112d etcd-ca no
etcd-peer Mar 26, 2024 06:41 UTC 112d etcd-ca no
etcd-server Mar 26, 2024 06:41 UTC 112d etcd-ca no
front-proxy-client Mar 26, 2024 06:41 UTC 112d front-proxy-ca no
scheduler.conf Mar 26, 2024 06:41 UTC 112d ca no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Mar 24, 2033 06:41 UTC 9y no
etcd-ca Mar 24, 2033 06:41 UTC 9y no
front-proxy-ca Mar 24, 2033 06:41 UTC 9y no
# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Mon 2023-12-04 19:35:18 EST; 1h 29min ago
Docs: https://kubernetes.io/docs/
Main PID: 12205 (kubelet)
Tasks: 18
Memory: 56.7M
CGroup: /system.slice/kubelet.service
└─12205 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --net...
Dec 04 19:35:25 k8s-master kubelet[12205]: I1204 19:35:25.897437 12205 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtvr...
Dec 04 19:35:25 k8s-master kubelet[12205]: I1204 19:35:25.897523 12205 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxdj...
Dec 04 19:35:25 k8s-master kubelet[12205]: I1204 19:35:25.897600 12205 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ln7...
Dec 04 19:35:25 k8s-master kubelet[12205]: I1204 19:35:25.897666 12205 reconciler.go:224] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-646x...
Dec 04 19:35:25 k8s-master kubelet[12205]: I1204 19:35:25.897763 12205 reconciler.go:157] "Reconciler: start to sync state"
Dec 04 19:35:27 k8s-master kubelet[12205]: I1204 19:35:27.176146 12205 request.go:668] Waited for 1.173272046s due to client-side throttling, not priority and fairness, ...o-node/token
Dec 04 19:35:27 k8s-master kubelet[12205]: I1204 19:35:27.545330 12205 prober_manager.go:255] "Failed to trigger a manual run" probe="Readiness"
Dec 04 19:35:27 k8s-master kubelet[12205]: I1204 19:35:27.547211 12205 prober_manager.go:255] "Failed to trigger a manual run" probe="Readiness"
Dec 04 19:35:28 k8s-master kubelet[12205]: I1204 19:35:28.571466 12205 prober_manager.go:255] "Failed to trigger a manual run" probe="Readiness"
Dec 04 19:35:28 k8s-master kubelet[12205]: I1204 19:35:28.571502 12205 prober_manager.go:255] "Failed to trigger a manual run" probe="Readiness"
[3]. 인증서 갱신 작업
쿠버네티스 인증서를 갱신작업 진행
kubeadm certs check-expiration
kubeadm certs renew all
kill -s SIGHUP $(pidof kube-apiserver)
kill -s SIGHUP $(pidof kube-controller-manager)
kill -s SIGHUP $(pidof kube-scheduler)
kill -s SIGHUP $(pidof etcd)
systemctl restart kubelet
systemctl daemon-reload
systemctl restart docker
kubeadm certs check-expiration으로 인증서 갱신 내용확인
# kubeadm certs check-expiration
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Dec 04, 2024 02:18 UTC 364d ca no
apiserver Dec 04, 2024 02:18 UTC 364d ca no
apiserver-etcd-client Dec 04, 2024 02:18 UTC 364d etcd-ca no
apiserver-kubelet-client Dec 04, 2024 02:18 UTC 364d ca no
controller-manager.conf Dec 04, 2024 02:18 UTC 364d ca no
etcd-healthcheck-client Dec 04, 2024 02:18 UTC 364d etcd-ca no
etcd-peer Dec 04, 2024 02:18 UTC 364d etcd-ca no
etcd-server Dec 04, 2024 02:18 UTC 364d etcd-ca no
front-proxy-client Dec 04, 2024 02:18 UTC 364d front-proxy-ca no
scheduler.conf Dec 04, 2024 02:18 UTC 364d ca no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Mar 24, 2033 06:41 UTC 9y no
etcd-ca Mar 24, 2033 06:41 UTC 9y no
front-proxy-ca Mar 24, 2033 06:41 UTC 9y no
kubectl 로 node 정상유무 확인
# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane,master,worker 252d v1.21.9
---
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-pod 1/1 Running 5 36d
nginx-pod-01 1/1 Running 3 34d
[APPENDIX]
https://velog.io/@ghkdtlwns987/Kubernetes-쿠버네티스-에러
[Kubernetes] 쿠버네티스 에러
쿠버네티스 공부좀 하려고 하는데 무슨 에러가 이렇게 많이 나는지 모르겠다.다음에 또 설치할 때 헷갈릴까봐 정리함.아래 명령어 실행이렇게 했는데도 잘 안됀다?Master에서 실행Worker에서 이전
velog.io
'Kubernetes' 카테고리의 다른 글
| nginx POD container /usr/share/nginx/html/index.html (0) | 2023.12.04 |
|---|
