环境及版本

操作系统 Debian GNU/Linux 11 (bullseye)
Kubernetes v1.25.10
KubeKey kk version: &version.Info{Major:"3", Minor:"0", GitVersion:"v3.0.13", GitCommit:"ac75d3ef3c22e6a9d999dcea201234d6651b3e72", GitTreeState:"clean", BuildDate:"2023-11-07T08:42:04Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/amd64"}

集群配置文件

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: control, address: 10.0.0.36, internalAddress: 10.0.0.36, user: root, password: "PASSWORD"}
  - {name: node1, address: 10.0.0.38, internalAddress: 10.0.0.38, user: root, password: "PASSWORD"}
  - {name: node2, address: 10.0.0.39, internalAddress: 10.0.0.39, user: root, password: "PASSWORD"}
  - {name: node3, address: 10.0.0.40, internalAddress: 10.0.0.40, user: root, password: "PASSWORD"}  
  roleGroups:
    etcd:
    - control
    control-plane: 
    - control
    worker:
    - node1
    - node2
    - node3
  controlPlaneEndpoint:
    ## Internal loadbalancer for apiservers 
    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.25.4
    clusterName: cluster.local
    autoRenewCerts: true
    containerManager: containerd
  etcd:
    type: kubekey
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    multusCNI:
      enabled: false
  registry:
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons: []

kubeadm init 报错 "unknown service runtime.v1alpha2.RuntimeService"

root@kubesphere:~# ./kk create cluster --skip-pull-images --with-local-storage  --with-kubernetes v1.25.10 --container-manager containerd  -y -f config.yaml


 _   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

14:44:28 CST [GreetingsModule] Greetings
14:44:29 CST message: [node3]
Greetings, KubeKey!
14:44:29 CST message: [node1]
Greetings, KubeKey!
14:44:29 CST message: [control]
Greetings, KubeKey!
14:44:30 CST message: [node2]
Greetings, KubeKey!
14:44:30 CST success: [node3]
14:44:30 CST success: [node1]
14:44:30 CST success: [control]
14:44:30 CST success: [node2]
14:44:30 CST [NodePreCheckModule] A pre-check on nodes
14:44:46 CST success: [control]
14:44:46 CST success: [node3]
14:44:46 CST success: [node2]
14:44:46 CST success: [node1]
14:44:46 CST [ConfirmModule] Display confirmation form
+---------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name    | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+---------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| control | y    | y    | y       | y        | y     | y     | y       | y         |        | 24.0.7 | 1.6.24     |            |             |                  | CST 14:44:37 |
| node1   | y    | y    | y       | y        | y     | y     | y       | y         |        | 24.0.7 | 1.6.24     |            |             |                  | CST 14:44:46 |
| node2   | y    | y    | y       | y        | y     | y     | y       | y         |        | 24.0.7 | 1.6.24     |            |             |                  | CST 14:44:46 |
| node3   | y    | y    | y       | y        | y     | y     | y       | y         |        | 24.0.7 | 1.6.24     |            |             |                  | CST 14:44:44 |
+---------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

14:44:46 CST success: [LocalHost]
14:44:46 CST [NodeBinariesModule] Download installation binaries
14:44:46 CST message: [localhost]
downloading amd64 kubeadm v1.25.10 ...
14:44:46 CST message: [localhost]
kubeadm is existed
14:44:46 CST message: [localhost]
downloading amd64 kubelet v1.25.10 ...
14:44:47 CST message: [localhost]
kubelet is existed
14:44:47 CST message: [localhost]
downloading amd64 kubectl v1.25.10 ...
14:44:48 CST message: [localhost]
kubectl is existed
14:44:48 CST message: [localhost]
downloading amd64 helm v3.9.0 ...
14:44:48 CST message: [localhost]
helm is existed
14:44:48 CST message: [localhost]
downloading amd64 kubecni v1.2.0 ...
14:44:48 CST message: [localhost]
kubecni is existed
14:44:48 CST message: [localhost]
downloading amd64 crictl v1.24.0 ...
14:44:50 CST message: [localhost]
crictl is existed
14:44:50 CST message: [localhost]
downloading amd64 etcd v3.4.13 ...
14:44:50 CST message: [localhost]
etcd is existed
14:44:50 CST message: [localhost]
downloading amd64 containerd 1.6.4 ...
14:44:54 CST message: [localhost]
containerd is existed
14:44:54 CST message: [localhost]
downloading amd64 runc v1.1.1 ...
14:44:54 CST message: [localhost]
runc is existed
14:44:54 CST message: [localhost]
downloading amd64 calicoctl v3.26.1 ...
14:44:56 CST message: [localhost]
calicoctl is existed
14:44:56 CST success: [LocalHost]
14:44:56 CST [ConfigureOSModule] Get OS release
14:44:57 CST success: [node1]
14:44:57 CST success: [node3]
14:44:57 CST success: [node2]
14:44:57 CST success: [control]
14:44:57 CST [ConfigureOSModule] Prepare to init OS
14:45:01 CST success: [node2]
14:45:01 CST success: [control]
14:45:01 CST success: [node1]
14:45:01 CST success: [node3]
14:45:01 CST [ConfigureOSModule] Generate init os script
14:45:01 CST success: [control]
14:45:01 CST success: [node3]
14:45:01 CST success: [node1]
14:45:01 CST success: [node2]
14:45:01 CST [ConfigureOSModule] Exec init os script
14:45:03 CST stdout: [node3]
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
net.core.netdev_max_backlog = 65535
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 1048576
net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_max_tw_buckets = 1048576
net.ipv4.tcp_max_orphans = 65535
net.ipv4.udp_rmem_min = 131072
net.ipv4.udp_wmem_min = 131072
net.ipv4.conf.all.arp_accept = 1
net.ipv4.conf.default.arp_accept = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
vm.max_map_count = 262144
vm.swappiness = 0
vm.overcommit_memory = 0
fs.inotify.max_user_instances = 524288
fs.inotify.max_user_watches = 524288
fs.pipe-max-size = 4194304
fs.aio-max-nr = 262144
kernel.pid_max = 65535
kernel.watchdog_thresh = 5
kernel.hung_task_timeout_secs = 5
14:45:03 CST stdout: [node1]
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
net.core.netdev_max_backlog = 65535
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 1048576
net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_max_tw_buckets = 1048576
net.ipv4.tcp_max_orphans = 65535
net.ipv4.udp_rmem_min = 131072
net.ipv4.udp_wmem_min = 131072
net.ipv4.conf.all.arp_accept = 1
net.ipv4.conf.default.arp_accept = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
vm.max_map_count = 262144
vm.swappiness = 0
vm.overcommit_memory = 0
fs.inotify.max_user_instances = 524288
fs.inotify.max_user_watches = 524288
fs.pipe-max-size = 4194304
fs.aio-max-nr = 262144
kernel.pid_max = 65535
kernel.watchdog_thresh = 5
kernel.hung_task_timeout_secs = 5
14:45:03 CST stdout: [control]
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
net.core.netdev_max_backlog = 65535
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 1048576
net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_max_tw_buckets = 1048576
net.ipv4.tcp_max_orphans = 65535
net.ipv4.udp_rmem_min = 131072
net.ipv4.udp_wmem_min = 131072
net.ipv4.conf.all.arp_accept = 1
net.ipv4.conf.default.arp_accept = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
vm.max_map_count = 262144
vm.swappiness = 0
vm.overcommit_memory = 0
fs.inotify.max_user_instances = 524288
fs.inotify.max_user_watches = 524288
fs.pipe-max-size = 4194304
fs.aio-max-nr = 262144
kernel.pid_max = 65535
kernel.watchdog_thresh = 5
kernel.hung_task_timeout_secs = 5
14:45:03 CST stdout: [node2]
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_local_reserved_ports = 30000-32767
net.core.netdev_max_backlog = 65535
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 1048576
net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 2048
net.ipv4.neigh.default.gc_thresh3 = 4096
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_max_tw_buckets = 1048576
net.ipv4.tcp_max_orphans = 65535
net.ipv4.udp_rmem_min = 131072
net.ipv4.udp_wmem_min = 131072
net.ipv4.conf.all.arp_accept = 1
net.ipv4.conf.default.arp_accept = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
vm.max_map_count = 262144
vm.swappiness = 0
vm.overcommit_memory = 0
fs.inotify.max_user_instances = 524288
fs.inotify.max_user_watches = 524288
fs.pipe-max-size = 4194304
fs.aio-max-nr = 262144
kernel.pid_max = 65535
kernel.watchdog_thresh = 5
kernel.hung_task_timeout_secs = 5
14:45:03 CST success: [node3]
14:45:03 CST success: [node1]
14:45:03 CST success: [control]
14:45:03 CST success: [node2]
14:45:03 CST [ConfigureOSModule] configure the ntp server for each node
14:45:03 CST skipped: [node3]
14:45:03 CST skipped: [node1]
14:45:03 CST skipped: [node2]
14:45:03 CST skipped: [control]
14:45:03 CST [KubernetesStatusModule] Get kubernetes cluster status
14:45:03 CST success: [control]
14:45:03 CST [InstallContainerModule] Sync containerd binaries
14:45:04 CST skipped: [control]
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [node2]
14:45:04 CST skipped: [node3]
14:45:04 CST [InstallContainerModule] Sync crictl binaries
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [control]
14:45:04 CST skipped: [node2]
14:45:04 CST skipped: [node3]
14:45:04 CST [InstallContainerModule] Generate containerd service
14:45:04 CST skipped: [node3]
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [control]
14:45:04 CST skipped: [node2]
14:45:04 CST [InstallContainerModule] Generate containerd config
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [node3]
14:45:04 CST skipped: [control]
14:45:04 CST skipped: [node2]
14:45:04 CST [InstallContainerModule] Generate crictl config
14:45:04 CST skipped: [node3]
14:45:04 CST skipped: [node2]
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [control]
14:45:04 CST [InstallContainerModule] Enable containerd
14:45:04 CST skipped: [node3]
14:45:04 CST skipped: [node1]
14:45:04 CST skipped: [node2]
14:45:04 CST skipped: [control]
14:45:04 CST [ETCDPreCheckModule] Get etcd status
14:45:04 CST success: [control]
14:45:04 CST [CertsModule] Fetch etcd certs
14:45:04 CST success: [control]
14:45:04 CST [CertsModule] Generate etcd Certs
[certs] Using existing ca certificate authority
[certs] Using existing admin-control certificate and key on disk
[certs] Using existing member-control certificate and key on disk
[certs] Using existing node-control certificate and key on disk
14:45:05 CST success: [LocalHost]
14:45:05 CST [CertsModule] Synchronize certs file
14:45:06 CST success: [control]
14:45:06 CST [CertsModule] Synchronize certs file to master
14:45:06 CST skipped: [control]
14:45:06 CST [InstallETCDBinaryModule] Install etcd using binary
14:45:14 CST success: [control]
14:45:14 CST [InstallETCDBinaryModule] Generate etcd service
14:45:14 CST success: [control]
14:45:14 CST [InstallETCDBinaryModule] Generate access address
14:45:14 CST success: [control]
14:45:14 CST [ETCDConfigureModule] Health check on exist etcd
14:45:14 CST skipped: [control]
14:45:14 CST [ETCDConfigureModule] Generate etcd.env config on new etcd
14:45:15 CST success: [control]
14:45:15 CST [ETCDConfigureModule] Refresh etcd.env config on all etcd
14:45:15 CST success: [control]
14:45:15 CST [ETCDConfigureModule] Restart etcd
14:45:22 CST success: [control]
14:45:22 CST [ETCDConfigureModule] Health check on all etcd
14:45:22 CST success: [control]
14:45:22 CST [ETCDConfigureModule] Refresh etcd.env config to exist mode on all etcd
14:45:22 CST success: [control]
14:45:22 CST [ETCDConfigureModule] Health check on all etcd
14:45:22 CST success: [control]
14:45:22 CST [ETCDBackupModule] Backup etcd data regularly
14:45:22 CST success: [control]
14:45:22 CST [ETCDBackupModule] Generate backup ETCD service
14:45:22 CST success: [control]
14:45:22 CST [ETCDBackupModule] Generate backup ETCD timer
14:45:23 CST success: [control]
14:45:23 CST [ETCDBackupModule] Enable backup etcd service
14:45:23 CST success: [control]
14:45:23 CST [InstallKubeBinariesModule] Synchronize kubernetes binaries
14:47:09 CST success: [control]
14:47:09 CST success: [node1]
14:47:09 CST success: [node3]
14:47:09 CST success: [node2]
14:47:09 CST [InstallKubeBinariesModule] Change kubelet mode
14:47:10 CST success: [node1]
14:47:10 CST success: [node2]
14:47:10 CST success: [node3]
14:47:10 CST success: [control]
14:47:10 CST [InstallKubeBinariesModule] Generate kubelet service
14:47:10 CST success: [control]
14:47:10 CST success: [node1]
14:47:10 CST success: [node2]
14:47:10 CST success: [node3]
14:47:10 CST [InstallKubeBinariesModule] Enable kubelet service
14:47:17 CST success: [control]
14:47:17 CST success: [node1]
14:47:17 CST success: [node3]
14:47:17 CST success: [node2]
14:47:17 CST [InstallKubeBinariesModule] Generate kubelet env
14:47:17 CST success: [node3]
14:47:17 CST success: [control]
14:47:17 CST success: [node1]
14:47:17 CST success: [node2]
14:47:17 CST [InitKubernetesModule] Generate kubeadm config
14:47:25 CST success: [control]
14:47:25 CST [InitKubernetesModule] Generate audit policy
14:47:25 CST skipped: [control]
14:47:25 CST [InitKubernetesModule] Generate audit webhook
14:47:25 CST skipped: [control]
14:47:25 CST [InitKubernetesModule] Init cluster using kubeadm
14:47:30 CST stdout: [control]
W1116 14:47:25.450344    5586 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:25.456621    5586 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:25.467627    5586 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:29.274943    5594 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:29+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
14:47:31 CST stdout: [control]
[preflight] Running pre-flight checks
W1116 14:47:30.664190    5610 removeetcdmember.go:85] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W1116 14:47:30.684346    5610 cleanupnode.go:123] [reset] Failed to evaluate the "/var/lib/kubelet" directory. Skipping its unmount and cleanup: lstat /var/lib/kubelet: no such file or directory
W1116 14:47:30.999757    5610 cleanupnode.go:94] [reset] Failed to remove containers: output: E1116 14:47:30.995639    5615 remote_runtime.go:356] "ListPodSandbox with filter from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService" filter="&PodSandboxFilter{Id:,State:nil,LabelSelector:map[string]string{},}"
time="2023-11-16T14:47:30+08:00" level=fatal msg="listing pod sandboxes: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[reset] Deleting contents of directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: []

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
14:47:31 CST message: [control]
init kubernetes cluster failed: Failed to exec command: sudo -E /bin/bash -c "/usr/local/bin/kubeadm init --config=/etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=FileExisting-crictl,ImagePull" 
W1116 14:47:25.450344    5586 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:25.456621    5586 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:25.467627    5586 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:29.274943    5594 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:29+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher: Process exited with status 1
14:47:31 CST retry: [control]
14:47:36 CST stdout: [control]
W1116 14:47:36.160307    5620 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:36.163936    5620 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:36.167566    5620 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:36.249148    5628 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:36+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
14:47:36 CST stdout: [control]
[preflight] Running pre-flight checks
W1116 14:47:36.683819    5644 removeetcdmember.go:85] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W1116 14:47:36.707326    5644 cleanupnode.go:123] [reset] Failed to evaluate the "/var/lib/kubelet" directory. Skipping its unmount and cleanup: lstat /var/lib/kubelet: no such file or directory
W1116 14:47:36.773397    5644 cleanupnode.go:94] [reset] Failed to remove containers: output: E1116 14:47:36.770078    5650 remote_runtime.go:356] "ListPodSandbox with filter from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService" filter="&PodSandboxFilter{Id:,State:nil,LabelSelector:map[string]string{},}"
time="2023-11-16T14:47:36+08:00" level=fatal msg="listing pod sandboxes: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[reset] Deleting contents of directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: []

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
14:47:36 CST message: [control]
init kubernetes cluster failed: Failed to exec command: sudo -E /bin/bash -c "/usr/local/bin/kubeadm init --config=/etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=FileExisting-crictl,ImagePull" 
W1116 14:47:36.160307    5620 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:36.163936    5620 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:36.167566    5620 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:36.249148    5628 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:36+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher: Process exited with status 1
14:47:36 CST retry: [control]
14:47:42 CST stdout: [control]
W1116 14:47:41.904954    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.915256    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.921761    5656 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:41.993124    5664 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:41+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
14:47:42 CST stdout: [control]
[preflight] Running pre-flight checks
W1116 14:47:42.195015    5679 removeetcdmember.go:85] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W1116 14:47:42.201486    5679 cleanupnode.go:123] [reset] Failed to evaluate the "/var/lib/kubelet" directory. Skipping its unmount and cleanup: lstat /var/lib/kubelet: no such file or directory
W1116 14:47:42.213294    5679 cleanupnode.go:94] [reset] Failed to remove containers: output: E1116 14:47:42.212507    5684 remote_runtime.go:356] "ListPodSandbox with filter from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService" filter="&PodSandboxFilter{Id:,State:nil,LabelSelector:map[string]string{},}"
time="2023-11-16T14:47:42+08:00" level=fatal msg="listing pod sandboxes: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[reset] Deleting contents of directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: []

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
14:47:42 CST message: [control]
init kubernetes cluster failed: Failed to exec command: sudo -E /bin/bash -c "/usr/local/bin/kubeadm init --config=/etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=FileExisting-crictl,ImagePull" 
W1116 14:47:41.904954    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.915256    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.921761    5656 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:41.993124    5664 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:41+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher: Process exited with status 1
14:47:42 CST failed: [control]
error: Pipeline[CreateClusterPipeline] execute failed: Module[InitKubernetesModule] exec failed: 
failed: [control] [KubeadmInit] exec failed after 3 retries: init kubernetes cluster failed: Failed to exec command: sudo -E /bin/bash -c "/usr/local/bin/kubeadm init --config=/etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=FileExisting-crictl,ImagePull" 
W1116 14:47:41.904954    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.915256    5656 common.go:84] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta2". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
W1116 14:47:41.921761    5656 utils.go:69] The recommended value for "clusterDNS" in "KubeletConfiguration" is: [10.233.0.10]; the provided value is: [169.254.25.10]
[init] Using Kubernetes version: v1.25.10
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: E1116 14:47:41.993124    5664 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-11-16T14:47:41+08:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher: Process exited with status 1

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.

Unfortunately, an error has occurred:
        timed out waiting for the condition

This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
        - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher: Process exited with status 1

解决：执行以下命令后再执行初始化命令

rm /etc/containerd/config.toml 
systemctl restart containerd

参考1
参考2

报错"Initial timeout of 40s passed"

使用systemctl status kubelet和journalctl -xeu kubelet命令查看具体错误，根据具体错误进行排错

root@control:~# journalctl -xeu kubelet
Nov 16 15:40:28 control kubelet[9539]: E1116 15:40:28.705824    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:28 control kubelet[9539]: E1116 15:40:28.806741    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:28 control kubelet[9539]: E1116 15:40:28.908486    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.009059    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.109998    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.210834    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.312573    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.412798    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.514513    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: W1116 15:40:29.546882    9539 reflector.go:424] vendor/k8s.io/client-go/informers/factory.go:134: failed to list *v1.Node: Get "https://l>
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.547094    9539 reflector.go:140] vendor/k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to lis>
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.615722    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.716488    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.817362    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:29 control kubelet[9539]: E1116 15:40:29.918731    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:30 control kubelet[9539]: E1116 15:40:30.019897    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"
Nov 16 15:40:30 control kubelet[9539]: E1116 15:40:30.121739    9539 kubelet.go:2448] "Error getting node" err="node \"control\" not found"

由于 Kubernetes 暂不支持大写 NodeName， hostname 中包含大写字母将导致后续安装过程无法正常结束。所以一定要注意hostname的大小写，我这里第一使用大写，导致一致无法成功创建集群，直到查看Kubekey文档时才发现。
因为我的系统初始环境已经安装好docker，但是创建集群的配置里容器是containerd。暂时不确定上面的错误是因为hostname引起的还是docker引起的，重新换了干净的没有装docker的系统并使用小写的hostname,集群正确创建

KubeKey文档

KubeKey是一个开源的轻量级工具，用于部署Kubernetes集群。它提供了一种灵活、快速、方便的方式来安装Kubernetes/K3s、Kubernetes/K3s和KubeSphere，以及相关的云原生附加组件。它也是扩展和升级集群的有效工具。使用KubeKey来创建Kubernetes集群确实方便了很多