1. 介绍
Gitlab+Jenkins+Docker+Harbor+K8S集群的CICD搭建教程- 在搭建好的
CICD平台上持续集成部署hexo博客系统 - 其中
Gitlab+Jenkins+Harbor都是通过容器化部署 篇幅有限,关于CD环境
k8s集群这里用之前部署好的,并且已经做了kubeconfig证书下面为涉及到的机器:
| 用到的机器 | ip |
|---|---|
| 客户机 | 本地物理机 |
| Gitlab+Jenkins+Docker | 192.168.173.10 |
| docker镜像仓库:harbor | 192.168.173.20 |
| k8s集群-master节点 | 192.168.173.30 |
| k8s集群-node节点 | 192.168.173.40 |
| k8s集群-node节点 | 192.168.173.50 |
- 这里客户机用本地的IDE持续编码,然后push代码到gitlab。
- gitlab中的web钩子触发jenkins中配置好的构建触发器,通过shell命令拉取gitlab仓库中的代码。
- 通过拉取的应用源码和Dockerfile文件来构建应用镜像。
- 构建完成后将应用镜像push到harbor私有镜像仓库。
- 通过shell命令的方式在jenkins中用kubelet客户端将镜像从私有仓库拉取到k8s集群并更新其deploy中的镜像。
- 默认deploy更新副本的方式为滚动更新,整个流程中,只有客户机push代码是手手动的方式,其他全是自动。
2. CICD环境搭建
CI即为持续集成(Continue Integration,简称CI),用通俗的话讲,就是持续的整合版本库代码编译后制作应用镜像。建立有效的持续集成环境可以减少开发过程中一些不必要的问题、提高代码质量、快速迭代等,
- 常用的工具和平台有:
- Jenkins:基于Java开发的一种持续集成工具,用于监控持续重复的工作,旨在提供一个开放易用的软件平台,使软件的持续集成变成可能。
- Bamboo: 是一个企业级商用软件,可以部署在大规模生产环境中。
CD即持续交付Continuous Delivery和持续部署Continuous Deployment,用通俗的话说,即可以持续的部署到生产环境给客户使用,这里分为两个阶段,持续交付我理解为满足上线条件的过程,但是没有上线,持续部署,即为上线应用的过程
关于CD环境,我们使用以前搭建好的K8s集群,K8s集群可以实现应用的健康检测,动态扩容,滚动更新等优点,关于K8s集群的搭建,小伙伴可以看看我的其他文章
2.1 docker环境
2.2 GitLab配置
GitLab是一个基于Git的版本控制平台,,提供了Git仓库管理、代码审查、问题跟踪、活动反馈和wiki
[root@jenkins ~]# docker pull beginor/gitlab-ce
- 创建共享卷目录
[root@jenkins ~]# mkdir -p /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data
[root@jenkins ~]# chmod 777 /data/gitlab/etc/ /data/gitlab/log/ /data/gitlab/data/
- 创建gitlab容器
[root@jenkins ~]# docker run -itd \
--name=gitlab --restart=always \
--privileged=true \
-p 8443:443 -p 80:80 -p 222:22 \
-v /data/gitlab/etc:/etc/gitlab \
-v /data/gitlab/log:/var/log/gitlab \
-v /data/gitlab/data:/var/opt/gitlab beginor/gitlab-ce
[root@jenkins ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c3557530ea88 beginor/gitlab-ce "/assets/wrapper" 11 seconds ago Up 10 seconds (health: starting) 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:222->22/tcp, [::]:222->22/tcp, 0.0.0.0:8443->443/tcp, [::]:8443->443/tcp gitlab
切记:这里的端口要设置成80,要不push项目会提示没有报错,如果宿主机端口被占用,需要把这个端口腾出来
- 关闭容器修改配置文件
[root@jenkins ~]# docker stop gitlab
external_url 'http://192.168.173.10'
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep external_url |grep -Ev ^#
external_url 'http://192.168.173.10'
gitlab_rails['gitlab_ssh_host'] = '192.168.173.10'
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb |grep gitlab_ssh_host
gitlab_rails['gitlab_ssh_host'] = '192.168.173.10'
gitlab_rails[gitlab_shell_ssh_port] = 222
[root@jenkins ~]# cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh
gitlab_rails['gitlab_shell_ssh_port'] = 222
- 修改
/data/gitlab/data/gitlab-rails/etc/gitlab.yml中的host信息
[root@jenkins ~]# vim /data/gitlab/data/gitlab-rails/etc/gitlab.yml
gitlab:
## Web server settings (note: host is the FQDN, do not include http://)
host: 192.168.173.10
port: 80
https: false
- 修改完配置文件之后。直接启动容器
[root@jenkins ~]# docker start gitlab
- 等待一段时间启动,在宿主机所在的物理机访问,
http://192.168.173.10/,会自动跳转到修改密码(root用户),如果密码设置的没有满足一定的复杂性,则会报500,需要重新设置
- 使用root用户登录进系统
- 登录后的页面
- 下面是创建一个项目用于熟悉和测试gitlab
- 将自己的公钥传上去
- 复制公钥
- 随便上传点东西测试,参考代码在网页上
- 上传的代码
C:\Users\aaronxu\Desktop\flask-demo>git init
C:\Users\aaronxu\Desktop\flask-demo>git remote add origin ssh://git@192.168.173.10:222/root/test-repo.git
C:\Users\aaronxu\Desktop\flask-demo>git add .
C:\Users\aaronxu\Desktop\flask-demo>git commit -m "Initial commit"
C:\Users\aaronxu\Desktop\flask-demo>git push -u origin master
The authenticity of host '[192.168.173.10]:222 ([192.168.173.10]:222)' can't be established.
ED25519 key fingerprint is SHA256:RjbHUmJ7hvYyxq4Yu7/TXk1/WW390UpVIyY8f6MAdJ4.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[192.168.173.10]:222' (ED25519) to the list of known hosts.
Enumerating objects: 14, done.
Counting objects: 100% (14/14), done.
Delta compression using up to 16 threads
Compressing objects: 100% (12/12), done.
Writing objects: 100% (14/14), 3.13 KiB | 1.57 MiB/s, done.
Total 14 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To ssh://192.168.173.10:222/root/test-repo.git
* [new branch] master -> master
branch 'master' set up to track 'origin/master'.
- 确认代码上传成功
2.3 Harbor配置
- 这里仓库我们选择harbor,因为有web页面,当然也可以使用 registry
- 首先需要设置selinux、防火墙
[root@harbor ~]# cat /etc/selinux/config | grep -Ev "^#|^$"
SELINUX=disabled
SELINUXTYPE=targeted
[root@harbor ~]# getenforce
Disabled
[root@harbor ~]# systemctl disable --now firewalld
- 安装并启动docker并安装docker-compose
[root@harbor ~]# mv docker-compose-linux-x86_64 /bin/docker-compose
[root@harbor ~]# chmod +x /bin/docker-compose
[root@harbor ~]# docker-compose -v
Docker Compose version v5.0.0
- 解压harbor 安装包:harbor-offline-installer-v2.14.1.tgz,导入相关镜像
[root@harbor ~]# tar xzvf harbor-offline-installer-v2.14.1.tgz
[root@harbor ~]# docker load -i harbor/harbor.v2.14.1.tar.gz
- 修改配置文件
[root@harbor ~]# cd harbor
[root@harbor harbor]# cp harbor.yml.tmpl harbor.yml
[root@harbor harbor]# vim harbor.yml
- 设置主机地址和密码, 把https部分的内容注释掉
[root@harbor harbor]# cat harbor.yml |grep -Ev "^ *#|^ *$"
hostname: 192.168.173.20
...........
harbor_admin_password: Harbor12345
...........
[root@harbor harbor]# ./prepare
prepare base dir is set to /root/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
copy /data/secret/tls/harbor_internal_ca.crt to shared trust ca dir as name harbor_internal_ca.crt ...
ca file /hostfs/data/secret/tls/harbor_internal_ca.crt is not exist
copy to shared trust ca dir as name storage_ca_bundle.crt ...
copy None to shared trust ca dir as name redis_tls_ca.crt ...
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
- 开始安装
[root@harbor harbor]# ./install.sh
.............
✔ ----Harbor has been installed and started successfully.----
- 查看相关的容器
[root@harbor harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
10f0f49ded81 goharbor/harbor-jobservice:v2.14.1 "/harbor/entrypoint.…" 48 seconds ago Up 40 seconds (healthy) harbor-jobservice
d4c0e7b72e22 goharbor/nginx-photon:v2.14.1 "nginx -g 'daemon of…" 48 seconds ago Up 46 seconds (healthy) 0.0.0.0:80->8080/tcp, [::]:80->8080/tcp nginx
1a57e98a9b1f goharbor/harbor-core:v2.14.1 "/harbor/entrypoint.…" 49 seconds ago Up 47 seconds (healthy) harbor-core
deb551d07bc1 goharbor/redis-photon:v2.14.1 "redis-server /etc/r…" 49 seconds ago Up 47 seconds (healthy) redis
5614fe4b2dfd goharbor/harbor-registryctl:v2.14.1 "/home/harbor/start.…" 49 seconds ago Up 47 seconds (healthy) registryctl
ede80c0bf943 goharbor/registry-photon:v2.14.1 "/home/harbor/entryp…" 49 seconds ago Up 47 seconds (healthy) registry
e57caba4f757 goharbor/harbor-portal:v2.14.1 "nginx -g 'daemon of…" 49 seconds ago Up 47 seconds (healthy) harbor-portal
3faf547ef47d goharbor/harbor-db:v2.14.1 "/docker-entrypoint.…" 49 seconds ago Up 47 seconds (healthy) harbor-db
156547ee999a goharbor/harbor-log:v2.14.1 "/bin/sh -c /usr/loc…" 49 seconds ago Up 48 seconds (healthy) 127.0.0.1:1514->10514/tcp harbor-log
- 使用浏览器访问,并且完成登录
- CI服务器的docker配置
- 这里因为我们要在192.168.173.10(CI服务器)上push镜像到192.168.173.20(私仓),所有需要修改CI服务器上的Docker配置。添加仓库地址
[root@jenkins ~]# cat /etc/docker/daemon.json
{
"insecure-registries": ["192.168.173.20"]
}
[root@jenkins ~]# systemctl daemon-reload
[root@jenkins ~]# systemctl restart docker
- 测试一下
[root@jenkins ~]# docker login 192.168.173.20
Username: admin
Password:
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
Login Succeeded
- push一个镜像
[root@jenkins ~]# docker pull nginx
[root@jenkins ~]# docker tag nginx 192.168.173.20/library/nginx
[root@jenkins ~]# docker push 192.168.173.20/library/nginx
2.4 Jenkins配置
- 镜像jenkins拉取
[root@jenkins ~]# docker pull jenkins/jenkins:latest
- 创建共享卷,修改所属组和用户,和容器里相同
- 这里要改成 1000,是因为容器里是以 jenkins 用户的身份去读写数据,而在容器里jenkins 的 uid 是 1000
[root@jenkins ~]# mkdir /jenkins
[root@jenkins ~]# chown 1000:1000 /jenkins
- 创建 jenkins 容器
[root@jenkins ~]# docker run -dit -p 8080:8080 \
-p 50000:50000 \
--name jenkins --privileged=true \
--restart=always \
-v /jenkins:/var/jenkins_home jenkins/jenkins:latest
[root@jenkins ~]# docker ps | grep jenkins
cf6baf3b3ba5 jenkins/jenkins:latest "/usr/bin/tini -- /u…" 26 seconds ago Up 25 seconds 0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp, 0.0.0.0:50000->50000/tcp, [::]:50000->50000/tcp jenkins
- 浏览器访问
- 为了修改配置,关闭 jenkins 容器
[root@jenkins ~]# docker stop jenkins
jenkins
- 修改Jenkins插件下载服务器为国内地址
[root@jenkins ~]# cat /jenkins/hudson.model.UpdateCenter.xml
<?xml version='1.1' encoding='UTF-8'?>
<sites>
<site>
<id>default</id>
<url>https://mirrors.huaweicloud.com/jenkins/update-center.json</url>
</site>
</sites>
- 修改谷歌为百度,因为此处json文件为换行,格式比较混乱,使用jq工具查看比较方便
[root@jenkins ~]# yum -y install jq
[root@jenkins ~]# sed -i s#https://www.google.com/#https://www.baidu.com/#g /jenkins/updates/default.json
[root@jenkins ~]# cat /jenkins/updates/default.json | jq '.connectionCheckUrl'
"https://www.baidu.com/"
[root@jenkins ~]# cat /jenkins/updates/default.json | jq 'keys' # 查看所有的可配置项,也就是json的key
[
"connectionCheckUrl",
"core",
"deprecations",
"generationTimestamp",
"id",
"plugins",
"signature",
"updateCenterVersion",
"warnings"
]
- 重启docker,获取登录密匙,下面初始化的时候用到
[root@jenkins ~]# docker restart jenkins
jenkins
[root@jenkins ~]# cat /jenkins/secrets/initialAdminPassword
fa049883d9a843ab826216069aa7224f
- 需要修改jenkins绑定的docker的启动参数, 让Jenkins可以通过tcp远程操作容器(模拟需要CI的机器在远端,不然本地的话,sock一样可以使用)
[root@jenkins ~]# cat /lib/systemd/system/docker.service |grep -Ev "^#|^$"
.......
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
.......
[root@jenkins ~]# systemctl daemon-reload
[root@jenkins ~]# systemctl restart docker
- 打开Jenkins,完成初始化
- 安装推荐的插件, 如果此处显示离线了,说明上面的Jenkins插件下载服务器失效了,可以重新找一个
- 插件安装完成后,创建管理员用户
- 依次点击
Manage Jenkins->Manage Plugins->AVAILABLE->Search搜索docker、docker-build-step
- 安装完成之后,返回主页
- 将docker运行的服务器添加上去
- 修改镜像库启动参数
[root@harbor ~]# cat /lib/systemd/system/docker.service |grep -Ev "^#|^$"
.......
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
.......
[root@harbor ~]# systemctl daemon-reload
[root@harbor ~]# systemctl restart docker
- 填写docker的参数
- 将构建docker的服务器也关联上去
- jenkins 安全设置
- 后面 gitlab 要和 jenkins 进行联动,所以必须要需要对 jenkins 的安全做一些设置,依次点击 系统管理-全局安全配置-授权策略,勾选”匿名用户具有可读权限”
- 添加 JVM 运行参数 -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true 运行跨站请求访问
[root@jenkins ~]# docker exec -u root -it jenkins bash
root@cf6baf3b3ba5:/# sed -i 's/exec java/exec java -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true/' /usr/local/bin/jenkins.sh
root@cf6baf3b3ba5:/# cat /usr/local/bin/jenkins.sh |grep 'exec java'
root@cf6baf3b3ba5:/# exit
- 下载kubectl客户端工具
- 这里的话我们要通过jenkins上的kubectl客户端连接k8s,所以我们需要安装一个k8s的客户端kubectl,下载k8s客户端
# 此处使用安装kubernetes的kubectl方式安装即可,此处只用到客户端功能
[root@jenkins ~]# systemctl enable kubelet && systemctl start kubelet
[root@jenkins ~]# kubectl version --client=true
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
- 然后拷贝kubeconfig 证书,k8s集群(一主两从)中查看证书位置/etc/kubernetes/admin.conf
[root@jenkins ~]# scp root@node01:/etc/kubernetes/admin.conf .
The authenticity of host 'node01 (192.168.173.30)' can't be established.
ED25519 key fingerprint is SHA256:DnvMlpn/LVtTpk8VRIPsOGjoYWW8AICLR23S5E2jKFA.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'node01' (ED25519) to the list of known hosts.
root@node01's password:
admin.conf 100% 5658 1.1MB/s 00:00
- 拷贝证书和k8s集群客户端工具到jenkins容器内
[root@jenkins ~]# docker cp admin.conf jenkins:/
Successfully copied 7.68kB to jenkins:/
[root@jenkins ~]# docker cp /usr/bin/kubectl jenkins:/usr/bin/
Successfully copied 49.7MB to jenkins:/usr/bin/
- kubectl命令测试,赋予jenkins用户对/admin.conf的权限
[root@jenkins ~]# docker exec -u root -it jenkins bash
root@cf6baf3b3ba5:/# chown jenkins:jenkins /admin.conf
root@cf6baf3b3ba5:/# exit
exit
[root@jenkins ~]# docker exec -it jenkins bash
jenkins@cf6baf3b3ba5:/$ kubectl --kubeconfig=/admin.conf get nodes
NAME STATUS ROLES AGE VERSION
node01 Ready control-plane 4d18h v1.29.2
node02 Ready <none> 4d18h v1.29.2
node03 Ready <none> 4d18h v1.29.2
3. hexo系统CICD实战
3.1 配置hexo高可用
- 要部署Nginx来运行hexo博客系统,hexo编译完后为一堆静态文件,所以我们需要创建一个svc和一个deploy,使用SVC提供服务,使用deploy提供服务能力,使用Nginx+hexo的静态文件构成的镜像
[root@jenkins ~]# mkdir /jenkins/kubernetes
[root@jenkins ~]# cd /jenkins/kubernetes
[root@jenkins kubernetes]# vim nginx.yaml
- nginx.yaml文件
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginxdep
spec:
replicas: 2
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: web
resources:
requests:
cpu: 100m
restartPolicy: Always
- 将nginx部署
[root@jenkins ~]# docker exec -it -u root jenkins bash
root@cf6baf3b3ba5:/# cd /var/jenkins_home/kubernetes/
root@cf6baf3b3ba5:/var/jenkins_home/kubernetes# kubectl apply -f nginx.yaml --kubeconfig=/admin.conf
root@cf6baf3b3ba5:/var/jenkins_home/kubernetes# kubectl --kubeconfig=/admin.conf get pods
NAME READY STATUS RESTARTS AGE
nginxdep-7dc8c7dc56-kxvxl 1/1 Running 0 44s
nginxdep-7dc8c7dc56-qqfzc 1/1 Running 0 44s
root@cf6baf3b3ba5:/var/jenkins_home/kubernetes# kubectl --kubeconfig=/admin.conf get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdep 2/2 2 2 7m59s
- service创建
root@cf6baf3b3ba5:/var/jenkins_home/kubernetes# kubectl --kubeconfig=/admin.conf expose deploy nginxdep --port=8888 --target-port=80 --type=NodePort
service/nginxdep exposed
root@cf6baf3b3ba5:/var/jenkins_home/kubernetes# kubectl --kubeconfig=/admin.conf get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 2d20h
nginxdep NodePort 10.6.216.0 <none> 8888:31446/TCP 81s
- 访问测试
3.2 集群配置私仓地址
- 通过kubectl set命令更新deploy的镜像时,获取的镜像是通过私仓获取的,所以需要在启动参数添加私仓地址
- 需要修改三个节点,因为三个node才是需要去获取镜像的
[root@node01,02,03,harbor ~]# vim /usr/lib/systemd/system/docker.service
......
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.173.20 -H fd:// --containerd=/run/containerd/containerd.sock
.....
[root@node01,02,03,harbor ~]# systemctl daemon-reload
[root@node01,02,03,harbor ~]# systemctl restart docker
3.3 jenkins配置CICD
- jenkins上配置整个CICD流程,从而实现自动化
- 这里的Token为了安全,可以设置复杂一些,同时需要记住访问方式:JENKINS_URL/job/hexo-cicd/build?token=TOKEN_NAME
- 并且记录下来,不要忘记,此处我填入的是
38381f8eee9f40e9a4fece7f64924d5f
- 添加构建步骤——执行shell
- 这一步是将新的网站源码获取到
cd ~
rm -rf blog
git clone http://192.168.173.10/root/blog.git
- 再次增加构建步骤
- 这一步是将下载的源码在容器从进行处理,可以得到能够运行的容器环境
- 本案例就是将前端代码放入nginx的网站根目录,然后构造一个新的镜像
- 再次添加构建步骤
- 这一步是修改kubernetes集群中运行的镜像,使其更新为最新的版本
export KUBECONFIG=/kcl;
/kubectl --kubeconfig=/admin.conf set image deployment/nginxdep *="192.168.173.20/library/blog:${BUILD_NUMBER}"
3.4 Hexo仓库配置
- 新建blog仓库
- 配置 gitlab 和 jenkins 的联动
- 配置触发的钩子
3.5 编写Dockerfile
- 编写Dockerfile,用于将新的前端代码加入nginx的镜像中
# 将Dockerfile文件加入到hexo的源码目录中
FROM 192.168.173.20/library/nginx:latest
MAINTAINER Aaronxu
ADD ./public /usr/share/nginx/html/
EXPOSE 80
CMD ["nginx", "-g","daemon off;"]
3.6 测试
- 本地部署hexo博客,然后将代码推送到仓库中
- 本地找一个空文件夹,然后cmd进入文件夹中
D:\blog>hexo init # 安装hexo请参考官网教程
D:\blog>hexo g
D:\blog>git init
D:\blog>git remote add origin ssh://git@192.168.173.10:222/root/blog.git
D:\blog>git add .
D:\blog>git commit -m "Initial commit"
D:\blog>git push -u origin master
Enumerating objects: 25, done.
Counting objects: 100% (25/25), done.
Delta compression using up to 16 threads
Compressing objects: 100% (18/18), done.
Writing objects: 100% (25/25), 278.72 KiB | 5.36 MiB/s, done.
Total 25 (delta 3), reused 0 (delta 0), pack-reused 0 (from 0)
To ssh://192.168.173.10:222/root/blog.git
* [new branch] master -> master
branch 'master' set up to track 'origin/master'.
- 确保仓库中代码已成功上传
- 确认成功执行
- 查看到pods自动更新的状态
[root@node01 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginxdep-7dc8c7dc56-kxvxl 1/1 Terminating 1 (46h ago) 46h
nginxdep-bbffc4f57-8cnzd 1/1 Running 0 30s
nginxdep-bbffc4f57-kjh7b 1/1 Running 0 58s
- 访问页面,确认成功