容器版Jenkins连接K8s

目的

本文详解介绍了容器版jenkins连接k8s的配置和使用详情。特别注意：必须用谷歌浏览器，而且非容器版jenkins是无法安装kubernetes插件的，所以无法连接k8s。搭建k8s集群教程

环境

操作系统版本: Ubuntu 18.04 TLS

软件版本: Jenkins 2.121.2

教程

安装k8s插件

系统管理->管理插件->可选插件, 搜索kubernetes plugin（没有选择kubernetes）并选择安装。

配置插件连接k8s集群

点击系统管理->系统设置->添加一个云, 在下拉菜单中选择kubernets并添加，如下图所示：

云kubernetes配置

注：Name值任意添加，Kubernetes URL值添加K8S apiserver连接地址和端口，jenkins URL值添加jenkins UI访问地址和端口,如下图所示：

添加云pod template并配置

参考教程

配置云连接K8S集群的验证文件

#在jenkins所在的node节点上操作

#创建目录
mkdir -p /opt/crt/

#获取K8S的/root/.kube/config文件
#获取/root/.kube/config中certificate-authority-data的内容并转化成base64 encoded文件

[root@test2 ~]# echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR2akNDQXFhZ0F3SUJBZ0lVYTdJRGZlOENZdmc0ZGpBR0cxQS81WHBLZVRNd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEU0TURjeE9UQTBNVEl3TUZvWERUSXpNRGN4T0RBME1USXdNRm93WlRFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RERBSwpCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByZFdKbGNtNWxkR1Z6Ck1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBdTRxK1l2TDZMMjRZcEx3Y0cvMmoKdllFTTY5QmIxMmFzZFBtNXBLekMzc3o5Q2JnZ3llb2ZzZFNxSmFIVkNHRkFmNjBEN3J5elhPMVgxY1RGaGdYdQp5SGtrcnBLdGVyNm1aNXpRVGxrQXlyM1AyOXd0NXhJRHJINkhKOE9FcGZNYWlCRlkrcHhTKzJicEZNVUJ3UTcwCksxVE5tWXFsN0lIOEt5V290UXhKQURtQS9VWUpDWHgwNnB5bVlvRzRmcTVjeUVJK3hBSkRsZ3l5blZpc2Y1cEkKUGo2dEl3SHc1UzVGbGVNUE04WVFMSE5JNUlwUEYvR0txNmdaYldLTHFOamNvL2dVTExXSThsWm5mSlVSbE16VwpLa05XYkxETktwSzZOOG5jc09Yc2lFTjNvMW0zeGg2RjlJUWsyWU95d3hTM1dvWTlUQWp3d0VxdTV0bUtVZ1lhCjN3SURBUUFCbzJZd1pEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0VnWURWUjBUQVFIL0JBZ3dCZ0VCL3dJQkFqQWQKQmdOVkhRNEVGZ1FVQ1QyM3ora3J5clcvNHJGblR4cFlBalJqa253d0h3WURWUjBqQkJnd0ZvQVVDVDIzeitrcgp5clcvNHJGblR4cFlBalJqa253d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFHNmRWZitPTnlUcVJzRlBVaWRxClVvODJSUSs3RytVOWdxN3VpRjFyamx6cms4d0JxQ0J3SEVwOGFKbzhKa1hBN3JNZUptVk9hcmJPK1lsempmbTcKUlM5QnJNV3I0ZEt5TUw0UTIvWHgreCs5Um1GemxabmZoTFR4SE9XTTJiam9HTjliNk90RnF1VmFqaWtjU0Zkdwp3cTF2cS9Ud05ock41aXlIRko1V3h6YXRpa3BlVmZQdnhsS0xKbmpQOGhxeVh2RWxYcEIydnV1ZStJeEdmUWJiCjROQUFJcVIwOUsvS1FVQ3dQK0lQNjlxK0FMazJIWWRrNVB0SmpOMnpOSEw5SUppWHVQUGViZGtSYVNnMGIvZzMKc3BNeFZTY05jNDR3MGFyQ0ZFU2Foa0tMbHJFRGNjbjcxSVpPb2dEMStaMHgycFB4RDc5OWNlQjlxMUtsNnhOWQp6T2s9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K | base64 -d > /opt/crt/ca.crt

#获取/root/.kube/config中client-certificate-data和client-key-data的内容并转化成base64 encoded文件

[root@test2 ~]# echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQzVENDQXNXZ0F3SUJBZ0lVSkw2RzR5YjlsaGwvWlIyT01laENmbVBOK2s4d0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEU0TURjeE9UQTJORFF3TUZvWERUSTRNRGN4TmpBMk5EUXdNRm93YXpFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RnpBVgpCZ05WQkFvVERuTjVjM1JsYlRwdFlYTjBaWEp6TVE4d0RRWURWUVFMRXdaVGVYTjBaVzB4RGpBTUJnTlZCQU1UCkJXRmtiV2x1TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2Ui9GZ2krKzJMdmUKU0ttNU11TG9DeVZ6aUVsOEVCVFhIS0tacWczMG5iOWgvUHAxZEV2a3JFbHFtOFJHN3JOOGFFRkkyZi9mcHF4SwpMdFlTWmVYTjEybWFRQTBiV1JlSG1nYWxnTzBZbVpYV1psc0FOR0xLcEtJOEF4czVXeDB3WlNIdWlxeFlCOXlnCkdPR0dhMW41SmZPUDBRWFZNeTVDZlcweWtLRnZmbDJ6TENwSnRNTzJtdWZxR20zM2wrd0l0SlNKaEJUdUc0RDUKa0pnOFFSNUZzUDk1eXF2RWNnT1NrNUZpNFVKZkcwakY1Rnd3UGt1ekhnTmNFQ29YMVNrTlkrZFJ3aSt5OUNZYQpSdWcvbXBrd1F5RWFBVW5BbzNhaTlWTFd5dnRkc3h1YlFMMUJMUGczZE4xTFRPeUxFaGMzZVJNdWFUR0dhNDc4CmwxUWJYeTV2K3dJREFRQUJvMzh3ZlRBT0JnTlZIUThCQWY4RUJBTUNCYUF3SFFZRFZSMGxCQll3RkFZSUt3WUIKQlFVSEF3RUdDQ3NHQVFVRkJ3TUNNQXdHQTFVZEV3RUIvd1FDTUFBd0hRWURWUjBPQkJZRUZCRFdIRTIxUzhLSApHeFBCT3lNem1LQjZqc1V3TUI4R0ExVWRJd1FZTUJhQUZBazl0OC9wSzhxMXYrS3haMDhhV0FJMFk1SjhNQTBHCkNTcUdTSWIzRFFFQkN3VUFBNElCQVFBRkpaSFpYbmR3ZGJQQVcxSUtMTFUzNlRTUm9pUXVIVk5nWWwzZFRJcVYKMzllTFcxSFJ5djIzdTlOSkhXRkhyZHBWbjNWM0ZTR1lsUW5jRSsxU1o5MUFWZXhGcDMxeWNnbVo0WEdjZ0pVZwphcExoZFJOdHNSVmpHKzF0OFplL2VCdHVXYXFlNXluZUFtT3ZzdmpPcDU5cVpDakxSbDk0YWZ4WVYrVlNiVFByCnAvZHRCcHh3MUlOK3puajh1SXRnRDhQTEpzUXViYWsyeTU2QThWUTJCTmVkSG5ZK3lVaC9NVHgzQ1BkSlB3OTUKbnk3T0ovaWNCV1drbS95Qy9weURqSFhEczdtNVkvWG80VnJ2YUQwd05ucllOWWluTTh0VHRReU1zUEpBaVlLYQpmZDAycEs1eGFzVjUweWdpL3AxaUl4MzEybHMxaWc1eG8zRjAyb2xocWtNKwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== | base64 -d > /opt/crt/client.crt


[root@test2 ~]# echo LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdlIvRmdpKysyTHZlU0ttNU11TG9DeVZ6aUVsOEVCVFhIS0tacWczMG5iOWgvUHAxCmRFdmtyRWxxbThSRzdyTjhhRUZJMmYvZnBxeEtMdFlTWmVYTjEybWFRQTBiV1JlSG1nYWxnTzBZbVpYV1psc0EKTkdMS3BLSThBeHM1V3gwd1pTSHVpcXhZQjl5Z0dPR0dhMW41SmZPUDBRWFZNeTVDZlcweWtLRnZmbDJ6TENwSgp0TU8ybXVmcUdtMzNsK3dJdEpTSmhCVHVHNEQ1a0pnOFFSNUZzUDk1eXF2RWNnT1NrNUZpNFVKZkcwakY1Rnd3ClBrdXpIZ05jRUNvWDFTa05ZK2RSd2kreTlDWWFSdWcvbXBrd1F5RWFBVW5BbzNhaTlWTFd5dnRkc3h1YlFMMUIKTFBnM2ROMUxUT3lMRWhjM2VSTXVhVEdHYTQ3OGwxUWJYeTV2K3dJREFRQUJBb0lCQUQzbVNqVEQvOGpjSkhMUAo2aWUza0k4bFlOejRnRHliTlpUUHUwK25aYXJEMndSN3pUbVZKWEVtVGxoUk00NG8vTXo2b1NlSTBlQ3hmMDQ1CkRxaC9RSklDcEZQV2RsOEFqb2RoS1lZN0U5UWc4SjFycDNOOTZpbGNXQndFS3craFRCZXR0Vzk1M1E0bHJkaTIKNTlIM0RzN1hHdmtrMlpUNHpSWlVTVHFCUEFhMWY4c2ZuSzFicHNwOEd2OWxZQTZTa1FKNG5XSDIvZXlMVXdZNQpoTXVDZW94TGc3STVIN24vOGpJWGV0Wkk0RW96MkpZa0dJN3FYZVNtdkE5U0JadFVBNGlzSVFSK0lzdVRManpXCmNrOGVrWkpkZU5PeFhrRXpaTC9iSUxwQ1ZZWVdRa2c1R2pCaFdyVGxqWGpWZzltcDJNeENLRlEwT0VGM3FZU0kKa1lZWkFSRUNnWUVBN3VmWnlPZ0o4RFh2NTdUNlFFNmdyM3U2YXdKTXZxcWhJU1l6RnNlRXJ3U0Z0b3hGY2Y1eQpWWHFIOElEUkFDMmRhMjFvV01GcDBrQUN4cjVBK014R2pxUXRTQXZSRjJES09TQk1HUHJiOG9YaktNNytZOHp5CmdueXhaMm5yYjFOdGRqN21Nc1B4UnU0V3FLeWdLRnE3a1YzOFM3Sm5aN0xRQUxiSGJmOUFMSGNDZ1lFQXlxZ00KVlpXYmIyTjZhaGRZdi8vUkVmdXo5Q0c2RGxETmhNNW0xRFlJdjNzMVZEcmhqNHM1MDR2c3JFbHZwVFlLT1o3VQppbUNjSDBZUXpsZnN2UUsxQTNSNFpvNmpFdWNMcUE0S1JWbGlyQk9jODhqSi9hUEdHa1lzQ2NBUzZ0d2pZdy9oCjliQUxVWTJISTFsOVlhYmtSYXdEYk10bi9MWWZFaExMTU5EZjdaMENnWUVBdWw5WXdLaDBDRmFyZndEcU1QeWwKMGdBZDM1ajlzY2greHROOEM0cytjU0tBQlhiTVBpK1hsaU51cFNwNDRVQzBpN2ZnTFUxRmRtWEZSTEhyRWF5YQpabkNoZXBEdFh1VjlISytiYmVsVmFJOFdOU0cxeHJsOWZsbzBNMDZvQWtMOUk3L1I2VXgranl6eHRFaG04TlJICmV4SHMza2lnN243S1VhUkZWQVJLVmVVQ2dZQW9HTjN2NVIwUENnakRpd0VGWkRGU3RKR2pnVFRWOWtqanVROEIKZC90OUgzeXF3TWUyWmg2MzY1eVZiaVpIOHd4TTRFOC9YZVFtRCsvdFU5cEVmNCtmTW1GTU1YYTBtOEJqclB0OQpRelZSeE1PdVBKRXl2VC9LSFE1RGs1eHFtY25xcE03WmxNNTRnVjgyc0ZNdGloN3FaaUY3V2plbCtjYm1CWS9zCmhiZDR4UUtCZ0hsYkNDeFhEOFVVSktIQjNraWVnSHYrOTZlN2oxUHNUTGdjZG9adXp6ejZZK0ZERGdITzk0ZWoKeXdndm12cW1aZmZ0cmdxK1BmOW54WmRGaHc1WmxjbDRtZTltalB6UFc5WFMyVHVsandGZytabDNrNjRESEo3VwpmM2VwOGIvSlpHTmFOWFQ3akF2ZnNhclhYNVQzNUJpS3J4Z0tFOTdvMStFbVhBVENaMXprCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== | base64 -d > /opt/crt/client.key

生产Client P12认证文件cert.pfx，并下载至本地windows

openssl pkcs12 -export -out /opt/crt/cert.pfx -inkey /opt/crt/client.key -in /opt/crt/client.crt -certfile /opt/crt/ca.crt
Enter Export Password: 123
Verifying - Enter Export Password: 123

sz /opt/crt/cert.pfx

#注：自定义一个password并牢记

在云k8s中添加凭证

添加凭证–> 首先密码填写123–> 类型 –> Certificate –> Upload PKCS#12 certificate –> 上传证书 –> 选择文件–> cert –> 打开 –> 上传 –> 添加 –> 选择凭证

注：Upload certificate上次刚生成并下载至本地的cert.pfx文件，Password值添加生成cert.pfx文件时输入的密钥

添加命名空间

Kubernetes 命名空间中的值添加/root/.kube/config文件中cluster部分中name的内容（否则连接失败）

测试连接kubernetes集群

配置jenkins jnlp代理端口

系统管理->全局安全配置中的”代理”项,指定端口为50000

拍错

显示证书不对

Error testing connection https://192.168.0.91:6443: Get Key failed: Given final block not properly padded. Such issues can arise if a bad key is used during decryption

解决：

是因为添加平局那一界面的下面没有输入之前设置的密码（123）。输入密码、再次导入即可解决

显示无法连接

解决：

cert.pfx 可能没有生成好；或者ca.cert没有生成好，有空格，重新生成一遍就好