在EKS上构建KubeFlow 1.0.2

介绍

在"安装Kubeflow"页面上尝试"云部署"。请按照此处的步骤操作。

*问：为什么不使用GCPA。因为该公司使用AWS。
如果您仅使用k8s，那么GCP非常容易使用。请参阅比较文章。

环境

开发PC(macOS Catalina 10.15.6(19G73))
AWS

程序

AWS CLI安装

安装AWS CLI。有关步骤，请参阅此处。
您已经配置了AWS CLI。有关步骤，请参阅此处。

1 2	$ aws --version aws-cli/1.18.105 Python/3.6.5 Darwin/19.6.0 botocore/1.17.28

安装kubectl

在开发PC上安装kubectl。有关步骤，请参阅此处。可以使用Homebrew安装。

1 2	$ kubectl version Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-18T02:59:13Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}

安装eksctl(0.1.31版或更高版本)和aws-iam-authenticator

安装eksctl，请参阅此处的步骤。可以使用Homebrew安装。

1 2	$ eksctl version 0.24.0

已安装适用于Kubernetes的AWS IAM Authenticator，但如果您安装了AWS CLI版本1.16.156或更高版本，则它将包含在AWS CLI中，因此您无需安装它。请使用aws eks get-token命令。这次，我们将使用AWS CLI。
请参考这里的安装步骤。

EKS群集创建

本文对EKS的解释非常容易理解，因此建议您事先阅读。

使用

eksctl创建集群。根据现有集群的部署概述，截至2020年7月，Kubeflow 1.0仅在Kubernetes版本1.14或1.15上进行了测试。另一方面，根据AWS用户指南，eksctl支持的Kubernetes版本为1.15、1.16和1.17。因此，这次我们将使用同时满足两个要求的1.15版。

最低系统要求中描述了工作节点的要求。

4个CPU
50 GB的存储空间
12 GB内存

使用m5.large作为满足此要求的实例。您可以在此处检查实例规格。卷大小是在创建集群时由命令指定的。

使用以下命令创建集群。这次，我们将创建一个名为kubeflow的集群。我选择了东京作为区域(有关可用区域，请参见此处)。工作节点数为3，卷大小为70GB(建议3个或更多，因为如果工作节点数少，则辅助IP不足，将发生挂起)。另外，由于工作程序节点使用EC2，请使用--managed选项。
该命令的详细信息在EKS评论文章中易于理解。或参考官方的eksctl文档。

1
2
3
4
5
6
7
8
9
10
11

$ eksctl create cluster \
--name kubeflow \
--version 1.15 \
--region ap-northeast-1 \
--nodegroup-name standard-workers \
--node-type m5.large \
--nodes 3 \
--nodes-min 2 \
--nodes-max 3 \
--node-volume-size 70 \
--managed

eksctl创建多个AWS资源以创建使用kubernetes的环境。按类方法撰写的文章很容易理解eksctl在做什么。

以下日志将输出到标准输出，并将开始创建集群。设置群集通常需要10到15分钟。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

[?] eksctl version 0.24.0
[?] using region ap-northeast-1
[?] setting availability zones to [ap-northeast-1a ap-northeast-1c ap-northeast-1d]
[?] subnets for ap-northeast-1a - public:192.168.0.0/19 private:192.168.96.0/19
[?] subnets for ap-northeast-1c - public:192.168.32.0/19 private:192.168.128.0/19
[?] subnets for ap-northeast-1d - public:192.168.64.0/19 private:192.168.160.0/19
[?] nodegroup "standard-workers" will use "ami-0bb8387c124770444" [AmazonLinux2/1.15]
[?] using Kubernetes version 1.15
[?] creating EKS cluster "kubeflow" in "ap-northeast-1" region with un-managed nodes
[?] will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[?] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=ap-northeast-1 --cluster=kubeflow'
[?] CloudWatch logging will not be enabled for cluster "kubeflow" in "ap-northeast-1"
[?] you can enable it with 'eksctl utils update-cluster-logging --region=ap-northeast-1 --cluster=kubeflow'
[?] Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "kubeflow" in "ap-northeast-1"
[?] 2 sequential tasks: { create cluster control plane "kubeflow", 2 sequential sub-tasks: { no tasks, create nodegroup "standard-workers" } }
[?] building cluster stack "eksctl-kubeflow-cluster"
[?] deploying stack "eksctl-kubeflow-cluster"

创建后，检查是否使用以下命令返回了响应。

1
2
3
4
5
6
7

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 19m

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-16T00:04:31Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.11-eks-14f01f", GitCommit:"14f01fe8f04411d5e187b220034ca2117d79f7de", GitTreeState:"clean", BuildDate:"2020-05-23T21:32:47Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}

您可以检查在

控制台中创建的资源。

スクリーンショット 2020-07-26 13.49.52.png

スクリーンショット 2020-07-26 13.49.30.png

スクリーンショット 2020-07-26 13.53.18.png

スクリーンショット 2020-07-26 13.54.08.png

スクリーンショット 2020-07-26 13.56.23.png

スクリーンショット 2020-07-26 13.51.28.png

让我们尝试一下是否可以使用EKS。

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$ kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4
deployment.apps/hello-node created

$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
hello-node 1/1 1 1 72s

$ kubectl get replicasets
NAME DESIRED CURRENT READY AGE
hello-node-bcbcd7f76 1 1 1 2m17s

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-node-bcbcd7f76-skspt 1/1 Running 0 8s

您现在已经创建了一个示例部署。
然后使用kubectl暴露命令将pod暴露给互联网。

1
2
3
4
5
6
7

$ kubectl expose deployment hello-node --type=LoadBalancer --port=8080
service/hello-node exposed

$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-node LoadBalancer 10.100.115.107 xxxxx.ap-northeast-1.elb.amazonaws.com 8080:30979/TCP 31s
kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 40m

当您访问

xxxxx.ap-northeast-1.elb.amazonaws.com:8080时，将返回以下响应，您可以看到服务器已打开。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

CLIENT VALUES:
client_address=xx.xx.xx.xx
command=GET
real path=/
query=nil
request_version=1.1
request_uri=http://xxxxx.ap-northeast-1.elb.amazonaws.com:8080/

SERVER VALUES:
server_version=nginx: 1.10.0 - lua: 10001

HEADERS RECEIVED:
accept=text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding=gzip, deflate
accept-language=ja-JP,ja;q=0.9,en-US;q=0.8,en;q=0.7
cache-control=max-age=0
connection=keep-alive
host=xxxxx.ap-northeast-1.elb.amazonaws.com:8080
upgrade-insecure-requests=1
user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36
BODY:
-no body in request-

如果可以确认，请删除资源。

1
2
3
4
5

$ kubectl delete service hello-node
service "hello-node" deleted

$ kubectl delete deployment hello-node
deployment.extensions "hello-node" deleted

您可以在此处检查此Hello World。

kubernetes集群现已准备就绪。

KubeFlow施工

请按照以下步骤进行安装。

首先，安装kfctl。由于是Mac，因此平台是darwin。

1
2
3
4

$ wget https://github.com/kubeflow/kfctl/releases/download/v1.0.2/kfctl_v1.0.2-0-ga476281_darwin.tar.gz
$ tar -xvf kfctl_v1.0.2-0-ga476281_darwin.tar.gz
$ mv kfctl /usr/local/bin/kfctl
$ rm kfctl_v1.0.2-0-ga476281_darwin.tar.gz

检查是否通过了

路径，然后检查版本。

1 2	$ kfctl version kfctl v1.0.2-0-ga476281

设置安装所需的变量。

1
2
3
4
5

$ export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v1.0-branch/kfdef/kfctl_aws.v1.0.2.yaml"
$ export AWS_CLUSTER_NAME=kubeflow
$ export KF_NAME=${AWS_CLUSTER_NAME}
$ export BASE_DIR=~/kubeflow
$ export KF_DIR=${BASE_DIR}/${KF_NAME}

下载配置文件。

1
2
3
4
5

$ mkdir -p ${KF_DIR}
$ cd ${KF_DIR}

$ wget -O kfctl_aws.yaml $CONFIG_URI
$ export CONFIG_FILE=${KF_DIR}/kfctl_aws.yaml

编辑配置。在KubeFlow中，似乎AIA IAM角色可以从v1.0.1开始用于KubeFlow的服务帐户，但是当我在v1.0.2中尝试使用该服务时，出现一个错误，提示无法创建角色，并且该角色不起作用。因此，使用常规的"使用节点组角色"方法设置角色。

将配置中的kubeflow-aws替换为${AWS_CLUSTER_NAME}。

1
2
3
4
5
6
7
8

$ sed -i'.bak' -e 's/kubeflow-aws/'"$AWS_CLUSTER_NAME"'/' ${CONFIG_FILE}
$ aws iam list-roles \
| jq -r ".Roles[] \
| select(.RoleName \
| startswith("eksctl-$AWS_CLUSTER_NAME") and contains("NodeInstanceRole")) \
.RoleName"

eksctl-kubeflow-nodegroup-standar-NodeInstanceRole-1X5H1J5YPHLPC

将角色名称输出复制到

标准输出，然后在配置中将其设置为.spec.plugins.spec.roles。

1
2
3
4
5
6
7
8
9
10
11
12
13

plugins:
- kind: KfAwsPlugin
metadata:
name: aws
spec:
auth:
basicAuth:
password:
name: password
username: admin
region: ap-northeast-1
roles:
- eksctl-kubeflow-nodegroup-standar-NodeInstanceRole-1X5H1J5YPHLPC

部署KubeFlow。

1 2	$ cd ${KF_DIR} $ kfctl apply -V -f ${CONFIG_FILE}

等待所有资源准备就绪。您可以使用以下命令进行检查。

1	$ kubectl -n kubeflow get all

如果查看

状态并且它是Running或Completed，则应该没有问题。 nvidia-device-plugin-daemonset仍处于待处理状态，可能是因为我未使用GPU实例(可能)。所以这次我会经历。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

NAME READY STATUS RESTARTS AGE
pod/admission-webhook-bootstrap-stateful-set-0 1/1 Running 0 19h
pod/admission-webhook-deployment-569558c8b6-dbwqc 1/1 Running 0 19h
pod/alb-ingress-controller-7c4f854447-d2pwk 1/1 Running 0 19h
pod/application-controller-stateful-set-0 1/1 Running 0 19h
pod/argo-ui-7ffb9b6577-5rkzb 1/1 Running 0 19h
pod/centraldashboard-659bd78c-j5zqs 1/1 Running 0 19h
pod/jupyter-web-app-deployment-679d5f5dc4-2xwn2 1/1 Running 0 19h
pod/katib-controller-7f58569f7d-dsb92 1/1 Running 1 19h
pod/katib-db-manager-54b66f9f9d-g6pxr 1/1 Running 1 19h
pod/katib-mysql-dcf7dcbd5-mvq42 1/1 Running 0 19h
pod/katib-ui-6f97756598-5fc7k 1/1 Running 0 19h
pod/kfserving-controller-manager-0 2/2 Running 1 19h
pod/metacontroller-0 1/1 Running 0 19h
pod/metadata-db-65fb5b695d-442jv 1/1 Running 0 19h
pod/metadata-deployment-65ccddfd4c-xn7cp 1/1 Running 0 19h
pod/metadata-envoy-deployment-7754f56bff-fjcck 1/1 Running 0 19h
pod/metadata-grpc-deployment-5c6db9749-pb752 1/1 Running 4 19h
pod/metadata-ui-7c85545947-nl8h4 1/1 Running 0 19h
pod/minio-6b67f98977-jcz5j 1/1 Running 0 19h
pod/ml-pipeline-6cf777c7bc-d7dlc 1/1 Running 0 19h
pod/ml-pipeline-ml-pipeline-visualizationserver-6d744dd449-rr8df 1/1 Running 0 19h
pod/ml-pipeline-persistenceagent-5c549847fd-4c5bb 1/1 Running 0 19h
pod/ml-pipeline-scheduledworkflow-674777d89c-rvrtc 1/1 Running 0 19h
pod/ml-pipeline-ui-549b5b6744-hkv7d 1/1 Running 0 19h
pod/ml-pipeline-viewer-controller-deployment-fc7f7cb65-thjm5 1/1 Running 0 19h
pod/mpi-operator-548d8cdbbd-ndwkc 1/1 Running 0 19h
pod/mysql-85bc64f5c4-gvwph 1/1 Running 0 19h
pod/notebook-controller-deployment-5c55f5845b-n9xgn 1/1 Running 0 19h
pod/nvidia-device-plugin-daemonset-5rmbb 0/1 Pending 0 19h
pod/nvidia-device-plugin-daemonset-7pqv2 0/1 Pending 0 19h
pod/nvidia-device-plugin-daemonset-j4jc7 1/1 Running 0 18m
pod/profiles-deployment-57d44f597d-mhl87 2/2 Running 0 19h
pod/pytorch-operator-cf8c5c497-gbt6z 1/1 Running 0 19h
pod/seldon-controller-manager-6b4b969447-5vzqx 1/1 Running 0 19h
pod/spark-operatorcrd-cleanup-h8454 0/2 Completed 0 19h
pod/spark-operatorsparkoperator-76dd5f5688-7d4fv 1/1 Running 0 19h
pod/spartakus-volunteer-57d9875c96-sr8lj 1/1 Running 0 19h
pod/tensorboard-5f685f9d79-745tf 1/1 Running 0 19h
pod/tf-job-operator-5fb85c5fb7-79hn9 1/1 Running 0 19h
pod/workflow-controller-689d6c8846-t2wvp 1/1 Running 0 19h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/admission-webhook-service ClusterIP 10.100.51.140 <none> 443/TCP 19h
service/application-controller-service ClusterIP 10.100.15.143 <none> 443/TCP 19h
service/argo-ui NodePort 10.100.18.201 <none> 80:31463/TCP 19h
service/centraldashboard ClusterIP 10.100.16.227 <none> 80/TCP 19h
service/jupyter-web-app-service ClusterIP 10.100.60.58 <none> 80/TCP 19h
service/katib-controller ClusterIP 10.100.11.97 <none> 443/TCP,8080/TCP 19h
service/katib-db-manager ClusterIP 10.100.250.190 <none> 6789/TCP 19h
service/katib-mysql ClusterIP 10.100.27.243 <none> 3306/TCP 19h
service/katib-ui ClusterIP 10.100.215.210 <none> 80/TCP 19h
service/kfserving-controller-manager-metrics-service ClusterIP 10.100.40.10 <none> 8443/TCP 19h
service/kfserving-controller-manager-service ClusterIP 10.100.41.230 <none> 443/TCP 19h
service/kfserving-webhook-server-service ClusterIP 10.100.124.190 <none> 443/TCP 19h
service/metadata-db ClusterIP 10.100.174.250 <none> 3306/TCP 19h
service/metadata-envoy-service ClusterIP 10.100.80.246 <none> 9090/TCP 19h
service/metadata-grpc-service ClusterIP 10.100.247.63 <none> 8080/TCP 19h
service/metadata-service ClusterIP 10.100.59.187 <none> 8080/TCP 19h
service/metadata-ui ClusterIP 10.100.234.158 <none> 80/TCP 19h
service/minio-service ClusterIP 10.100.78.186 <none> 9000/TCP 19h
service/ml-pipeline ClusterIP 10.100.0.209 <none> 8888/TCP,8887/TCP 19h
service/ml-pipeline-ml-pipeline-visualizationserver ClusterIP 10.100.115.213 <none> 8888/TCP 19h
service/ml-pipeline-tensorboard-ui ClusterIP 10.100.148.88 <none> 80/TCP 19h
service/ml-pipeline-ui ClusterIP 10.100.112.177 <none> 80/TCP 19h
service/mysql ClusterIP 10.100.92.26 <none> 3306/TCP 19h
service/notebook-controller-service ClusterIP 10.100.107.0 <none> 443/TCP 19h
service/profiles-kfam ClusterIP 10.100.176.124 <none> 8081/TCP 19h
service/pytorch-operator ClusterIP 10.100.53.116 <none> 8443/TCP 19h
service/seldon-webhook-service ClusterIP 10.100.11.184 <none> 443/TCP 19h
service/tensorboard ClusterIP 10.100.165.203 <none> 9000/TCP 19h
service/tf-job-operator ClusterIP 10.100.7.225 <none> 8443/TCP 19h

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/nvidia-device-plugin-daemonset 3 3 1 3 1 <none> 19h

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/admission-webhook-deployment 1/1 1 1 19h
deployment.apps/alb-ingress-controller 1/1 1 1 19h
deployment.apps/argo-ui 1/1 1 1 19h
deployment.apps/centraldashboard 1/1 1 1 19h
deployment.apps/jupyter-web-app-deployment 1/1 1 1 19h
deployment.apps/katib-controller 1/1 1 1 19h
deployment.apps/katib-db-manager 1/1 1 1 19h
deployment.apps/katib-mysql 1/1 1 1 19h
deployment.apps/katib-ui 1/1 1 1 19h
deployment.apps/metadata-db 1/1 1 1 19h
deployment.apps/metadata-deployment 1/1 1 1 19h
deployment.apps/metadata-envoy-deployment 1/1 1 1 19h
deployment.apps/metadata-grpc-deployment 1/1 1 1 19h
deployment.apps/metadata-ui 1/1 1 1 19h
deployment.apps/minio 1/1 1 1 19h
deployment.apps/ml-pipeline 1/1 1 1 19h
deployment.apps/ml-pipeline-ml-pipeline-visualizationserver 1/1 1 1 19h
deployment.apps/ml-pipeline-persistenceagent 1/1 1 1 19h
deployment.apps/ml-pipeline-scheduledworkflow 1/1 1 1 19h
deployment.apps/ml-pipeline-ui 1/1 1 1 19h
deployment.apps/ml-pipeline-viewer-controller-deployment 1/1 1 1 19h
deployment.apps/mpi-operator 1/1 1 1 19h
deployment.apps/mysql 1/1 1 1 19h
deployment.apps/notebook-controller-deployment 1/1 1 1 19h
deployment.apps/profiles-deployment 1/1 1 1 19h
deployment.apps/pytorch-operator 1/1 1 1 19h
deployment.apps/seldon-controller-manager 1/1 1 1 19h
deployment.apps/spark-operatorsparkoperator 1/1 1 1 19h
deployment.apps/spartakus-volunteer 1/1 1 1 19h
deployment.apps/tensorboard 1/1 1 1 19h
deployment.apps/tf-job-operator 1/1 1 1 19h
deployment.apps/workflow-controller 1/1 1 1 19h

NAME DESIRED CURRENT READY AGE
replicaset.apps/admission-webhook-deployment-569558c8b6 1 1 1 19h
replicaset.apps/alb-ingress-controller-7c4f854447 1 1 1 19h
replicaset.apps/argo-ui-7ffb9b6577 1 1 1 19h
replicaset.apps/centraldashboard-659bd78c 1 1 1 19h
replicaset.apps/jupyter-web-app-deployment-679d5f5dc4 1 1 1 19h
replicaset.apps/katib-controller-7f58569f7d 1 1 1 19h
replicaset.apps/katib-db-manager-54b66f9f9d 1 1 1 19h
replicaset.apps/katib-mysql-dcf7dcbd5 1 1 1 19h
replicaset.apps/katib-ui-6f97756598 1 1 1 19h
replicaset.apps/metadata-db-65fb5b695d 1 1 1 19h
replicaset.apps/metadata-deployment-65ccddfd4c 1 1 1 19h
replicaset.apps/metadata-envoy-deployment-7754f56bff 1 1 1 19h
replicaset.apps/metadata-grpc-deployment-5c6db9749 1 1 1 19h
replicaset.apps/metadata-ui-7c85545947 1 1 1 19h
replicaset.apps/minio-6b67f98977 1 1 1 19h
replicaset.apps/ml-pipeline-6cf777c7bc 1 1 1 19h
replicaset.apps/ml-pipeline-ml-pipeline-visualizationserver-6d744dd449 1 1 1 19h
replicaset.apps/ml-pipeline-persistenceagent-5c549847fd 1 1 1 19h
replicaset.apps/ml-pipeline-scheduledworkflow-674777d89c 1 1 1 19h
replicaset.apps/ml-pipeline-ui-549b5b6744 1 1 1 19h
replicaset.apps/ml-pipeline-viewer-controller-deployment-fc7f7cb65 1 1 1 19h
replicaset.apps/mpi-operator-548d8cdbbd 1 1 1 19h
replicaset.apps/mysql-85bc64f5c4 1 1 1 19h
replicaset.apps/notebook-controller-deployment-5c55f5845b 1 1 1 19h
replicaset.apps/profiles-deployment-57d44f597d 1 1 1 19h
replicaset.apps/pytorch-operator-cf8c5c497 1 1 1 19h
replicaset.apps/seldon-controller-manager-6b4b969447 1 1 1 19h
replicaset.apps/spark-operatorsparkoperator-76dd5f5688 1 1 1 19h
replicaset.apps/spartakus-volunteer-57d9875c96 1 1 1 19h
replicaset.apps/tensorboard-5f685f9d79 1 1 1 19h
replicaset.apps/tf-job-operator-5fb85c5fb7 1 1 1 19h
replicaset.apps/workflow-controller-689d6c8846 1 1 1 19h

NAME READY AGE
statefulset.apps/admission-webhook-bootstrap-stateful-set 1/1 19h
statefulset.apps/application-controller-stateful-set 1/1 19h
statefulset.apps/kfserving-controller-manager 1/1 19h
statefulset.apps/metacontroller 1/1 19h

NAME COMPLETIONS DURATION AGE
job.batch/spark-operatorcrd-cleanup 1/1 69s 19h

使用

istio-ingress网关进行端口转发。

1
2
3

$ kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

访问

127.0.0.1:8080以打开仪表板。

スクリーンショット 2020-07-27 19.31.07.png

单击开始设置。

スクリーンショット 2020-07-27 19.31.33.png

设置名称空间。这次，我使用了默认的anonymous。

スクリーンショット 2020-07-27 19.32.59.png

现在您可以使用KubeFlow。

建筑旁边

当KubeFlow准备就绪时，请尝试以下操作。

在KubeFlow上准备机器学习环境并将模型保存在MLFlow上
其他

另外，由于KubeFlow本身没有身份验证，因此有必要使用Cognito准备身份验证机制。创建负载平衡器更加方便，以便可以从外部对其进行访问。另外，尽管在创建集群时将此设置作为选项提供，但创建配置文件cluster.yaml更为方便。

奖金

集群规模

当我尝试制作笔记本容器时，它变成了不足的容器，因此我将其增加。
如下缩放集群。

1	eksctl scale nodegroup --cluster=kubeflow --nodes=5 standard-workers --nodes-max 5

删除资源

删除KubeFlow。

1 2	cd ${KF_DIR} kfctl delete -f ${CONFIG_FILE}

删除kubernetes。

1	$ eksctl delete cluster kubeflow

删除群集时可能会出现以下错误。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

[?] eksctl version 0.24.0
[?] using region ap-northeast-1
[?] deleting EKS cluster "kubeflow"
[?] deleted 0 Fargate profile(s)
[?] kubeconfig has been updated
[?] cleaning up LoadBalancer services
[?] 2 sequential tasks: { delete nodegroup "standard-workers", delete cluster control plane "kubeflow" [async] }
[?] will delete stack "eksctl-kubeflow-nodegroup-standard-workers"
[?] waiting for stack "eksctl-kubeflow-nodegroup-standard-workers" to get deleted
[?] unexpected status "DELETE_FAILED" while waiting for CloudFormation stack "eksctl-kubeflow-nodegroup-standard-workers"
[?] fetching stack events in attempt to troubleshoot the root cause of the failure
[?] AWS::CloudFormation::Stack/eksctl-kubeflow-nodegroup-standard-workers: DELETE_FAILED – "The following resource(s) failed to delete: [NodeInstanceRole]. "
[?] AWS::IAM::Role/NodeInstanceRole: DELETE_FAILED – "Cannot delete entity, must delete policies first. (Service: AmazonIdentityManagement; Status Code: 409; Error Code: DeleteConflict; Request ID: 238db5e1-9d5e-4698-9d3e-eecfe5b229aa)"
[?] 1 error(s) occurred while deleting cluster with nodegroup(s)
[?] waiting for CloudFormation stack "eksctl-kubeflow-nodegroup-standard-workers": ResourceNotReady: failed waiting for successful resource state
Error: failed to delete cluster with nodegroup(s)

在这种情况下，请从IAM控制台中搜索Node，删除该角色，然后重试。