6

Try to deploy Argo on GCP Autopilot

 2 years ago
source link: http://www.donghao.org/2022/02/18/try-to-deploy-argo-on-gcp-autopilot/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Try to deploy Argo on GCP Autopilot

Creating an Autopilot cluster in GCP K8S is quite easy. But after deploying Argo and launching our pipeline, the Argo report errors:

Failed to pull image "eu-docker.pkg.dev/project-name/mytag:123456789"
b85d23bf513ba037f4b2fbd5e": rpc error: code = Unknown desc = failed to pull and unpack image eu-docker.pkg.dev/project-name/mytag:123456789": failed to resolve reference "eu-docker.pkg.dev/project-name/mytag:123456789": failed to authorize: failed to fetch oauth token: unexpected status: 403 Forbidden
Python
Failed to pull image "eu-docker.pkg.dev/project-name/mytag:123456789"
b85d23bf513ba037f4b2fbd5e": rpc error: code = Unknown desc = failed to pull and unpack image eu-docker.pkg.dev/project-name/mytag:123456789": failed to resolve reference "eu-docker.pkg.dev/project-name/mytag:123456789": failed to authorize: failed to fetch oauth token: unexpected status: 403 Forbidden

The solution is (give k8s cluster the permission to pull docker image from our docker repository):

kubectl create secret docker-registry gcr-json-key  --docker-server=eu-docker.pkg.dev  --docker-username=_json_key  --docker-password="$(cat our_service_account.json)"  [email protected] -n argo
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}' -n argo
Python
xxxxxxxxxx
kubectl create secret docker-registry gcr-json-key  --docker-server=eu-docker.pkg.dev  --docker-username=_json_key  --docker-password="$(cat our_service_account.json)"  [email protected] -n argo
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}' -n argo

Then the second problem jumped out: 

admission webhook "validation.gatekeeper.sh" deni
ed the request: [denied by autogke-no-write-mode-hostpath] hostPath volume docker-sock used in container wait
uses path /var/run/docker.sock which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes a
re: ["/var/log/"]. Requesting user: <system:serviceaccount:argo:argo> and groups: <["system:serviceaccounts",
"system:serviceaccounts:argo", "system:authenticated"]>
Python
xxxxxxxxxx
admission webhook "validation.gatekeeper.sh" deni
ed the request: [denied by autogke-no-write-mode-hostpath] hostPath volume docker-sock used in container wait
uses path /var/run/docker.sock which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes a
re: ["/var/log/"]. Requesting user: <system:serviceaccount:argo:argo> and groups: <["system:serviceaccounts",
"system:serviceaccounts:argo", "system:authenticated"]>

The solution is to set emissary as containerRuntimeExecutor by modifying the file of Argo’s install.yaml:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
data:
  config: |
    containerRuntimeExecutor: emissary
    containerRuntimeExecutors:
      - name: emissary
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: emissary
      - name: pns
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: pns
      - name: k8sapi
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: k8sapi
Python
xxxxxxxxxx
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
data:
  config: |
    containerRuntimeExecutor: emissary
    containerRuntimeExecutors:
      - name: emissary
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: emissary
      - name: pns
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: pns
      - name: k8sapi
        selector:
          matchLabels:
            workflows.argoproj.io/container-runtime-executor: k8sapi

Finally, seems all problems have been solved. My colleague Tianchu find out that Autopilot couldn’t support a pod with memory larger than 80GB:

Since many of our applications need memory more than 80 GB, Autopilot can’t be our choice in recent limitations.

Like this:

Loading...

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK