Promote MAAP staging hubs to prod #7219
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,45 +5,234 @@ userServiceAccount: | |
| annotations: | ||
| eks.amazonaws.com/role-arn: arn:aws:iam::916098889494:role/maap-prod | ||
| jupyterhub: | ||
| custom: | ||
| 2i2c: | ||
| add_staff_user_ids_to_admin_users: false | ||
| homepage: | ||
| gitRepoBranch: master | ||
| gitRepoUrl: https://github.com/MAAP-Project/maap-hub-homepage | ||
| singleuser: | ||
| extraEnv: | ||
| SCRATCH_BUCKET: s3://maap-scratch-prod/$(JUPYTERHUB_USER) | ||
| MAAP_API_HOST: api.maap-project.org | ||
| DOCKERIMAGE_PATH_DEFAULT: mas.maap-project.org/root/maap-workspaces/custom_images/maap_base:v5.0.0 | ||
| DOCKERIMAGE_PATH_BASE_IMAGE: $(JUPYTER_IMAGE) | ||
| WORKSPACE_BUCKET: maap-ops-workspace | ||
| nodeSelector: | ||
| 2i2c/hub-name: prod | ||
| # NOTE the only difference between profileList for staging and prod is the image tags | ||
| # prod should always have the latest official release and staging has develop unless we are | ||
| # actively doing a release in which case staging images point to our release tag | ||
| profileList: | ||
|
Member
There was a problem hiding this comment. Normally, we would like to keep having different images in staging and prod could cause problems here, in case the images being different causes failure when migrating. It could also cause the other parts of profile Lists (such as resource config) to drift out of sync between these two. However, we also recognize that you want to probably test out different images as you're onboarding an existing userbase to this hub, and want to be flexible. So I see two paths forward:
I wanna unblock y'all asap, so while I have a preference for (1) happy to do either.
Contributor
There was a problem hiding this comment. Sorry jumping into this conversation as I come back from leave. Let me know if I am phrasing this correctly - Does that make sense?
Contributor
There was a problem hiding this comment. On MAAP, the DIT, UAT and OPS venues come with their associated deployments of the API and data processing clusters which impact the jupyter extensions used in the images. So in terms of testing, we are not just testing the images, but also entire the deployment venue which is isolated in its own cloud env.
Contributor
There was a problem hiding this comment. I added a block comment above profileList and we would like to go with option 2
Member
There was a problem hiding this comment. Thanks @grallewellyn! I've retitled the PR slightly and merged this! @sujen1412 I opened #7233 to split off the other conversation so we don't lose track of it! |
||
| - display_name: Choose your environment and resources | ||
| default: true | ||
| profile_options: | ||
| image: | ||
| display_name: Environment | ||
| dynamic_image_building: | ||
| enabled: true | ||
| unlisted_choice: | ||
| enabled: true | ||
| display_name: Custom image | ||
| validation_regex: ^.+:.+$ | ||
| validation_message: Must be a publicly available docker image, of form <image-name>:<tag> | ||
| kubespawner_override: | ||
| image: '{value}' | ||
| choices: | ||
| 01-pangeo: | ||
| display_name: Modified Pangeo Notebook | ||
| description: Pangeo based notebook with a Python environment | ||
| kubespawner_override: | ||
| image: mas.maap-project.org/root/maap-workspaces/2i2c/pangeo:v5.0.0 | ||
| init_containers: | ||
| - name: jupyterhub-gitpuller-init | ||
| image: public.ecr.aws/nasa-veda/jupyterhub-gitpuller-init:97eb45f9d23b128aff810e45911857d5cffd05c2 | ||
| env: | ||
| - name: TARGET_PATH | ||
| value: veda-docs | ||
| - name: SOURCE_REPO | ||
| value: https://github.com/NASA-IMPACT/veda-docs | ||
| volumeMounts: | ||
| - name: home | ||
| mountPath: /home/jovyan | ||
| subPath: '{escaped_username}' | ||
| securityContext: | ||
| runAsUser: 1000 | ||
| runAsGroup: 1000 | ||
| 02-rocker: | ||
| display_name: Rocker Geospatial | ||
| description: JupyterHub environment with many R geospatial libraries pre-installed | ||
| kubespawner_override: | ||
| image: mas.maap-project.org/root/maap-workspaces/2i2c/r:v5.0.0 | ||
| init_containers: | ||
| - name: jupyterhub-gitpuller-init | ||
| image: public.ecr.aws/nasa-veda/jupyterhub-gitpuller-init:97eb45f9d23b128aff810e45911857d5cffd05c2 | ||
| env: | ||
| - name: TARGET_PATH | ||
| value: veda-docs | ||
| - name: SOURCE_REPO | ||
| value: https://github.com/NASA-IMPACT/veda-docs | ||
| volumeMounts: | ||
| - name: home | ||
| mountPath: /home/jovyan | ||
| subPath: '{escaped_username}' | ||
| securityContext: | ||
| runAsUser: 1000 | ||
| runAsGroup: 1000 | ||
| 03-isce3: | ||
| display_name: isce3 | ||
| description: Pangeo based notebook with a Python environment and isce3 | ||
| kubespawner_override: | ||
| image: mas.maap-project.org/root/maap-workspaces/2i2c/isce3:v5.0.0 | ||
| init_containers: | ||
| - name: jupyterhub-gitpuller-init | ||
| image: public.ecr.aws/nasa-veda/jupyterhub-gitpuller-init:97eb45f9d23b128aff810e45911857d5cffd05c2 | ||
| env: | ||
| - name: TARGET_PATH | ||
| value: veda-docs | ||
| - name: SOURCE_REPO | ||
| value: https://github.com/NASA-IMPACT/veda-docs | ||
| volumeMounts: | ||
| - name: home | ||
| mountPath: /home/jovyan | ||
| subPath: '{escaped_username}' | ||
| securityContext: | ||
| runAsUser: 1000 | ||
| runAsGroup: 1000 | ||
| 04-qgis: | ||
| display_name: QGIS on Linux Desktop | ||
| description: Linux desktop in the browser, with qgis installed | ||
| kubespawner_override: | ||
| # Launch people directly into the Linux desktop when they start | ||
| default_url: /desktop | ||
| # Built from https://github.com/2i2c-org/nasa-qgis-image | ||
| image: quay.io/2i2c/nasa-qgis-image:d76118ea0c15 | ||
| resource_allocation: | ||
| display_name: Resource Allocation | ||
| choices: | ||
| mem_1_9: | ||
| display_name: 1.9 GB RAM, upto 3.7 CPUs | ||
| allowed_groups: | ||
| - CPU:XS | ||
| kubespawner_override: | ||
| mem_guarantee: 1991244775 | ||
| mem_limit: 1991244775 | ||
| cpu_guarantee: 0.2328125 | ||
| cpu_limit: 3.725 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.xlarge | ||
| mem_3_7: | ||
| display_name: 3.7 GB RAM, upto 3.7 CPUs | ||
| allowed_groups: | ||
| - CPU:S | ||
| kubespawner_override: | ||
| mem_guarantee: 3982489550 | ||
| mem_limit: 3982489550 | ||
| cpu_guarantee: 0.465625 | ||
| cpu_limit: 3.725 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.xlarge | ||
| mem_7_4: | ||
| display_name: 7.4 GB RAM, upto 3.7 CPUs | ||
| allowed_groups: | ||
| - CPU:M | ||
| kubespawner_override: | ||
| mem_guarantee: 7964979101 | ||
| mem_limit: 7964979101 | ||
| cpu_guarantee: 0.93125 | ||
| cpu_limit: 3.725 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.xlarge | ||
| mem_14_8: | ||
| display_name: 14.8 GB RAM, upto 3.7 CPUs | ||
| allowed_groups: | ||
| - CPU:L | ||
| kubespawner_override: | ||
| mem_guarantee: 15929958203 | ||
| mem_limit: 15929958203 | ||
| cpu_guarantee: 1.8625 | ||
| cpu_limit: 3.725 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.xlarge | ||
| default: true | ||
| mem_29_7: | ||
| display_name: 29.7 GB RAM, upto 3.7 CPUs | ||
| allowed_groups: | ||
| - CPU:XL | ||
| kubespawner_override: | ||
| mem_guarantee: 31859916406 | ||
| mem_limit: 31859916406 | ||
| cpu_guarantee: 3.725 | ||
| cpu_limit: 3.725 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.xlarge | ||
| mem_60_6: | ||
| display_name: 60.6 GB RAM, upto 15.6 CPUs | ||
| allowed_groups: | ||
| - CPU:XXL | ||
| kubespawner_override: | ||
| mem_guarantee: 65094448840 | ||
| mem_limit: 65094448840 | ||
| cpu_guarantee: 7.8475 | ||
| cpu_limit: 15.695 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.4xlarge | ||
| mem_121_2: | ||
| display_name: 121.2 GB RAM, upto 15.6 CPUs | ||
| allowed_groups: | ||
| - CPU:XXXL | ||
| kubespawner_override: | ||
| mem_guarantee: 130188897681 | ||
| mem_limit: 130188897681 | ||
| cpu_guarantee: 15.695 | ||
| cpu_limit: 15.695 | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: r5.4xlarge | ||
| - display_name: NVIDIA Tesla T4, ~16 GB, ~4 CPUs | ||
| description: Start a container on a dedicated node with a GPU | ||
| slug: gpu | ||
| allowed_groups: | ||
| - GPU:T4 | ||
| profile_options: | ||
| image: | ||
| display_name: Environment | ||
| dynamic_image_building: | ||
| enabled: true | ||
| unlisted_choice: | ||
| enabled: true | ||
| display_name: Custom image | ||
| validation_regex: ^.+:.+$ | ||
| validation_message: Must be a publicly available docker image of form <image-name>:<tag> | ||
| kubespawner_override: | ||
| image: '{value}' | ||
| choices: | ||
| pytorch: | ||
| display_name: Pangeo PyTorch ML Notebook | ||
| default: false | ||
| slug: pytorch | ||
| kubespawner_override: | ||
| image: quay.io/pangeo/pytorch-notebook:2024.11.11 | ||
| tensorflow2: | ||
| display_name: Pangeo Tensorflow2 ML Notebook | ||
| default: true | ||
| slug: tensorflow2 | ||
| kubespawner_override: | ||
| image: quay.io/pangeo/ml-notebook:2024.11.11 | ||
| kubespawner_override: | ||
| environment: | ||
| NVIDIA_DRIVER_CAPABILITIES: compute,utility | ||
| mem_limit: | ||
| mem_guarantee: 14G | ||
| node_selector: | ||
| node.kubernetes.io/instance-type: g4dn.xlarge | ||
| extra_resource_limits: | ||
| nvidia.com/gpu: '1' | ||
| hub: | ||
| config: | ||
| GenericOAuthenticator: | ||
| oauth_callback_url: https://hub.maap-project.org/hub/oauth_callback | ||
| token_url: https://auth.openveda.cloud/realms/maap/protocol/openid-connect/token | ||
| authorize_url: https://auth.openveda.cloud/realms/maap/protocol/openid-connect/auth | ||
| ingress: | ||
| hosts: [hub.maap-project.org] | ||
| tls: | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will set this as the environment variable no matter what image is used. Is that what was expected? In the last PR, I saw this was set to be the same as the name of the image, in which case it should use
$(JUPYTER_IMAGE)as the value.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, extracted out DOCKERIMAGE_PATH_BASE_IMAGE!