Can this driver support hot loading of vgpu resources inside the pod?

[SeaSoonKeun]

May I ask if driver can suppor hot loading of vgpu resources within the pod?

I have read the article on zhihu , and it ultimately only verified that declared GPU resources can be seen inside the pod. So, I would like to ask, is it possible to achieve GPU hot-loading synchronization within a pod?

Such as GPU usage is only activated when tasks requiring GPU resources are started

Looking forward to your reply

[Shouren]

May I ask if driver can suppor hot loading of vgpu resources within the pod?

I have read the article on zhihu , and it ultimately only verified that declared GPU resources can be seen inside the pod. So, I would like to ask, is it possible to achieve GPU hot-loading synchronization within a pod?

Such as GPU usage is only activated when tasks requiring GPU resources are started

Looking forward to your reply

@SeaSoonKeun Can you describe a scenario in detail how it should be like when GPU hot-loading synchronization is supported ?

Such as GPU usage is only activated when tasks requiring GPU resources are started

For this description, where to get the GPU usage? And to whom tasks are requiring? container runtime, driver api or k8s api ?

[SeaSoonKeun]

Thank you for your response. I may not have explained myself clearly.

What I meant is that during the pod application process, the required GPU resources are specified.

However, after the pod starts, it does not immediately utilize the underlying GPU resources. 【For example, upon accessing the pod, running the nvidia-smi command does not show any GPU information.】

The GPUs are only dynamically allocated to the pod when it begins executing tasks that require GPU resources.

[Shouren]

However, after the pod starts, it does not immediately utilize the underlying GPU resources. 【For example, upon accessing the pod, running the nvidia-smi command does not show any GPU information.】

The GPUs are only dynamically allocated to the pod when it begins executing tasks that require GPU resources.

The above case makes no sense to me. Consider the case that you create a Pod to launch a model with vLLM, both nviida-smi and vLLM will call the nvml API to get the GPU information, so you can not tell which call is from nvidia-smi unless there is an explicit setup for GPU resource. And this setup requires the support of container runtime and is beyond the goal of this project.

@SeaSoonKeun Let me know if there is any container runtime support this feature 😄