Is it advised to have different node selectors for vGPU and GPU nodes. Should these 2 kind of jobs coexist on a single node?
I have seen a scenario where a A100 node(8 GPUs) using "nvidia.com/gpu" resource type are already occupied but still volcano schedules the vGPU based jobs on the same node.This job is stuck in pending state until one of the GPU is free.
I was looking at the volcano code but could not find any logic which checks both GPU and vGPU usage before assigning a node to the task.