[fix] replacing torch.cuda.set_device with CUDA_VISIBLE_DEVICES

[oandreeva-nv]

in my experiments with loading multiple models onto different gpus torch.cuda.set_device wasn't behaving as a GPU "router", thus using CUDA_VISIBLE_DEVICES
before the fix, tp=1 count=2 test GPU utilisation:

Test Matrix: model='vllm_opt_KIND_GPU_tp1_count2', kind='KIND_GPU', tp='1', instance_count='2'


=============== Before Loading vLLM Model ===============
GPU 0 Memory Utilization: 1426849792 bytes
GPU 1 Memory Utilization: 1426849792 bytes
=============== After Loading vLLM Model ===============
GPU 0 Memory Utilization: 44708986880 bytes
GPU 1 Memory Utilization: 1866727424 bytes

With this fix:

Test Matrix: model='vllm_opt_KIND_GPU_tp1_count2', kind='KIND_GPU', tp='1', instance_count='2'


=============== Before Loading vLLM Model ===============
GPU 0 Memory Utilization: 1426849792 bytes
GPU 1 Memory Utilization: 1426849792 bytes
=============== After Loading vLLM Model ===============
GPU 0 Memory Utilization: 43918819328 bytes
GPU 1 Memory Utilization: 43916722176 bytes

• also tested with multiple TP=1 models + different GPU ids in a config.pbtxt in a cluster