Skip to content

Commit 92c0e01

Browse files
committed
fix(vllm_client): use localhost for weight broadcasting
- Changed the host parameter from self.host to "0.0.0.0" in StatelessProcessGroup.create() - This modification ensures that weight broadcasting works correctly in theVLLM client
1 parent 50f2600 commit 92c0e01

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

trl/extras/vllm_client.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,7 +246,7 @@ def init_communicator(self):
246246
raise Exception(f"Request failed: {response.status_code}, {response.text}")
247247

248248
# Set up the communication group for weight broadcasting
249-
pg = StatelessProcessGroup.create(host=self.host, port=self.group_port, rank=self.rank, world_size=world_size)
249+
pg = StatelessProcessGroup.create(host="0.0.0.0", port=self.group_port, rank=self.rank, world_size=world_size)
250250
self.pynccl_comm = PyNcclCommunicator(pg, device=0)
251251

252252
def update_named_param(self, name: str, weights: torch.Tensor):

0 commit comments

Comments
 (0)