Skip to content

__setitem__: mistake in docstring and possible issue with Partition.BROADCAST and nccl comm #183

@mrava87SW

Description

@mrava87SW

In DistributedArray.__setitem__ the documentation should be fixed as it talks about UNSAFE_SCATTER instead of UNSAFE_BROADCAST (should mention that self.local_array gets updated for each individual rank without ensure it is communicated (like for BROADCAST) -

`Partition.UNSAFE_SCATTER` - Local Arrays are assigned their

Also, some tests seem to suggest that when we have BROADCAST and use a multi-node multi-gpu setup, the DistributedArray.__setitem__ method seems to hung, likely at self._bcast - more investigation required.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions