In DistributedArray.__setitem__ the documentation should be fixed as it talks about UNSAFE_SCATTER instead of UNSAFE_BROADCAST (should mention that self.local_array gets updated for each individual rank without ensure it is communicated (like for BROADCAST) -
|
`Partition.UNSAFE_SCATTER` - Local Arrays are assigned their |
Also, some tests seem to suggest that when we have BROADCAST and use a multi-node multi-gpu setup, the DistributedArray.__setitem__ method seems to hung, likely at self._bcast - more investigation required.