Fix for atomicv on CXI #1147
Fix for atomicv on CXI #1147
Conversation
| shmem_internal_fetch_atomic(ctx, ((uint8_t *) target) + (i * type_size), | ||
| ((uint8_t *) source) + (i * type_size), &tmp_fetch, type_size, | ||
| pe, op, datatype); | ||
| shmem_transport_get_wait((shmem_transport_ctx_t *)ctx); |
There was a problem hiding this comment.
Can we move this outside the loop over count? I don't quite see why we need to wait after every atomic?
There was a problem hiding this comment.
Nope, we cant as the loop itself can go beyond the number of atomics that causes the issue on CXI.
src/shmem_comm.h
Outdated
| ((uint8_t *) source) + (i * type_size), &tmp_fetch, type_size, | ||
| pe, op, datatype); | ||
| shmem_transport_get_wait((shmem_transport_ctx_t *)ctx); | ||
| #ifdef SOS_OFI_CXI_QUIET_RATE |
There was a problem hiding this comment.
Is there any chance we can remove this after the above fix?
There was a problem hiding this comment.
I think based on the atomic OP, we cant get rid of this completely. But, will check.
davidozog
left a comment
davidozog
left a comment
There was a problem hiding this comment.
As discussed offline, perhaps SHMEM_REDUCE_ALGORITHM=recdbl (maybe ring?) is a better workaround (when disabling non-fetching AMO) given this could dramatically impair the performance of linear and tree algorithms.
* Fix for atomicv on CXI * Removing periodic quiet
2 participants
No description provided.