Transpiler: add Multi-GPU support #178
Draft
+2,432
−192
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Multi gpu support for kernel reduce
AlltoAll & Fix the redundant division across gpus on kn input bug
update meta in multi-gpu scenario
Finish all to all
Add ReduceScatter Epilogue
… problem (changed to nvshmem_rt.h) 2. Debug wait and send logic for allgather in transpiler and runtime
Allgather jjn
reduce scatter fix: Enable strided reduction in reduction kernel Fix sync issues in demo
jiazhihao
reviewed
Apr 24, 2025
There was a problem hiding this comment.
@NorthmanPKU @linsj20 @xinhaoc It seems these changes are general fixes to Mirage's build pipeline and can benefit other ongoing projects. Can we push these chanegs to the main branch in a separate PR?
Python Interface For NVSHMEM Memory Management and Initialization
update demos
…ge in reducescatter
Communication Buffer Bug Fix
fix rs output tensor reference & ag input elem cnt
Merge main and fix merge bugs
fix profiler for comm
implement signal for rs and a2a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes:
draft pr for collaboration
TODO List