support cudagraph and w8a8 inference for lmdeploy #515

FlamingoPg started this conversation in General

FlamingoPg
Feb 1, 2024

finished PR:

fix and change all logger using get_logger lmdeploy#1064
https://github.com/InternLM/lmdeploy/actions/runs/7740695656/job/21106180511?pr=1088
TODO
1.finish w8a8 cudagraph
2.finsh torch.compile

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment