[Feat] Add a new kind of linear operation: LayerShardLinear

[clrs97]

What this PR does / why we need it?

To cut down the memory usage of large weight matrices, we often rely on various linear operations:

• ReplicatedLinear: Stores the entire matrix, consuming excessive memory.
• RowParallelLinear: Requires an all_reduce to merge answer, introducing additional communication overhead and potential accuracy loss. Each token is handled across multiple devices rather than a single device, which is undesirable in SP scenario.
• ...

This PR introduces a new kind of linear operation: LayerShardLinear, which combines the advantages of the existing approaches:

• It evenly distributes a set of layers with identical structures across devices. Each layer retains its complete weights, eliminating redundant memory usage, and no need to merge answer.
• It supports asynchronous broadcasting to prefetch weights for upcoming layers.
• It preserves the custom process_weights_after_loading() method to make keeping NZ format possible.

Does this PR introduce any user-facing change?

No

How was this patch tested?

vLLM main: vllm-project/vllm@f4a948f

• vLLM version: v0.10.2
• vLLM main: vllm-project/vllm@eeb135e

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Code Review

This pull request introduces LayerShardLinear, a new linear operation to reduce memory usage by sharding layers across devices and prefetching weights. The implementation is comprehensive, but there are a few critical issues related to handling small layer clusters and a high-risk monkey-patching pattern that should be addressed to ensure correctness and maintainability.

[CalvinXKY]

Maybe we should add an example of this calling relationship to the tests.

[CalvinXKY]

Default values are used for start_layer and end_layer if not set.