Skip to content

Commit 3206aca

Browse files
zejunchen-zejunBoyuanFeng
authored andcommitted
[LMCache][Example] Align the PYTHONHASHSEED for prefillers and decoders for KV chunks hashing (vllm-project#21161)
Signed-off-by: zejunchen-zejun <zejun.chen@amd.com> Signed-off-by: Boyuan Feng <boyuan@meta.com>
1 parent c63b4f8 commit 3206aca

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

examples/others/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,14 @@ else
1515
MODEL=$2
1616
fi
1717

18+
# The prefillers and decoders in LMCache use the same hash seed for all chunk keys.
19+
# This seed must be aligned so that decoders can identify and retrieve KV cache
20+
# entries stored by prefillers.
21+
#
22+
# WARNING: Using a fixed hash seed is insecure and makes the application vulnerable to
23+
# denial-of-service attacks. In a production environment, this should be set to a
24+
# secure random value. This is set to a fixed value for demonstration purposes only.
25+
export PYTHONHASHSEED=${VLLM_PYTHON_HASH_SEED:-123}
1826

1927
if [[ $1 == "prefiller" ]]; then
2028
# Prefiller listens on port 8100

0 commit comments

Comments
 (0)