Skip to content

Commit e41312a

Browse files
kitaekattclaude
andauthored
[Bugfix] Skip generation config fallback for GGUF to prevent multi-process hang (#30209)
Co-authored-by: Claude Opus 4.5 <[email protected]>
1 parent 7b35011 commit e41312a

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

vllm/transformers_utils/config.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -954,6 +954,13 @@ def try_get_generation_config(
954954
revision: str | None = None,
955955
config_format: str | ConfigFormat = "auto",
956956
) -> GenerationConfig | None:
957+
# GGUF files don't have generation_config.json - their config is embedded
958+
# in the file header. Skip all filesystem lookups to avoid re-reading the
959+
# memory-mapped file, which can hang in multi-process scenarios when the
960+
# EngineCore process already has the file mapped.
961+
if is_gguf(model):
962+
return None
963+
957964
try:
958965
return GenerationConfig.from_pretrained(
959966
model,

0 commit comments

Comments
 (0)