⚡️ Your qwen3.8:27b OpenAI client is not hanging. It is reloading ~19 GB on ev… · @QwenAIFans · TwKit

@QwenAIFans

⚡️ Your qwen3.8:27b OpenAI client is not hanging. It is reloading ~19 GB on every /v1 call. ollama run and /api/chat feel fine. POST /v1/chat/completions just sits there. Catch: that endpoint ignores num_ctx and draft_num_predict in options, so defaults disagree with the warm runner and it cold-loads again. Fix that worked: bake them into the Modelfile PARAMETER num_ctx 65536 PARAMETER draft_num_predict 2 Uniform ~180s replies usually mean one cold load, not slow generation. grep evicting on the server log. Anyone else lose an afternoon to this before checking Modelfile? via ollama#17790 👇 https://github.com/ollama/ollama/issues/17790

曝光 372 · 评论 1 · 点赞 4 · 书签 0