火星文专家 (@uoqocjx)有用vllm部署Qwen3.8-27B-FP8的佬吗 中发帖

4张3090部署Qwen3.8-27B-FP8 
为啥token输出速度只有50tps 
驱动版本是 Driver Version: 590.44.01      CUDA Version: 13.1 
下面是部署配置 
services:
  vllm-Qwen3.8-27B:
    image: vllm/vllm-openai:v0.27.0
    ipc: host
    restart: always
    ulimits:
      memlock: -1
      stack: 67108864
    gpus: all
    ports:
      - "8324:8000"
    volumes:
      - ./cache:/root/.cache
      - ./triton:/root/.triton
    environment:
 ...
 
 
Back to Top