mo
c154c94c6d
Mirror gpu-lab compose files and model-manager catalog from GPU-Dev; document Dockhand mapping and SGLang cutover target.
1.2 KiB
1.2 KiB
GPU Lab Runbook — VM303 (atc-gpu-dev)
See also config/gpu-lab/RUNBOOK.md (synced from VM303).
Activate via UI
GPU Lab → select model → Activate. Writes /root/gpu-lab/active/current.env, stops other gpu-lab containers, starts matching compose stack.
Manual activate (SGLang + Qwen 3.6 27B AWQ)
cd /root/gpu-lab
cat > active/current.env << 'EOF'
MODEL_PATH=/models/qwen36-27b-awq
SERVED_NAME=qwen3.6-27b-awq
TP_SIZE=4
GPU_MEM_UTIL=0.9
MAX_MODEL_LEN=8192
EOF
docker compose -f compose.vllm-legacy.yml down
docker compose -f compose.vllm-modern.yml down
docker compose -f compose.sglang.yml up -d
curl -s http://127.0.0.1:8010/v1/models
systemctl restart vllm-gateway.service
Rollback to systemd vLLM
cd /root/gpu-lab
docker compose -f compose.vllm-legacy.yml down
docker compose -f compose.vllm-modern.yml down
docker compose -f compose.sglang.yml down
systemctl start vllm.service
systemctl restart vllm-gateway.service litellm-proxy.service
Logs
docker logs -f gpu-lab-sglang
journalctl -u model-manager -f
SGLang on V100
Always use --disable-flashinfer and --disable-flashinfer-sampling (in compose.sglang.yml).