atc-GPU/docs/runbook-vm303-source.md

GPU Lab Runbook — VM303 (atc-gpu-dev)

Runtimes (Docker)

Runtime ID	Compose	Image
vllm-legacy-040	compose.vllm-legacy.yml	vllm/vllm-openai:v0.4.0.post1
vllm-modern-080	compose.vllm-modern.yml	vllm/vllm-openai:v0.8.5.post1
sglang-modern	compose.sglang.yml	lmsysorg/sglang:v0.5.12-cu129-runtime

Only one inference container runs at a time (port 8010).

Activate (UI)

GPU Lab → pick model → Activate. Writes /root/gpu-lab/active/current.env, stops all gpu-lab containers, starts the matching compose stack.

Manual activate

cd /root/gpu-lab
# edit active/current.env (MODEL_PATH, TP_SIZE, VLLM_EXTRA_ARGS, …)
docker compose -f compose.vllm-legacy.yml down
docker compose -f compose.vllm-legacy.yml up -d
curl -s http://127.0.0.1:8010/v1/models

Rollback to systemd vLLM

cd /root/gpu-lab
docker compose -f compose.vllm-legacy.yml down
docker compose -f compose.vllm-modern.yml down
docker compose -f compose.sglang.yml down
systemctl start vllm.service
systemctl restart vllm-gateway.service litellm-proxy.service

Logs

docker logs -f gpu-lab-vllm-legacy
docker logs -f gpu-lab-vllm-modern
docker logs -f gpu-lab-sglang
journalctl -u model-manager -f

SGLang on V100

Always use --disable-flashinfer and --disable-flashinfer-sampling (already in compose.sglang.yml).

Dockhand

• Environment GPU-Dev → 10.0.20.106:2375
• Stacks: gpu-lab-vllm-legacy, gpu-lab-vllm-modern, gpu-lab-sglang (compose in Dockhand data volume)
• Running workload appears as stack gpu-lab (compose project on VM303)
• API: GET /api/stacks?env=8 (not environmentId)
• UI: http://atc-docker01.dell-atc.lan:8082/

Docker data

Images stored under /root/docker-data (not /var — small partition).