48 lines
1.2 KiB
Markdown
48 lines
1.2 KiB
Markdown
|
# GPU Lab Runbook — VM303 (atc-gpu-dev)
|
||
|
|
|
||
|
|
See also `config/gpu-lab/RUNBOOK.md` (synced from VM303).
|
||
|
|
|
||
|
|
## Activate via UI
|
||
|
|
|
||
|
|
GPU Lab → select model → **Activate**. Writes `/root/gpu-lab/active/current.env`, stops other gpu-lab containers, starts matching compose stack.
|
||
|
|
|
||
|
|
## Manual activate (SGLang + Qwen 3.6 27B AWQ)
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd /root/gpu-lab
|
||
|
|
cat > active/current.env << 'EOF'
|
||
|
|
MODEL_PATH=/models/qwen36-27b-awq
|
||
|
|
SERVED_NAME=qwen3.6-27b-awq
|
||
|
|
TP_SIZE=4
|
||
|
|
GPU_MEM_UTIL=0.9
|
||
|
|
MAX_MODEL_LEN=8192
|
||
|
|
EOF
|
||
|
|
docker compose -f compose.vllm-legacy.yml down
|
||
|
|
docker compose -f compose.vllm-modern.yml down
|
||
|
|
docker compose -f compose.sglang.yml up -d
|
||
|
|
curl -s http://127.0.0.1:8010/v1/models
|
||
|
|
systemctl restart vllm-gateway.service
|
||
|
|
```
|
||
|
|
|
||
|
|
## Rollback to systemd vLLM
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd /root/gpu-lab
|
||
|
|
docker compose -f compose.vllm-legacy.yml down
|
||
|
|
docker compose -f compose.vllm-modern.yml down
|
||
|
|
docker compose -f compose.sglang.yml down
|
||
|
|
systemctl start vllm.service
|
||
|
|
systemctl restart vllm-gateway.service litellm-proxy.service
|
||
|
|
```
|
||
|
|
|
||
|
|
## Logs
|
||
|
|
|
||
|
|
```bash
|
||
|
|
docker logs -f gpu-lab-sglang
|
||
|
|
journalctl -u model-manager -f
|
||
|
|
```
|
||
|
|
|
||
|
|
## SGLang on V100
|
||
|
|
|
||
|
|
Always use `--disable-flashinfer` and `--disable-flashinfer-sampling` (in `compose.sglang.yml`).
|