Add GPU Lab configs, docs, and runbooks for VM303.
Mirror gpu-lab compose files and model-manager catalog from GPU-Dev; document Dockhand mapping and SGLang cutover target.
This commit is contained in:
@@ -0,0 +1,47 @@
|
||||
# GPU Lab Runbook — VM303 (atc-gpu-dev)
|
||||
|
||||
See also `config/gpu-lab/RUNBOOK.md` (synced from VM303).
|
||||
|
||||
## Activate via UI
|
||||
|
||||
GPU Lab → select model → **Activate**. Writes `/root/gpu-lab/active/current.env`, stops other gpu-lab containers, starts matching compose stack.
|
||||
|
||||
## Manual activate (SGLang + Qwen 3.6 27B AWQ)
|
||||
|
||||
```bash
|
||||
cd /root/gpu-lab
|
||||
cat > active/current.env << 'EOF'
|
||||
MODEL_PATH=/models/qwen36-27b-awq
|
||||
SERVED_NAME=qwen3.6-27b-awq
|
||||
TP_SIZE=4
|
||||
GPU_MEM_UTIL=0.9
|
||||
MAX_MODEL_LEN=8192
|
||||
EOF
|
||||
docker compose -f compose.vllm-legacy.yml down
|
||||
docker compose -f compose.vllm-modern.yml down
|
||||
docker compose -f compose.sglang.yml up -d
|
||||
curl -s http://127.0.0.1:8010/v1/models
|
||||
systemctl restart vllm-gateway.service
|
||||
```
|
||||
|
||||
## Rollback to systemd vLLM
|
||||
|
||||
```bash
|
||||
cd /root/gpu-lab
|
||||
docker compose -f compose.vllm-legacy.yml down
|
||||
docker compose -f compose.vllm-modern.yml down
|
||||
docker compose -f compose.sglang.yml down
|
||||
systemctl start vllm.service
|
||||
systemctl restart vllm-gateway.service litellm-proxy.service
|
||||
```
|
||||
|
||||
## Logs
|
||||
|
||||
```bash
|
||||
docker logs -f gpu-lab-sglang
|
||||
journalctl -u model-manager -f
|
||||
```
|
||||
|
||||
## SGLang on V100
|
||||
|
||||
Always use `--disable-flashinfer` and `--disable-flashinfer-sampling` (in `compose.sglang.yml`).
|
||||
Reference in New Issue
Block a user