mo
c154c94c6d
Mirror gpu-lab compose files and model-manager catalog from GPU-Dev; document Dockhand mapping and SGLang cutover target.
31 lines
920 B
Markdown
31 lines
920 B
Markdown
# GPU Lab Landscape
|
||
|
||
## Nodes
|
||
|
||
| Host | IP | Role |
|
||
|------|-----|------|
|
||
| VM303 GPU-Dev | 10.0.20.106 | GPU Lab UI (:9000), inference (:8010), gateway (:8001) |
|
||
| atc-docker01 | 10.0.21.45 | Homepage, Dockhand (:8082) |
|
||
| Bart-GPU | — | Reference node (documented in model-manager) |
|
||
| Mo-GPU | — | Reference node |
|
||
|
||
## VM303 specs
|
||
|
||
See `config/model-manager/vm-specs.json`: 4× Tesla V100 32GB, Docker with NVIDIA runtime.
|
||
|
||
## Inference runtimes
|
||
|
||
Only one runtime is active at a time on port **8010**:
|
||
|
||
| Runtime ID | Engine | Compose file |
|
||
|------------|--------|--------------|
|
||
| `vllm-legacy-040` | vLLM 0.4.x | `compose.vllm-legacy.yml` |
|
||
| `vllm-modern-080` | vLLM 0.8.x | `compose.vllm-modern.yml` |
|
||
| `sglang-modern` | SGLang 0.5.12 | `compose.sglang.yml` |
|
||
|
||
Models live under `/root/models/` on VM303 (not in git).
|
||
|
||
## Docker data
|
||
|
||
Images stored under `/root/docker-data` (avoid small `/var` partition).
|