31 lines
920 B
Markdown
31 lines
920 B
Markdown
|
# GPU Lab Landscape
|
|||
|
|
|
|||
|
|
## Nodes
|
|||
|
|
|
|||
|
|
| Host | IP | Role |
|
|||
|
|
|------|-----|------|
|
|||
|
|
| VM303 GPU-Dev | 10.0.20.106 | GPU Lab UI (:9000), inference (:8010), gateway (:8001) |
|
|||
|
|
| atc-docker01 | 10.0.21.45 | Homepage, Dockhand (:8082) |
|
|||
|
|
| Bart-GPU | — | Reference node (documented in model-manager) |
|
|||
|
|
| Mo-GPU | — | Reference node |
|
|||
|
|
|
|||
|
|
## VM303 specs
|
|||
|
|
|
|||
|
|
See `config/model-manager/vm-specs.json`: 4× Tesla V100 32GB, Docker with NVIDIA runtime.
|
|||
|
|
|
|||
|
|
## Inference runtimes
|
|||
|
|
|
|||
|
|
Only one runtime is active at a time on port **8010**:
|
|||
|
|
|
|||
|
|
| Runtime ID | Engine | Compose file |
|
|||
|
|
|------------|--------|--------------|
|
|||
|
|
| `vllm-legacy-040` | vLLM 0.4.x | `compose.vllm-legacy.yml` |
|
|||
|
|
| `vllm-modern-080` | vLLM 0.8.x | `compose.vllm-modern.yml` |
|
|||
|
|
| `sglang-modern` | SGLang 0.5.12 | `compose.sglang.yml` |
|
|||
|
|
|
|||
|
|
Models live under `/root/models/` on VM303 (not in git).
|
|||
|
|
|
|||
|
|
## Docker data
|
|||
|
|
|
|||
|
|
Images stored under `/root/docker-data` (avoid small `/var` partition).
|