115 lines
3.0 KiB
Markdown
115 lines
3.0 KiB
Markdown
|
# Deployment
|
||
|
|
|
||
|
|
## Lab hosts
|
||
|
|
|
||
|
|
| Host | Role | Notes |
|
||
|
|
|------|------|-------|
|
||
|
|
| `atc-portal01` `10.0.21.49` | Runs cockpit + OpenManage AI containers | Primary deploy target |
|
||
|
|
| `cov-omeprod01` `10.0.21.104` | OME 4.7.0 | HTTPS, lab admin account |
|
||
|
|
| `atc-gpu-prod` `10.0.10.106` | vLLM `:8000`, GPU metrics `:9110` | Model `llama3-70b-gptq` |
|
||
|
|
| `atc-mgt01` | Forgejo `:3001` | Source of truth for git |
|
||
|
|
|
||
|
|
**Paths on portal**
|
||
|
|
|
||
|
|
- Cockpit: `/home/lab/ome-cockpit` (symlink/convention; compose here)
|
||
|
|
- OpenManage AI: `/home/lab/openmanage`
|
||
|
|
|
||
|
|
**Published ports**
|
||
|
|
|
||
|
|
- Cockpit UI/API: **3090 → 8090**
|
||
|
|
- OpenManage AI: **3080**
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Install / upgrade
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd /home/lab/ome-cockpit
|
||
|
|
cp .env.example .env # first time
|
||
|
|
# edit secrets
|
||
|
|
docker compose up -d --build
|
||
|
|
docker compose ps
|
||
|
|
curl -sS http://127.0.0.1:3090/api/health
|
||
|
|
```
|
||
|
|
|
||
|
|
Dependency changes (e.g. `asyncssh`) require **image rebuild**.
|
||
|
|
UI/`main.py` edits apply via bind mounts; hard-refresh browsers (`Ctrl+Shift+R`). Index is served `Cache-Control: no-store`.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Environment reference
|
||
|
|
|
||
|
|
See `.env.example`. Critical keys:
|
||
|
|
|
||
|
|
| Variable | Purpose |
|
||
|
|
|----------|---------|
|
||
|
|
| `OME_URL` | e.g. `https://cov-omeprod01.dell-atc.lan` |
|
||
|
|
| `OME_USER` / `OME_PASSWORD` | API session |
|
||
|
|
| `POLL_INTERVAL` | Seconds between OME polls |
|
||
|
|
| `VLLM_URL` | e.g. `http://10.0.10.106:8000/v1` |
|
||
|
|
| `VLLM_MODEL` | Default chat model id |
|
||
|
|
| `VLLM_MAX_TOKENS` | Completion cap (lab ~700) |
|
||
|
|
| `CHAT_SYSTEM_CHARS` | System prompt budget (~5500) |
|
||
|
|
| `GPU_METRICS_URL` | e.g. `http://10.0.10.106:9110` |
|
||
|
|
| `OPENWEBUI_URL` | Companion AI |
|
||
|
|
| `OPENWEBUI_EMAIL` / `OPENWEBUI_PASSWORD` | Service login for model list + virtual models |
|
||
|
|
| `COCKPIT_DATA` | SQLite directory (default `/data`) |
|
||
|
|
|
||
|
|
**Never commit `.env`.**
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Compose volumes
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
volumes:
|
||
|
|
- ./ui:/ui:ro
|
||
|
|
- ./api/main.py:/app/main.py:ro
|
||
|
|
- ./data:/data
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Health checks
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl -sS http://127.0.0.1:3090/api/health
|
||
|
|
curl -sS http://127.0.0.1:3090/api/fleet | python3 -c "import sys,json;d=json.load(sys.stdin);print(d['summary'])"
|
||
|
|
curl -sS http://127.0.0.1:3090/api/ssh/sessions
|
||
|
|
docker logs ome-cockpit --tail 100
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Git workflow
|
||
|
|
|
||
|
|
Remote: `http://atc-mgt01.dell-atc.lan:3001/mo/ome-cockpit.git`
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd /home/lab/ome-cockpit
|
||
|
|
git status
|
||
|
|
# commit after reviewing diffs — never stage .env / data
|
||
|
|
git push origin main # requires Forgejo credentials/token for user mo
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
| Symptom | Check |
|
||
|
|
|---------|-------|
|
||
|
|
| Empty topology | Filters (Servers/iDRACs), OME creds, `docker logs` |
|
||
|
|
| Chat 400/502 | Context size; vLLM up; try Reset chat / smaller history |
|
||
|
|
| SSH modal blank | `/vendor/xterm/xterm.min.js` 200; hard refresh |
|
||
|
|
| SSH “not in fleet” | Device has no management IP in snapshot |
|
||
|
|
| Inventory vanishes | Should be fixed via soft refresh; hard reload if old JS cached |
|
||
|
|
| GPU empty | Sidecar `:9110`, firewall from portal → GPU host |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Backup
|
||
|
|
|
||
|
|
- Code: Forgejo
|
||
|
|
- Tickets: backup `/home/lab/ome-cockpit/data/ops.db`
|
||
|
|
- Secrets: out-of-band password manager — not git
|