English docs and passwordless SSH mesh for lab fleet
- Translate all documentation to English - Add docs/ssh-mesh.md and scripts/setup/setup-ssh-mesh.sh - Add config/hosts/atc-lab.hosts for /etc/hosts on all VMs - Ignore atc_cluster private key in git; mesh deployed on 12 hosts
This commit is contained in:
+28
-28
@@ -1,84 +1,84 @@
|
||||
# Disaster recovery
|
||||
|
||||
Doel: volledige **atc-docker01** dashboard-stack en configuratie terugzetten vanuit git.
|
||||
Goal: fully restore the **atc-docker01** dashboard stack and configuration from git.
|
||||
|
||||
## Vereisten
|
||||
## Requirements
|
||||
|
||||
- VM `atc-docker01` met Docker en git
|
||||
- Netwerk naar interne `.dell-atc.lan` hosts
|
||||
- Secrets: `.env` of bestaande `config/homepage/proxmox.yaml`
|
||||
- VM `atc-docker01` with Docker and git
|
||||
- Network to internal `.dell-atc.lan` hosts
|
||||
- Secrets: `.env` or existing `config/homepage/proxmox.yaml`
|
||||
- SSH mesh deployed ([ssh-mesh.md](ssh-mesh.md)) for multi-host config pull
|
||||
|
||||
## Stap 1 — Repository
|
||||
## Step 1 — Repository
|
||||
|
||||
```bash
|
||||
dnf install -y git docker-compose-plugin # of equivalent
|
||||
dnf install -y git docker-compose-plugin
|
||||
git clone http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse.git /root/lakehouse
|
||||
cd /root/lakehouse
|
||||
cp .env.example .env
|
||||
# Bewerk .env: SUPERSET_SECRET_KEY, eventueel Proxmox token
|
||||
# Edit .env: SUPERSET_SECRET_KEY, etc.
|
||||
```
|
||||
|
||||
## Stap 2 — Homepage + RSS + icons
|
||||
## Step 2 — Homepage + RSS + icons
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/docker-compose.homepage.yml up -d --build
|
||||
docker exec homepage wget -qO- http://127.0.0.1:3000/api/revalidate
|
||||
```
|
||||
|
||||
## Stap 3 — Superset
|
||||
## Step 3 — Superset
|
||||
|
||||
```bash
|
||||
docker compose -f compose/superset/docker-compose.yaml up -d --build
|
||||
```
|
||||
|
||||
## Stap 4 — Optioneel: Forgejo, LAM, cAdvisor
|
||||
## Step 4 — Optional: Forgejo, LAM, cAdvisor
|
||||
|
||||
```bash
|
||||
docker compose -f compose/forgejo/compose.yaml up -d
|
||||
docker compose -f compose/lam/compose.yml up -d
|
||||
# docker compose -f compose/cadvisor/compose.yaml up -d
|
||||
```
|
||||
|
||||
Of alles tegelijk:
|
||||
Or all at once:
|
||||
|
||||
```bash
|
||||
./scripts/deploy/deploy-atc-docker01.sh
|
||||
```
|
||||
|
||||
## Stap 5 — Andere hosts (handmatig)
|
||||
## Step 5 — Other hosts (manual)
|
||||
|
||||
Deze repo bevat **config-referenties**, geen volledige VM-images:
|
||||
This repo holds **config references**, not full VM images:
|
||||
|
||||
| Host | Herstel uit git |
|
||||
|------|-----------------|
|
||||
| Kafka connectors | `config/kafka/*.json` → deploy op Connect API |
|
||||
| Trino | `config/trino/config.properties` → `/etc/trino/` op lake01 |
|
||||
| Host | Restore from git |
|
||||
|------|------------------|
|
||||
| Kafka connectors | `config/kafka/*.json` → Connect REST API |
|
||||
| Trino | `config/trino/config.properties` → lake host |
|
||||
| Spark | `config/spark/spark-defaults.conf` |
|
||||
| Airflow | `config/airflow/` |
|
||||
| Data scripts | `config/scripts/data-generation/` |
|
||||
|
||||
Zie `inventory/containers-atc-docker01.json` voor laatst geëxporteerde containerstate.
|
||||
See `inventory/containers-atc-docker01.json` for last exported container state.
|
||||
|
||||
## Volumes (niet in git)
|
||||
## Volumes (not in git)
|
||||
|
||||
Back-up apart:
|
||||
Back up separately:
|
||||
|
||||
- `forgejo_forgejo` — Git data
|
||||
- `superset_superset_home` — Superset metadata
|
||||
- `lam_lam` — LAM configuratie
|
||||
- `lam_lam` — LAM configuration
|
||||
|
||||
```bash
|
||||
docker run --rm -v forgejo_forgejo:/data -v /backup:/backup alpine \
|
||||
tar czf /backup/forgejo_forgejo.tgz -C /data .
|
||||
```
|
||||
|
||||
## Verificatie
|
||||
## Verification
|
||||
|
||||
1. http://atc-docker01.dell-atc.lan/ — homepage laadt, 4 tabs
|
||||
1. http://atc-docker01.dell-atc.lan/ — homepage loads, 4 tabs
|
||||
2. http://atc-docker01.dell-atc.lan:8090/health — RSS proxy
|
||||
3. http://atc-docker01.dell-atc.lan:8088/health — Superset
|
||||
4. FEEDS-tab toont RSS-widgets
|
||||
4. FEEDS tab shows RSS widgets
|
||||
|
||||
## Proxmox-widget
|
||||
## Proxmox widget
|
||||
|
||||
Token staat in `config/homepage/proxmox.yaml`. Na restore: Proxmox → Permissions → API Tokens → controleer `root@pam!homepage`.
|
||||
Token in `config/homepage/proxmox.yaml`. After restore, verify in Proxmox → Permissions → API Tokens → `root@pam!homepage`.
|
||||
|
||||
Reference in New Issue
Block a user