English docs and passwordless SSH mesh for lab fleet

- Translate all documentation to English
- Add docs/ssh-mesh.md and scripts/setup/setup-ssh-mesh.sh
- Add config/hosts/atc-lab.hosts for /etc/hosts on all VMs
- Ignore atc_cluster private key in git; mesh deployed on 12 hosts
This commit is contained in:
Lakehouse Admin
2026-05-19 22:46:40 +02:00
parent f5d30824fc
commit f3d24634e4
12 changed files with 330 additions and 118 deletions
+28 -28
View File
@@ -1,84 +1,84 @@
# Disaster recovery
Doel: volledige **atc-docker01** dashboard-stack en configuratie terugzetten vanuit git.
Goal: fully restore the **atc-docker01** dashboard stack and configuration from git.
## Vereisten
## Requirements
- VM `atc-docker01` met Docker en git
- Netwerk naar interne `.dell-atc.lan` hosts
- Secrets: `.env` of bestaande `config/homepage/proxmox.yaml`
- VM `atc-docker01` with Docker and git
- Network to internal `.dell-atc.lan` hosts
- Secrets: `.env` or existing `config/homepage/proxmox.yaml`
- SSH mesh deployed ([ssh-mesh.md](ssh-mesh.md)) for multi-host config pull
## Stap 1 — Repository
## Step 1 — Repository
```bash
dnf install -y git docker-compose-plugin # of equivalent
dnf install -y git docker-compose-plugin
git clone http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse.git /root/lakehouse
cd /root/lakehouse
cp .env.example .env
# Bewerk .env: SUPERSET_SECRET_KEY, eventueel Proxmox token
# Edit .env: SUPERSET_SECRET_KEY, etc.
```
## Stap 2 — Homepage + RSS + icons
## Step 2 — Homepage + RSS + icons
```bash
docker compose -f deploy/docker-compose.homepage.yml up -d --build
docker exec homepage wget -qO- http://127.0.0.1:3000/api/revalidate
```
## Stap 3 — Superset
## Step 3 — Superset
```bash
docker compose -f compose/superset/docker-compose.yaml up -d --build
```
## Stap 4 — Optioneel: Forgejo, LAM, cAdvisor
## Step 4 — Optional: Forgejo, LAM, cAdvisor
```bash
docker compose -f compose/forgejo/compose.yaml up -d
docker compose -f compose/lam/compose.yml up -d
# docker compose -f compose/cadvisor/compose.yaml up -d
```
Of alles tegelijk:
Or all at once:
```bash
./scripts/deploy/deploy-atc-docker01.sh
```
## Stap 5 — Andere hosts (handmatig)
## Step 5 — Other hosts (manual)
Deze repo bevat **config-referenties**, geen volledige VM-images:
This repo holds **config references**, not full VM images:
| Host | Herstel uit git |
|------|-----------------|
| Kafka connectors | `config/kafka/*.json` deploy op Connect API |
| Trino | `config/trino/config.properties``/etc/trino/` op lake01 |
| Host | Restore from git |
|------|------------------|
| Kafka connectors | `config/kafka/*.json` → Connect REST API |
| Trino | `config/trino/config.properties`lake host |
| Spark | `config/spark/spark-defaults.conf` |
| Airflow | `config/airflow/` |
| Data scripts | `config/scripts/data-generation/` |
Zie `inventory/containers-atc-docker01.json` voor laatst geëxporteerde containerstate.
See `inventory/containers-atc-docker01.json` for last exported container state.
## Volumes (niet in git)
## Volumes (not in git)
Back-up apart:
Back up separately:
- `forgejo_forgejo` — Git data
- `superset_superset_home` — Superset metadata
- `lam_lam` — LAM configuratie
- `lam_lam` — LAM configuration
```bash
docker run --rm -v forgejo_forgejo:/data -v /backup:/backup alpine \
tar czf /backup/forgejo_forgejo.tgz -C /data .
```
## Verificatie
## Verification
1. http://atc-docker01.dell-atc.lan/ — homepage laadt, 4 tabs
1. http://atc-docker01.dell-atc.lan/ — homepage loads, 4 tabs
2. http://atc-docker01.dell-atc.lan:8090/health — RSS proxy
3. http://atc-docker01.dell-atc.lan:8088/health — Superset
4. FEEDS-tab toont RSS-widgets
4. FEEDS tab shows RSS widgets
## Proxmox-widget
## Proxmox widget
Token staat in `config/homepage/proxmox.yaml`. Na restore: Proxmox → Permissions → API Tokens → controleer `root@pam!homepage`.
Token in `config/homepage/proxmox.yaml`. After restore, verify in Proxmox → Permissions → API Tokens → `root@pam!homepage`.