Files

T

Lakehouse Admin f5d30824fc Docs-as-code: full deploy configs, compose stacks, and DR guide

- Add root README and docs/ (architecture, hosts, disaster-recovery, operations)
- Add compose/ for superset, forgejo, lam, cadvisor; unified deploy compose
- Add scripts/deploy (deploy-atc-docker01.sh, container inventory export)
- Add inventory/containers-atc-docker01.json snapshot
- Add config READMEs for kafka and airflow; .env.example; expand .gitignore
- Sync atclab helper scripts; update homepage/superset compose with env support

2026-05-19 22:28:01 +02:00

2.3 KiB

Raw Blame History

Disaster recovery

Doel: volledige atc-docker01 dashboard-stack en configuratie terugzetten vanuit git.

Vereisten

VM atc-docker01 met Docker en git
Netwerk naar interne .dell-atc.lan hosts
Secrets: .env of bestaande config/homepage/proxmox.yaml

Stap 1 — Repository

dnf install -y git docker-compose-plugin   # of equivalent
git clone http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse.git /root/lakehouse
cd /root/lakehouse
cp .env.example .env
# Bewerk .env: SUPERSET_SECRET_KEY, eventueel Proxmox token

Stap 2 — Homepage + RSS + icons

docker compose -f deploy/docker-compose.homepage.yml up -d --build
docker exec homepage wget -qO- http://127.0.0.1:3000/api/revalidate

Stap 3 — Superset

docker compose -f compose/superset/docker-compose.yaml up -d --build

Stap 4 — Optioneel: Forgejo, LAM, cAdvisor

docker compose -f compose/forgejo/compose.yaml up -d
docker compose -f compose/lam/compose.yml up -d
# docker compose -f compose/cadvisor/compose.yaml up -d

Of alles tegelijk:

./scripts/deploy/deploy-atc-docker01.sh

Stap 5 — Andere hosts (handmatig)

Deze repo bevat config-referenties, geen volledige VM-images:

Host	Herstel uit git
Kafka connectors	`config/kafka/*.json` → deploy op Connect API
Trino	`config/trino/config.properties` → `/etc/trino/` op lake01
Spark	`config/spark/spark-defaults.conf`
Airflow	`config/airflow/`
Data scripts	`config/scripts/data-generation/`

Zie inventory/containers-atc-docker01.json voor laatst geëxporteerde containerstate.

Volumes (niet in git)

Back-up apart:

forgejo_forgejo — Git data
superset_superset_home — Superset metadata
lam_lam — LAM configuratie

docker run --rm -v forgejo_forgejo:/data -v /backup:/backup alpine \
  tar czf /backup/forgejo_forgejo.tgz -C /data .

Verificatie

http://atc-docker01.dell-atc.lan/ — homepage laadt, 4 tabs
http://atc-docker01.dell-atc.lan:8090/health — RSS proxy
http://atc-docker01.dell-atc.lan:8088/health — Superset
FEEDS-tab toont RSS-widgets

Token staat in config/homepage/proxmox.yaml. Na restore: Proxmox → Permissions → API Tokens → controleer root@pam!homepage.

2.3 KiB Raw Blame History