ATC/Lakehouse
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

English docs and passwordless SSH mesh for lab fleet

Browse Source 
- Translate all documentation to English
- Add docs/ssh-mesh.md and scripts/setup/setup-ssh-mesh.sh
- Add config/hosts/atc-lab.hosts for /etc/hosts on all VMs
- Ignore atc_cluster private key in git; mesh deployed on 12 hosts
This commit is contained in:
Lakehouse Admin
2026-05-19 22:46:40 +02:00
parent f5d30824fc
commit f3d24634e4
12 changed files with 330 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+24 -22
View File
@@ -1,24 +1,24 @@
# ATC Lakehouse
Infrastructure-as-code en documentatie voor het Dell ATC Lakehouse lab — Forward Deployed Engineering (Bart & Mo).
Infrastructure-as-code and documentation for the Dell ATC Lakehouse lab — Forward Deployed Engineering (Bart & Mo).
## Repository-structuur
## Repository layout
```
├── config/ # Applicatie-configuratie (homepage, kafka, trino, spark, airflow, superset)
├── compose/ # Docker Compose per stack (bron van waarheid voor containers)
├── deploy/ # Gecombineerde compose-bestanden per host
├── docs/ # Documentatie als code
├── inventory/ # Export van draaiende containers (referentie)
├── rss-proxy/ # RSS-feed proxy voor homepage widgets
└── scripts/ # Deploy- en hulpscripts
├── config/ # Application config (homepage, kafka, trino, spark, airflow, superset)
├── compose/ # Docker Compose per stack (source of truth for containers)
├── deploy/ # Combined compose files per host
├── docs/ # Documentation as code
├── inventory/ # Exported container state (reference)
├── rss-proxy/ # RSS feed proxy for homepage widgets
└── scripts/ # Deploy and setup scripts
```
## Hosts (overzicht)
## Hosts (summary)
| Host | Rol | Belangrijkste services |
|------|-----|------------------------|
| `atc-docker01` | Docker hub | Homepage, Superset, Forgejo (lokaal), LAM, RSS-proxy, icons |
| Host | Role | Key services |
|------|------|----------------|
| `atc-docker01` | Docker hub | Homepage, Superset, Forgejo (local), LAM, RSS-proxy, icons |
| `atc-mgt01` | Management | Forgejo (Git), LDAP/LAM |
| `atc-kafka01` | Streaming | Kafka, Kafka UI |
| `atc-lake01` | Lakehouse | Spark, Trino, Debezium, Airflow |
@@ -27,22 +27,24 @@ Infrastructure-as-code en documentatie voor het Dell ATC Lakehouse lab — Forwa
| `atc-grafana` | Monitoring | Grafana |
| `pve01` (`10.0.10.65`) | Hypervisor | Proxmox VE |
Zie [docs/hosts.md](docs/hosts.md) voor poorten en IP-adressen.
See [docs/hosts.md](docs/hosts.md) for ports and IPs.
See [docs/ssh-mesh.md](docs/ssh-mesh.md) for passwordless SSH between VMs.
## Snel starten — atc-docker01
## Quick start — atc-docker01
```bash
cd /root/lakehouse
cp .env.example .env # vul secrets in
cp .env.example .env # fill in secrets
./scripts/deploy/deploy-atc-docker01.sh
```
## Documentatie
## Documentation
- [Architectuur](docs/architecture.md)
- [Hosts & poorten](docs/hosts.md)
- [Architecture](docs/architecture.md)
- [Hosts & ports](docs/hosts.md)
- [SSH mesh (passwordless fleet)](docs/ssh-mesh.md)
- [Disaster recovery](docs/disaster-recovery.md)
- [Dagelijkse operatie](docs/operations.md)
- [Day-to-day operations](docs/operations.md)
- [Homepage dashboard](config/homepage/README.md)
## Git remote
@@ -53,6 +55,6 @@ Forgejo: `http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse`
git clone http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse.git /root/lakehouse
```
## Geheimen
## Secrets
Bestanden met credentials (`proxmox.yaml`, `.env`, kafka connectors) staan in git voor **lab-restore**. Voor productie: gebruik `.env` + `*.example` en houd secrets buiten een publieke remote.
Files with credentials (`proxmox.yaml`, `.env`, kafka connectors) are in git for **lab disaster recovery**. For production, use `.env` + `*.example` and keep secrets off public remotes.