Add Airflow and Kafka broker configs to git

- config/airflow/airflow.cfg (fernet/API keys redacted) + live DAG sync
- config/kafka/kraft-server.properties (live KRaft broker on kafka01)
- Extend collect-fleet-config.sh for airflow/kafka collection
- Add docs/recommendations.md with prioritized next steps
This commit is contained in:
Lakehouse Admin
2026-05-19 23:08:04 +02:00
parent 3f85ec8034
commit 7de4f5d73a
10 changed files with 351 additions and 36 deletions
+30 -9
View File
@@ -1,14 +1,35 @@
# Airflow
# Airflow — atc-airflow01
| Bestand | Beschrijving |
|---------|--------------|
| `airflow.cfg` | Referentie-configuratie (deploy naar Airflow home op lake01) |
| `generate_data_dag.py` | DAG voor demo data-generatie |
| Item | Value |
|------|-------|
| Host | `atc-airflow01` / `10.0.21.55` |
| URL | http://atc-airflow01:8080/ or http://10.0.21.55:8080/ |
| Home | `/root/airflow/` |
| Config | `/root/airflow/airflow.cfg` |
| DAGs | `/root/airflow/dags/` |
| DB | SQLite (`airflow.db` — not in git) |
## URL
## Files in git
http://atc-lake01.dell-atc.lan:8080/ (of `10.0.21.55:8080` volgens netwerk)
| File | Notes |
|------|-------|
| `airflow.cfg` | Reference copy; `fernet_key` and `internal_api_secret_key` redacted |
| `generate_data_dag.py` | Synced from live DAGs folder |
| `../airflow/airflow.cfg` | Older path (legacy); prefer this directory |
## Deploy
## Restore
Kopieer `generate_data_dag.py` naar de Airflow `dags/` folder op de lake-host en herstart de scheduler/webserver.
```bash
# On atc-airflow01
mkdir -p /root/airflow/dags
cp airflow.cfg /root/airflow/
cp generate_data_dag.py /root/airflow/dags/
# Set real fernet_key in airflow.cfg or via env
systemctl restart airflow-webserver airflow-scheduler # if using systemd
```
## Refresh
```bash
./scripts/collect/collect-fleet-config.sh
```