Add Airflow and Kafka broker configs to git
- config/airflow/airflow.cfg (fernet/API keys redacted) + live DAG sync - config/kafka/kraft-server.properties (live KRaft broker on kafka01) - Extend collect-fleet-config.sh for airflow/kafka collection - Add docs/recommendations.md with prioritized next steps
This commit is contained in:
+22
-17
@@ -1,25 +1,30 @@
|
||||
# Kafka Connect — Debezium connectors
|
||||
# Kafka — atc-kafka01
|
||||
|
||||
JSON-definities voor CDC van databases naar Kafka.
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Host | `atc-kafka01` / `10.0.21.36` |
|
||||
| Mode | **KRaft** (no ZooKeeper) |
|
||||
| Broker | `10.0.21.36:9092` |
|
||||
| Controller | `localhost:9093` |
|
||||
| UI | Kafka UI Docker — port `9000` |
|
||||
| Data dir | `/home/kafka/data` |
|
||||
| Systemd | `kafka.service` → `kraft/server.properties` |
|
||||
|
||||
## Connectors
|
||||
## Files
|
||||
|
||||
| Bestand | Bron | Topic prefix |
|
||||
|---------|------|--------------|
|
||||
| `postgres-connector.json` | PostgreSQL op db02 | `postgres-sales` |
|
||||
| `mysql-connector.json` | MySQL | — |
|
||||
| `mongodb-connector.json` | MongoDB | — |
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `kraft-server.properties` | **Live** broker config (systemd) |
|
||||
| `server.properties` | Legacy ZK template (not used by current service) |
|
||||
|
||||
## Deploy
|
||||
## Connect
|
||||
|
||||
Vervang `PASSWORD_PLACEHOLDER` en pas host/poort aan. Registreer via Kafka Connect REST API op `atc-lake01` (Debezium `:8083`).
|
||||
Debezium Connect runs on **atc-lake01:8083** (`BOOTSTRAP_SERVERS=10.0.21.36:9092`).
|
||||
|
||||
Connector JSON: `../kafka/*.json` (postgres, mysql, mongodb).
|
||||
|
||||
## Refresh
|
||||
|
||||
```bash
|
||||
curl -X POST -H "Content-Type: application/json" \
|
||||
--data @postgres-connector.json \
|
||||
http://atc-lake01.dell-atc.lan:8083/connectors/
|
||||
./scripts/collect/collect-fleet-config.sh
|
||||
```
|
||||
|
||||
## Referentie
|
||||
|
||||
- Kafka UI: http://atc-kafka01.dell-atc.lan:9000/
|
||||
|
||||
Reference in New Issue
Block a user