Lakehouse Admin
7de4f5d73a
- config/airflow/airflow.cfg (fernet/API keys redacted) + live DAG sync - config/kafka/kraft-server.properties (live KRaft broker on kafka01) - Extend collect-fleet-config.sh for airflow/kafka collection - Add docs/recommendations.md with prioritized next steps
31 lines
751 B
Markdown
31 lines
751 B
Markdown
# Kafka — atc-kafka01
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Host | `atc-kafka01` / `10.0.21.36` |
|
|
| Mode | **KRaft** (no ZooKeeper) |
|
|
| Broker | `10.0.21.36:9092` |
|
|
| Controller | `localhost:9093` |
|
|
| UI | Kafka UI Docker — port `9000` |
|
|
| Data dir | `/home/kafka/data` |
|
|
| Systemd | `kafka.service` → `kraft/server.properties` |
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `kraft-server.properties` | **Live** broker config (systemd) |
|
|
| `server.properties` | Legacy ZK template (not used by current service) |
|
|
|
|
## Connect
|
|
|
|
Debezium Connect runs on **atc-lake01:8083** (`BOOTSTRAP_SERVERS=10.0.21.36:9092`).
|
|
|
|
Connector JSON: `../kafka/*.json` (postgres, mysql, mongodb).
|
|
|
|
## Refresh
|
|
|
|
```bash
|
|
./scripts/collect/collect-fleet-config.sh
|
|
```
|