Files

T

Lakehouse Admin df5ec93dc3 Full lab documentation and infrastructure as code

- Trino catalogs, Grafana, Spark jobs, LDAP LDIF, NPM compose
- Airflow DAG scripts, Proxmox VM inventory, network docs
- Ansible playbook, Gitea CI validate workflow
- Backup and health-check scripts, cron documentation
- Homepage DOCS tab with links to all documentation
- Extended collect-fleet-config.sh and populate-repo.py

2026-05-19 23:12:41 +02:00

1.6 KiB

Raw Permalink Blame History

Network — ATC Lakehouse lab

Subnets

flowchart TB
  subgraph mgmt [10.0.10.0/24 — Management]
    PVE[pve01 · 10.0.10.65]
    IDRAC[iDRAC · 10.0.41.102]
  end

  subgraph data20 [10.0.20.0/24 — Storage & legacy]
    OS[ObjectScale · 10.0.20.111]
    DB1[atc-db01 · 10.0.20.112]
    MGT[atc-mgt01 · 10.0.20.104]
    GF[atc-grafana · 10.0.20.103]
  end

  subgraph lake21 [10.0.21.0/24 — Lakehouse compute]
    D1[atc-docker01 · 10.0.21.45]
    D2[atc-docker02 · 10.0.21.47]
    KF[atc-kafka01 · 10.0.21.36]
    LK[atc-lake01 · 10.0.21.50]
    DB2[atc-db02 · 10.0.21.51]
    AF[atc-airflow01 · 10.0.21.55]
    EL[atc-elastic01 · 10.0.21.46]
  end

  PVE --> D1 & D2 & KF & LK & DB2 & OS & DB1 & MGT

DNS

Internal names: *.dell-atc.lan (see config/hosts/atc-lab.hosts).

Pattern	Example
Short hostname	`atc-lake01`
FQDN	`atc-lake01.dell-atc.lan`

Key ports (east-west)

Service	Port	Protocol
Kafka	9092	TCP
Debezium Connect	8083	HTTP
Spark UI	8080	HTTP
Trino	8089	HTTP
PostgreSQL	5432	TCP
ObjectScale S3	9020	HTTP
ObjectScale UI	443	HTTPS
Homepage	80	HTTP

SSH mesh

All lab VMs: /root/.ssh/atc_cluster — see ssh-mesh.md.

ObjectScale uses admin@ for initial bootstrap; root@ after key install.

Firewall notes

Lab assumes flat L2/L3 trust within 10.0.10/20/21.x.
ObjectScale management_clients: 0.0.0.0/0 in deploy.yml — tighten for production.
Proxmox API :8006 reachable from docker01 for homepage widget.

1.6 KiB Raw Permalink Blame History