Files
Lakehouse Admin 0402a4fbae
Validate Lakehouse config / validate (push) Has been cancelled
update (bart)
2026-06-28 17:08:57 +02:00

656 lines
23 KiB
YAML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
# ATC Lakehouse — Dell Technologies FDE Dashboard (Bart & Mo)
- Lakehouse · Architecture:
- Environment Map:
icon: mdi-sitemap
href: http://atc-docker01.dell-atc.lan:8080/docs/architecture.html
description: High-level diagram — data flow, hosts, and service map
color: "#007DB8"
- Git Docs:
icon: mdi-book-open-page-variant
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/docs/landscape.md
description: Application landscape (Markdown in Forgejo)
color: "#E8752A"
- Data Pipeline:
- Kafka UI:
icon: http://atc-docker01.dell-atc.lan:8080/apachekafka.svg
href: http://atc-kafka01.dell-atc.lan:9000/
description: atc-kafka01.dell-atc.lan:9000
siteMonitor: http://atc-kafka01.dell-atc.lan:9000/
color: "#E02014"
- Debezium:
icon: http://atc-docker01.dell-atc.lan:8080/debezium.png
href: http://atc-lake01.dell-atc.lan:8083/
description: atc-lake01.dell-atc.lan:8083
siteMonitor: http://atc-lake01.dell-atc.lan:8083/
color: "#4ECDC4"
- Spark:
icon: http://atc-docker01.dell-atc.lan:8080/apachespark.svg
href: http://atc-lake01.dell-atc.lan:8080/
description: atc-lake01.dell-atc.lan:8080
siteMonitor: http://atc-lake01.dell-atc.lan:8080/
color: "#E25A1C"
- Trino:
icon: http://atc-docker01.dell-atc.lan:8080/trino.svg
href: http://atc-lake01.dell-atc.lan:8089/ui/
description: atc-lake01.dell-atc.lan:8089/ui/
siteMonitor: http://atc-lake01.dell-atc.lan:8089/ui/
color: "#DD00A1"
- Airflow:
icon: http://atc-docker01.dell-atc.lan:8080/apacheairflow.svg
href: http://10.0.21.55:8080/
description: 10.0.21.55:8080
siteMonitor: http://10.0.21.55:8080/health
color: "#017CEE"
- Analytics:
- Superset:
icon: http://atc-docker01.dell-atc.lan:8080/apachesuperset.svg
href: http://atc-docker01.dell-atc.lan:8088/
description: atc-docker01.dell-atc.lan:8088
siteMonitor: http://atc-docker01.dell-atc.lan:8088/health
color: "#6c5ce7"
- Kibana:
icon: http://atc-docker01.dell-atc.lan:8080/kibana.svg
href: http://atc-elastic01.dell-atc.lan:5601/
description: atc-elastic01.dell-atc.lan:5601
siteMonitor: http://atc-elastic01.dell-atc.lan:5601/
color: "#F04E98"
- Elasticsearch:
icon: http://atc-docker01.dell-atc.lan:8080/elasticsearch.svg
ping: atc-elastic01.dell-atc.lan
description: |
Host: atc-elastic01.dell-atc.lan
Port: 9200 (no public HTTP)
curl: curl http://atc-elastic01.dell-atc.lan:9200
Use Kibana for browser UI
color: "#005571"
- Grafana:
icon: http://atc-docker01.dell-atc.lan:8080/grafana.svg
href: http://atc-grafana.dell-atc.lan:3000/
ping: atc-grafana.dell-atc.lan
description: |
Host: atc-grafana.dell-atc.lan
IP: 10.0.20.103
Port: 3000
Note: start with systemctl start grafana-server
siteMonitor: http://atc-grafana.dell-atc.lan:3000/
color: "#F46800"
- Infrastructure:
- ObjectScale UI:
icon: http://atc-docker01.dell-atc.lan:8080/dell.svg
href: https://10.0.20.111/
description: ECS admin · https://10.0.20.111 · luna.local
siteMonitor: https://10.0.20.111/
color: "#007DB8"
- ObjectScale S3:
icon: http://atc-docker01.dell-atc.lan:8080/minio.svg
href: http://10.0.20.111:9020/
description: S3 API :9020 · bucket data · Trino Iceberg + Spark
siteMonitor: http://10.0.20.111:9020/
color: "#C72C48"
- ObjectScale SSH:
icon: mdi-console
href: https://10.0.20.111/
description: admin@atc-objectscale · appliance SSH · see docs/objectscale.md
ping: 10.0.20.111
color: "#64748b"
- iDRAC:
icon: http://atc-docker01.dell-atc.lan:8080/dell.svg
href: https://10.0.41.102/
description: Dell iDRAC · 10.0.41.102
siteMonitor: https://10.0.41.102/
color: "#007DB8"
- Proxmox:
icon: http://atc-docker01.dell-atc.lan:8080/proxmox.svg
href: https://10.0.10.65:8006/
description: 10.0.10.65:8006
siteMonitor: https://10.0.10.65:8006/
color: "#E57000"
widget:
type: proxmox
url: https://10.0.10.65:8006
username: "root@pam!homepage"
password: "8890185b-0850-42b7-bab2-a690ea4dc3f1"
node: pve01
fields: ["vms", "lxc", "resources.cpu", "resources.mem"]
- Forgejo:
icon: http://atc-docker01.dell-atc.lan:8080/git.svg
href: http://atc-mgt01.dell-atc.lan:3001/
description: atc-mgt01.dell-atc.lan:3001
siteMonitor: http://atc-mgt01.dell-atc.lan:3001/
color: "#F05032"
- GPU Lab:
icon: mdi-gpu
href: http://10.0.20.106:9000/
description: atc-gpu-dev VM303 · 4× V100 · model manager + chat
siteMonitor: http://10.0.20.106:9000/
color: "#76B900"
- Dockhand:
icon: mdi-docker
href: http://atc-docker01.dell-atc.lan:8082/
description: Docker control plane · GPU-Dev + Bart-GPU + Mo-GPU
siteMonitor: http://atc-docker01.dell-atc.lan:8082/api/health
color: "#2496ED"
- ATC Command Center:
icon: mdi-robot-outline
href: http://10.0.21.33/
description: VM304 MCP · agent hub & ops floor · 10.0.21.33
siteMonitor: http://10.0.21.33/
color: "#0099cc"
- LDAP Admin:
icon: http://atc-docker01.dell-atc.lan:8080/ldap.png
href: http://atc-mgt01.dell-atc.lan/lam/
description: atc-mgt01.dell-atc.lan/lam/
siteMonitor: http://atc-mgt01.dell-atc.lan/lam/
color: "#0984e3"
- Databases:
- PostgreSQL · db01:
icon: http://atc-docker01.dell-atc.lan:8080/postgresql.svg
ping: atc-db01.dell-atc.lan
description: atc-db01:5432 · postgresql://USER@atc-db01.dell-atc.lan:5432/DB · psql -h atc-db01
color: "#4169E1"
- MySQL · db01:
icon: http://atc-docker01.dell-atc.lan:8080/mysql.svg
ping: atc-db01.dell-atc.lan
description: atc-db01:3306 · mysql://USER@atc-db01.dell-atc.lan:3306/DB · mysql -h atc-db01
color: "#4479A1"
- Mongo Express · db02:
icon: http://atc-docker01.dell-atc.lan:8080/mongodb.svg
href: http://atc-db02.dell-atc.lan:8081/
description: Web UI · supplychain · 3M events · users mo & bart
siteMonitor: http://atc-db02.dell-atc.lan:8081/
ping: atc-db02.dell-atc.lan
color: "#47A248"
- MongoDB · db02:
icon: http://atc-docker01.dell-atc.lan:8080/mongodb.svg
ping: atc-db02.dell-atc.lan
description: |
:27017 · mongodb_supplychain container
DB supplychain · collection events (~3M)
mongosh: mongodb://mo@atc-db02.dell-atc.lan:27017/supplychain?authSource=admin
Trino: mongodb_supplychain.supplychain.events
color: "#3d8b40"
- PostgreSQL · db02:
icon: http://atc-docker01.dell-atc.lan:8080/postgresql.svg
ping: atc-db02.dell-atc.lan
description: atc-db02:5432 · postgresql://USER@atc-db02.dell-atc.lan:5432/DB · psql -h atc-db02
color: "#336791"
- MySQL · db02:
icon: http://atc-docker01.dell-atc.lan:8080/mysql.svg
ping: atc-db02.dell-atc.lan
description: atc-db02:3306 · mysql://USER@atc-db02.dell-atc.lan:3306/DB · mysql -h atc-db02
color: "#00758F"
- Cassandra · db02:
icon: http://atc-docker01.dell-atc.lan:8080/apachecassandra.svg
ping: atc-db02.dell-atc.lan
description: atc-db02:9042 · cqlsh atc-db02.dell-atc.lan 9042 · cassandra://atc-db02:9042
color: "#1287B1"
- Neo4j · db02:
icon: http://atc-docker01.dell-atc.lan:8080/neo4j.svg
href: http://atc-db02.dell-atc.lan:7474/
ping: atc-db02.dell-atc.lan
description: HTTP :7474 · bolt://atc-db02:7687 · neo4j://atc-db02.dell-atc.lan:7687
siteMonitor: http://atc-db02.dell-atc.lan:7474/
color: "#008CC1"
- Intel · Hacker News:
- HN Front Page:
icon: mdi-newspaper
href: https://news.ycombinator.com/
description: Hacker News — front page
color: "#E8752A"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/hn-front
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- HN Data Engineering:
icon: mdi-database-search
href: https://hn.algolia.com/?q=data%20engineering
description: Hacker News — data engineering
color: "#3b82f6"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/hn-de
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- HN Kafka:
icon: http://atc-docker01.dell-atc.lan:8080/apachekafka.svg
href: https://hn.algolia.com/?q=kafka
description: Hacker News — Kafka & streaming
color: "#E02014"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/hn-kafka
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- HN Spark:
icon: http://atc-docker01.dell-atc.lan:8080/apachespark.svg
href: https://hn.algolia.com/?q=apache+spark
description: Hacker News — Apache Spark
color: "#E25A1C"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/hn-spark
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Lobsters:
icon: mdi-lobster
href: https://lobste.rs/
description: Computing & infra — curated links
color: "#ef4444"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/lobsters
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Intel · Data Architecture:
- Data Eng Weekly:
icon: mdi-calendar-week
href: https://www.dataengineeringweekly.com/
description: Weekly newsletter — pipelines & platforms
color: "#22d3ee"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/de-weekly
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Pragmatic Engineer:
icon: mdi-account-tie
href: https://blog.pragmaticengineer.com/
description: Big tech engineering & architecture
color: "#a855f7"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/pragmatic
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Martin Fowler:
icon: mdi-arch
href: https://martinfowler.com/
description: Software architecture & design
color: "#64748b"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/martinfowler
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- InfoQ:
icon: mdi-information-outline
href: https://www.infoq.com/
description: Architecture, data & dev practices
color: "#94a3b8"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/infoq
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- ByteByteGo:
icon: mdi-school
href: https://blog.bytebytego.com/
description: System design — scalable architectures
color: "#f59e0b"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/bytebytego
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- TLDR Data Eng:
icon: mdi-lightning-bolt
href: https://tldr.tech/dataengineering/
description: Daily data engineering digest
color: "#E8752A"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/tldr-de
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Seattle Data Guy:
icon: mdi-chart-bar
href: https://www.seattledataguy.com/
description: Practical data engineering tutorials
color: "#0ea5e9"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/seattle-de
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- RedMonk:
icon: mdi-chart-line
href: https://redmonk.com/
description: Developer-focused industry analysis
color: "#dc2626"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/redmonk
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Intel · Platforms & Blogs:
- Confluent Blog:
icon: http://atc-docker01.dell-atc.lan:8080/apachekafka.svg
href: https://www.confluent.io/blog/
description: Kafka & event streaming
color: "#007DB8"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/confluent
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Databricks Blog:
icon: mdi-layers-triple
href: https://www.databricks.com/blog
description: Lakehouse & Spark platform
color: "#fb923c"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/databricks
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Debezium CDC:
icon: http://atc-docker01.dell-atc.lan:8080/debezium.png
href: https://debezium.io/blog/
description: Change data capture patterns
color: "#4ECDC4"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/debezium
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- AWS Big Data:
icon: mdi-aws
href: https://aws.amazon.com/blogs/big-data/
description: AWS analytics & data lakes
color: "#FF9900"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/aws-bigdata
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Google Cloud Blog:
icon: mdi-google-cloud
href: https://cloud.google.com/blog
description: GCP data & analytics
color: "#4285F4"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/google-cloud
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Cloudflare Eng:
icon: mdi-cloud
href: https://blog.cloudflare.com/
description: Edge, networking & scale
color: "#F38020"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/cloudflare
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Meta Engineering:
icon: mdi-facebook
href: https://engineering.fb.com/
description: Large-scale infra & data
color: "#1877F2"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/fb-engineering
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Airflow Blog:
icon: http://atc-docker01.dell-atc.lan:8080/apacheairflow.svg
href: https://airflow.apache.org/blog/
description: Workflow orchestration & DAGs
color: "#017CEE"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/airflow
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Elastic Blog:
icon: http://atc-docker01.dell-atc.lan:8080/elasticsearch.svg
href: https://www.elastic.co/blog/
description: Search, observability & security
color: "#005571"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/elastic
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- The New Stack:
icon: mdi-newspaper-variant-multiple
href: https://thenewstack.io/
description: Cloud native & platform engineering
color: "#22c55e"
widget:
type: customapi
url: http://atc-docker01.dell-atc.lan:8090/feed/thenewstack
refreshInterval: 300000
display: dynamic-list
mappings:
items: items
name: title
href: link
target: _blank
- Documentation:
- Architecture Diagram:
icon: mdi-sitemap
href: http://atc-docker01.dell-atc.lan:8080/docs/architecture.html
description: High-level environment map (Mermaid)
color: "#007DB8"
- Application Landscape:
icon: mdi-book-open-variant
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/docs/landscape.md
description: docs/landscape.md in Forgejo
color: "#E8752A"
- Network Diagram:
icon: mdi-lan
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/docs/network.md
description: Subnets, ports, SSH mesh
color: "#22d3ee"
- Disaster Recovery:
icon: mdi-backup-restore
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/docs/disaster-recovery.md
description: Restore procedures from git
color: "#a855f7"
- Docker Inventory:
icon: mdi-docker
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/docs/docker-inventory.md
description: Containers per host
color: "#2496ED"
- Proxmox VMs:
icon: mdi-server
href: http://atc-mgt01.dell-atc.lan:3001/mo/Lakehouse/src/branch/master/inventory/proxmox-vms.json
description: VM inventory (JSON)
color: "#E57000"
- News & Resources:
- Dell Technologies:
icon: http://atc-docker01.dell-atc.lan:8080/dell-technologies.svg
href: https://www.delltechnologies.com/
description: Corporate site & solutions
color: "#007DB8"
- Dell Blog · Data:
icon: mdi-post-outline
href: https://www.dell.com/en-us/blog/categories/products-solutions-analytics
description: Analytics & data solutions
color: "#007DB8"
- Apache Spark:
icon: http://atc-docker01.dell-atc.lan:8080/apachespark.svg
href: https://spark.apache.org/docs/latest/
description: Spark documentation
color: "#E25A1C"
- Trino:
icon: http://atc-docker01.dell-atc.lan:8080/trino.svg
href: https://trino.io/docs/current/
description: Distributed SQL engine docs
color: "#DD00A1"
- Apache Iceberg:
icon: mdi-snowflake
href: https://iceberg.apache.org/docs/latest/
description: Open table format for lakes
color: "#38bdf8"
- Real Python:
icon: mdi-language-python
href: https://realpython.com/
description: Python tutorials & patterns
color: "#3776AB"
- MinIO Docs:
icon: http://atc-docker01.dell-atc.lan:8080/minio.svg
href: https://min.io/docs/minio/linux/index.html
description: S3-compatible object storage
color: "#C72C48"
- TLDR Tech:
icon: mdi-lightning-bolt-outline
href: https://tldr.tech/
description: Daily tech newsletter (web)
color: "#E8752A"
- Docker · atc-docker01:
- Homepage:
icon: mdi-view-dashboard
href: http://atc-docker01.dell-atc.lan/
description: atc-docker01.dell-atc.lan
server: my-docker
container: homepage
color: "#E8752A"
- Superset:
icon: http://atc-docker01.dell-atc.lan:8080/apachesuperset.svg
href: http://atc-docker01.dell-atc.lan:8088/
description: atc-docker01.dell-atc.lan:8088
server: my-docker
container: superset
color: "#6c5ce7"
- Forgejo Local:
icon: http://atc-docker01.dell-atc.lan:8080/git.svg
href: http://atc-docker01.dell-atc.lan:4002/
description: atc-docker01.dell-atc.lan:4002
siteMonitor: http://atc-docker01.dell-atc.lan:4002/
server: my-docker
container: forgejo
color: "#F05032"