mo
f36c8906bc
Adds native Command Center features (no new containers) integrated as sub-tabs in the existing Data Explorer and Data Quality views: - Continuous Data Quality (dq_monitor.py): live completeness/uniqueness/validity/ freshness scorecards via Trino with rolling trends → DataQuality "Live Monitoring". - Ownership & stewardship (catalog_governance.py): owner/steward/tier matrix, orphan detection, business glossary; local store best-effort synced to OpenMetadata (owner PATCH) → Data Explorer "Ownership". - Access & policy posture: per-dataset compliance combining PII masking, ownership, live DQ and observability alerts vs data contracts → Data Explorer "Access & Policies". - Lineage (lineage.py): staged source→CDC→Spark→S3→Iceberg→Trino→serving graph with live row counts and column-level PII/masking tracing → Data Explorer "Lineage". - Observability (observability.py): volume/freshness/schema-drift monitoring with alerts → Data Explorer "Observability". - Shared lake_meta.py dataset registry + bounded Trino client; fast native row-count and PK-indexed freshness so monitors stay cheap on 25-54M-row tables. - LLM context (lab_context.py) enriched with DQ scores, ownership and active alerts.
ATC Command Center
Autonomous agent hub for the Dell ATC lab — Ops Floor UI, FastAPI backend, 5 operator agents.
Quick start
docker compose up -d --build
Open: http://10.0.21.33/
Stack
- ui — React dashboard + operator sprites (cap + headset)
- api — FastAPI (status, feed, prompts, WebSocket)
- redis — live event bus
- caddy — reverse proxy (:80)
VM 304 (MCP)
- IP:
10.0.21.33(DHCP on VLAN 20 /br_20) - Proxmox VMID 304 on atc-gpu
- SSH:
root/Dell2026!
Gitea
http://atc-mgt01.dell-atc.lan:3001/mo/atc-agents
ATC Command Center — Gitea layout
This repo follows the same pattern as mo/atc-GPU and mo/Lakehouse.
Related repos (Gitea @ atc-mgt01:3001)
| Repo | Path on VM304 | Purpose |
|---|---|---|
| mo/atc-agents | /opt/atc-agents |
Command Center UI + API + docker-compose |
| mo/atc-data-quality | /opt/atc-data-quality |
DQ API + RAG API |
| mo/atc-GPU | GPU lab VM303 | vLLM, model-manager |
| mo/Lakehouse | lake01 / docker hosts | Kafka, Spark, Trino, ObjectScale config |
This repo structure
atc-agents/
├── api/ FastAPI backend
├── ui/ React dashboard
├── caddy/ Reverse proxy routes
├── config/ Deploy reference (mirrors production)
│ ├── command-center/ docker-compose, Caddyfile, .env.example
│ ├── data-quality/ Link to mo/atc-data-quality
│ └── jupyter/ JupyterLab service snippet
├── docs/ Runbooks
├── scripts/ deploy.sh
└── docker-compose.yml Production stack (clone with atc-data-quality sibling)
Deploy
git clone http://atc-mgt01.dell-atc.lan:3001/mo/atc-agents.git /opt/atc-agents
git clone http://atc-mgt01.dell-atc.lan:3001/mo/atc-data-quality.git /opt/atc-data-quality
cp config/command-center/.env.example /opt/atc-agents/.env # edit secrets
./scripts/deploy.sh
Open: http://10.0.21.33/
Services (port 80 via Caddy)
| Route | Service |
|---|---|
/ |
React UI |
/api/* |
Agents API |
/dq/* |
Data Quality API |
/rag/* |
Knowledge Chat / RAG |
/jupyter/* |
JupyterLab (S3 env preconfigured) |
:5001 |
Docling UI (direct) |