mo
b6d7d3dc74
- etl_offload.py: autonomous agent backfills/tails source DBs (PG/MySQL/ Mongo/Cassandra) to S3 as Parquet in small chunks, accumulates a live federated business matrix (/api/etl/status, /api/etl/business, /run, /config). - storage_s3.py: buffer generated CDC + masked curated rows to S3, overlay live last-write into analytics; put_object_bytes for Parquet parts. - trino_federated.py: capture generated rows + archive to S3; generator_active. - dataflow.py: pulse generate + kafka/spark->S3 archive edges when active. - StorageView: realtime ETL ingest panel; TrinoFederationView: realtime business KPIs/charts from /api/etl/business. - ChangesView: top KPIs/charts now overlay the live WS stream on server stats so they update in lock-step with the bottom feed; faster 2.5s refresh. - useCommandCenter: retain 800 live CDC changes.
ATC Command Center
Autonomous agent hub for the Dell ATC lab — Ops Floor UI, FastAPI backend, 5 operator agents.
Quick start
docker compose up -d --build
Open: http://10.0.21.33/
Stack
- ui — React dashboard + operator sprites (cap + headset)
- api — FastAPI (status, feed, prompts, WebSocket)
- redis — live event bus
- caddy — reverse proxy (:80)
VM 304 (MCP)
- IP:
10.0.21.33(DHCP on VLAN 20 /br_20) - Proxmox VMID 304 on atc-gpu
- SSH:
root/Dell2026!
Gitea
http://atc-mgt01.dell-atc.lan:3001/mo/atc-agents
ATC Command Center — Gitea layout
This repo follows the same pattern as mo/atc-GPU and mo/Lakehouse.
Related repos (Gitea @ atc-mgt01:3001)
| Repo | Path on VM304 | Purpose |
|---|---|---|
| mo/atc-agents | /opt/atc-agents |
Command Center UI + API + docker-compose |
| mo/atc-data-quality | /opt/atc-data-quality |
DQ API + RAG API |
| mo/atc-GPU | GPU lab VM303 | vLLM, model-manager |
| mo/Lakehouse | lake01 / docker hosts | Kafka, Spark, Trino, ObjectScale config |
This repo structure
atc-agents/
├── api/ FastAPI backend
├── ui/ React dashboard
├── caddy/ Reverse proxy routes
├── config/ Deploy reference (mirrors production)
│ ├── command-center/ docker-compose, Caddyfile, .env.example
│ ├── data-quality/ Link to mo/atc-data-quality
│ └── jupyter/ JupyterLab service snippet
├── docs/ Runbooks
├── scripts/ deploy.sh
└── docker-compose.yml Production stack (clone with atc-data-quality sibling)
Deploy
git clone http://atc-mgt01.dell-atc.lan:3001/mo/atc-agents.git /opt/atc-agents
git clone http://atc-mgt01.dell-atc.lan:3001/mo/atc-data-quality.git /opt/atc-data-quality
cp config/command-center/.env.example /opt/atc-agents/.env # edit secrets
./scripts/deploy.sh
Open: http://10.0.21.33/
Services (port 80 via Caddy)
| Route | Service |
|---|---|
/ |
React UI |
/api/* |
Agents API |
/dq/* |
Data Quality API |
/rag/* |
Knowledge Chat / RAG |
/jupyter/* |
JupyterLab (S3 env preconfigured) |
:5001 |
Docling UI (direct) |