mo
1e2cfe80f2
Agent terminals were idle (one-shot probe) while agents were busy in the background. Now every agent streams what it is actually doing: - agent_terminal: emit_threadsafe() so background threads can stream lines. - agent_ops: Data Custodian DML loop logs the real INSERT/UPDATE/DELETE SQL (+ Mongo ops) and Hadoop-offload Trino CTAS/INSERT to its terminal; ETL Guardian announces each orchestrated movement. - movements: trigger_and_watch streams the Airflow DAG / API call, conf, before/after Trino counts and result to the owning agent terminal. - etl_offload: per-dataset read + pyarrow->S3 parquet writes and cycle summaries stream to the ETL Guardian terminal. - agent_activity (new): round-robin live probes for Lakehouse Ops, Hadoop Ranger (NameNode JMX + YARN), Infra Sentinel (Dockhand inventory + host load) and Network Watcher (VLAN 20/21 path checks). - fix: YARN ResourceManager runs on 10.0.21.62:8088 (was .61). - ui: terminal dock merges the selected agent ops stream with the node probe.
12 lines
961 B
Docker
12 lines
961 B
Docker
FROM python:3.12-slim
|
|
|
|
WORKDIR /app
|
|
RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
|
|
COPY requirements.txt .
|
|
RUN pip install --no-cache-dir -r requirements.txt
|
|
COPY main.py lab_context.py agent_terminal.py workload.py node_registry.py node_ops.py topology_views.py supervisor.py approval_service.py db.py dockhand_envs.py presentation.py database_inventory.py presentation_upload.py presentation_static.py storage_s3.py elasticsearch_api.py sql_console.py hdfs_api.py ssh_terminal.py pipeline_ops.py hadoop_analytics.py agent_ops.py cdc_consumer.py movements.py dataflow.py streaming_ops.py spark_workbench.py hadoop_sql.py hdfs_kafka.py webhdfs_util.py pii_catalog.py platform_context.py trino_federated.py etl_offload.py agent_activity.py hive_bench_seed.json .
|
|
RUN mkdir -p /data
|
|
ENV DATABASE_URL=sqlite:////data/atc-agents.db
|
|
EXPOSE 3201
|
|
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "3201"]
|