feat: live per-agent terminal activity with real scripts/SQL
Agent terminals were idle (one-shot probe) while agents were busy in the background. Now every agent streams what it is actually doing: - agent_terminal: emit_threadsafe() so background threads can stream lines. - agent_ops: Data Custodian DML loop logs the real INSERT/UPDATE/DELETE SQL (+ Mongo ops) and Hadoop-offload Trino CTAS/INSERT to its terminal; ETL Guardian announces each orchestrated movement. - movements: trigger_and_watch streams the Airflow DAG / API call, conf, before/after Trino counts and result to the owning agent terminal. - etl_offload: per-dataset read + pyarrow->S3 parquet writes and cycle summaries stream to the ETL Guardian terminal. - agent_activity (new): round-robin live probes for Lakehouse Ops, Hadoop Ranger (NameNode JMX + YARN), Infra Sentinel (Dockhand inventory + host load) and Network Watcher (VLAN 20/21 path checks). - fix: YARN ResourceManager runs on 10.0.21.62:8088 (was .61). - ui: terminal dock merges the selected agent ops stream with the node probe.
This commit is contained in:
@@ -46,6 +46,7 @@ from hadoop_analytics import router as hadoop_router
|
||||
from elasticsearch_api import router as elasticsearch_router
|
||||
from sql_console import router as sql_router
|
||||
from agent_ops import router as agent_ops_router, agent_dml_loop, etl_agent_loop, custodian_offload_loop
|
||||
from agent_activity import agent_activity_loop
|
||||
from cdc_consumer import router as cdc_router, cdc_consumer_loop
|
||||
from movements import router as movements_router
|
||||
from movements import MOVEMENT_BY_ID, trigger_and_watch
|
||||
@@ -735,12 +736,15 @@ async def lifespan(app: FastAPI):
|
||||
cdc_task = asyncio.create_task(cdc_consumer_loop())
|
||||
etl_task = asyncio.create_task(etl_agent_loop())
|
||||
cust_task = asyncio.create_task(custodian_offload_loop())
|
||||
act_task = asyncio.create_task(agent_activity_loop())
|
||||
add_feed("infra-sentinel", "ATC Command Center API online", "info")
|
||||
yield
|
||||
task.cancel()
|
||||
dml_task.cancel()
|
||||
cdc_task.cancel()
|
||||
etl_task.cancel()
|
||||
cust_task.cancel()
|
||||
act_task.cancel()
|
||||
if redis_client:
|
||||
await redis_client.close()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user