3 Commits

Author	SHA1	Message	Date
mo	a724615a9a	feat(etl): Hadoop->Trino movement DAG + movements registry + autonomous ETL-agents ... - DAG hadoop_to_trino (deployed to Airflow) + worker hadoop_to_trino.py (on the Hadoop master) move HDFS historical_sales -> iceberg.hadoop.historical_sales_hdfs. - api/movements.py: movement registry, Airflow trigger+watch, run tracking (state/duration/rows), endpoints /api/movements, /{id}/run, /runs. - agent_ops.py: ETL-agent loop autonomously triggers movements on an interval and logs each run; /api/agent-ops/etl/toggle + etl status.	2026-06-27 01:59:35 +02:00
mo	5147538b05	fix(agents): use in-memory PK pools for DML (avoid full-table scans on 54M-row tables)	2026-06-27 01:27:56 +02:00
mo	3c9661e7f9	feat(agents): autonomous guard-railed DML on source DBs for Debezium CDC ... Add api/agent_ops.py: background loop performs small INSERT/UPDATE/DELETE on public.sales_orders (PG), hr.employee_events (MySQL) and supplychain.events (Mongo). Agent rows are tagged (notes/atc_agent); UPDATE/DELETE only ever touch agent-created rows. Env kill-switch + interval + per-tick row cap. Endpoints /api/agent-ops/{status,toggle,run-once}. Loop started in lifespan.	2026-06-27 01:25:08 +02:00