Commit Graph

2 Commits

Author SHA1 Message Date
mo ee9357aa31 feat(dataflow): live Data Flow graph API + PII catalog
api/dataflow.py: node-link landscape (generators->sources->CDC/Kafka->sinks,
HDFS->Iceberg, sources->curated_masked) with live overlays (movement run state,
CDC volume, Trino counts) and PII overlay. api/pii_catalog.py: column PII
classification via Trino information_schema (OM-ready). Endpoints /api/dataflow,
/api/dataflow/{id}/run, /api/pii.
2026-06-27 02:04:46 +02:00
mo 1a454f76cf feat(cdc): live Debezium CDC consumer + /api/changes for the Changes dashboard
Add api/cdc_consumer.py: aiokafka background consumer subscribes to the CDC
topics (postgres_sales/mysql_hr/mongodb_supplychain + cassandra/neo4j), parses
Debezium before/after envelopes, keeps a ring buffer and publishes each change
live as type=cdc_change. Endpoints /api/changes, /api/changes/stats, /status.
2026-06-27 01:31:18 +02:00