2 Commits

Author SHA1 Message Date
mo d066def8b4 feat(cdc): add Re-sync sources button + auto-heal for Debezium connectors
- streaming_ops: add connector status discovery, manual resync endpoint
  (POST /api/pipeline/streaming/resync), connectors status endpoint, and a
  background connector_autoheal_loop that restarts FAILED tasks automatically
- main: wire connector_autoheal_loop into app lifespan
- api.ts: add resyncSources() helper
- ChangesView: add "Re-sync sources" header button with live status
2026-06-29 13:31:11 +00:00
mo 46b9c50e73 feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware
- Data Hub with Hadoop tab (HDFS/Iceberg browser, Spark, pipeline)
- Databricks-style Lakehouse Workbench (Trino engine, live exec matrix,
  materialize to Iceberg/S3); reused & embedded in every source-DB UI
- HDFS -> Kafka -> Spark -> Iceberg/S3 pipeline; WebHDFS hostname resolver
- Data Flow master pulse switch (Run/Pause/Stop) gating animated edges
- Data Custodian autonomous Hadoop offload loop (batch counterpart to CDC),
  pulsing source -> HDFS edges; toggle in Data Flow
- LLM now autonomously aware of all latest platform changes (live platform
  context) and enforces masking policy: never reveals masked PII, still
  answers helpfully with aggregates/explanations
2026-06-27 19:37:50 +00:00