Commit Graph

3 Commits

Author SHA1 Message Date
mo 46b9c50e73 feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware
- Data Hub with Hadoop tab (HDFS/Iceberg browser, Spark, pipeline)
- Databricks-style Lakehouse Workbench (Trino engine, live exec matrix,
  materialize to Iceberg/S3); reused & embedded in every source-DB UI
- HDFS -> Kafka -> Spark -> Iceberg/S3 pipeline; WebHDFS hostname resolver
- Data Flow master pulse switch (Run/Pause/Stop) gating animated edges
- Data Custodian autonomous Hadoop offload loop (batch counterpart to CDC),
  pulsing source -> HDFS edges; toggle in Data Flow
- LLM now autonomously aware of all latest platform changes (live platform
  context) and enforces masking policy: never reveals masked PII, still
  answers helpfully with aggregates/explanations
2026-06-27 19:37:50 +00:00
mo 5828113f53 feat(ui): Neo4j graph explorer with interactive relationship visualization
Add Graph sub-tab in Data Sources UI for Neo4j: force-directed SVG view of
Product-Supplier nodes and SUPPLIES/RELATED_TO/PART_OF/COMPATIBLE_WITH edges.
Backend GET /api/sql/graph/neo4j with rel_type filter and edge limit.
2026-06-27 15:48:56 +00:00
mo a4c9b60079 feat(ui): Data Sources UI — enterprise browser for all 5 source databases
New "Data Sources UI" tab with per-database Browser (catalog + sample data),
Query Console (SqlWorkbench) and embedded interactive Shell (DbShell via SSH).

Backend:
- Extend sql_console.py with Cassandra (CQL) + Neo4j (Cypher) engines
- Add GET /api/sql/catalog/{engine} and GET /api/sql/sample/{engine}
- ssh_terminal: optional initial_command for auto-launching DB CLIs

Frontend:
- DataSourcesView with 5-DB rail, health dots, Browser/Console/Shell sub-tabs
- DbShell embedded xterm terminal with docker exec CLI per engine
- Deep-link topology DB nodes to Data Sources UI (no SQL dock on platform)
- WorkbenchPanel restricted to agent mode only — frees dashboard space
2026-06-27 15:42:37 +00:00