feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

- Data Hub with Hadoop tab (HDFS/Iceberg browser, Spark, pipeline)
- Databricks-style Lakehouse Workbench (Trino engine, live exec matrix,
  materialize to Iceberg/S3); reused & embedded in every source-DB UI
- HDFS -> Kafka -> Spark -> Iceberg/S3 pipeline; WebHDFS hostname resolver
- Data Flow master pulse switch (Run/Pause/Stop) gating animated edges
- Data Custodian autonomous Hadoop offload loop (batch counterpart to CDC),
  pulsing source -> HDFS edges; toggle in Data Flow
- LLM now autonomously aware of all latest platform changes (live platform
  context) and enforces masking policy: never reveals masked PII, still
  answers helpfully with aggregates/explanations

This commit is contained in:

2026-06-27 19:37:50 +00:00

parent 5828113f53

commit 46b9c50e73

39 changed files with 5476 additions and 725 deletions

									
										ui/index.html
									
		+1
		-1
	
												View File
												
				@@ -1,5 +1,5 @@

				<!DOCTYPE html>

				<html lang="en" class="light">

				<html lang="en" class="dark">

				  <head>

				    <meta charset="UTF-8" />

				    <meta name="viewport" content="width=device-width, initial-scale=1.0" />