mo 46b9c50e73 feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware ...

- Data Hub with Hadoop tab (HDFS/Iceberg browser, Spark, pipeline)
- Databricks-style Lakehouse Workbench (Trino engine, live exec matrix,
  materialize to Iceberg/S3); reused & embedded in every source-DB UI
- HDFS -> Kafka -> Spark -> Iceberg/S3 pipeline; WebHDFS hostname resolver
- Data Flow master pulse switch (Run/Pause/Stop) gating animated edges
- Data Custodian autonomous Hadoop offload loop (batch counterpart to CDC),
  pulsing source -> HDFS edges; toggle in Data Flow
- LLM now autonomously aware of all latest platform changes (live platform
  context) and enforces masking policy: never reveals masked PII, still
  answers helpfully with aggregates/explanations

2026-06-27 19:37:50 +00:00

..

agent_ops.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

agent_terminal.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

approval_service.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

cdc_consumer.py

feat(dataflow): live Data Flow graph API + PII catalog

2026-06-27 02:04:46 +02:00

database_inventory.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

dataflow.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

db.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

Dockerfile

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

dockhand_envs.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

elasticsearch_api.py

Full stack visibility: mo S3 buckets, Elasticsearch/Kibana UI and topology

2026-06-25 00:53:11 +00:00

hadoop_analytics.py

feat(hadoop): real Hive engine + filterable, detailed analytics with lineage

2026-06-26 17:08:57 +00:00

hadoop_sql.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

hdfs_api.py

Add HDFS browser, SSH terminal, presentation editor, lab health panel

2026-06-26 00:47:49 +00:00

hdfs_kafka.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

hive_bench_seed.json

feat(hadoop): real Hive engine + filterable, detailed analytics with lineage

2026-06-26 17:08:57 +00:00

lab_context.py

feat(llm): add governance context (CDC/movements/PII/lineage) to lab snapshot

2026-06-27 03:17:35 +02:00

main.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

movements.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

node_ops.py

Add HDFS browser, SSH terminal, presentation editor, lab health panel

2026-06-26 00:47:49 +00:00

node_registry.py

feat(om): OpenMetadata integration — registry node, link, PII tag source

2026-06-27 02:40:27 +02:00

pii_catalog.py

perf(pii): native source-DB lookups for masking enforcement

2026-06-27 11:44:16 +02:00

pipeline_ops.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

platform_context.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

presentation_static.py

Full stack visibility: mo S3 buckets, Elasticsearch/Kibana UI and topology

2026-06-25 00:53:11 +00:00

presentation_upload.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

presentation.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

requirements.txt

feat(cdc): live Debezium CDC consumer + /api/changes for the Changes dashboard

2026-06-27 01:31:18 +02:00

spark_workbench.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

sql_console.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

ssh_terminal.py

feat(ui): Data Sources UI — enterprise browser for all 5 source databases

2026-06-27 15:42:37 +00:00

storage_s3.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

streaming_ops.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

supervisor.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

topology_views.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00

webhdfs_util.py

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

2026-06-27 19:37:50 +00:00

workload.py

Add Command Center v2: DQ/RAG integration, S3 browser, Jupyter, GPU matrix.

2026-06-25 00:28:23 +00:00