ATC/atc-agents
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Spark Workbench everywhere, autonomous Hadoop offload & LLM masking-aware

Browse Source 
- Data Hub with Hadoop tab (HDFS/Iceberg browser, Spark, pipeline)
- Databricks-style Lakehouse Workbench (Trino engine, live exec matrix,
  materialize to Iceberg/S3); reused & embedded in every source-DB UI
- HDFS -> Kafka -> Spark -> Iceberg/S3 pipeline; WebHDFS hostname resolver
- Data Flow master pulse switch (Run/Pause/Stop) gating animated edges
- Data Custodian autonomous Hadoop offload loop (batch counterpart to CDC),
  pulsing source -> HDFS edges; toggle in Data Flow
- LLM now autonomously aware of all latest platform changes (live platform
  context) and enforces masking policy: never reveals masked PII, still
  answers helpfully with aggregates/explanations
This commit is contained in:
mo
2026-06-27 19:37:50 +00:00
parent 5828113f53
commit 46b9c50e73
39 changed files with 5476 additions and 725 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/nginx.conf
+25
View File
@@ -14,6 +14,31 @@ server {
proxy_buffering off;
}
location /spark-ui/ {
proxy_pass http://10.0.21.50:8080/;
proxy_http_version 1.1;
proxy_set_header Host 10.0.21.50:8080;
proxy_hide_header X-Frame-Options;
proxy_redirect http://10.0.21.50:8080/ /spark-ui/;
sub_filter_once off;
sub_filter 'href="/' 'href="/spark-ui/';
sub_filter 'src="/' 'src="/spark-ui/';
sub_filter "href='/" "href='/spark-ui/";
sub_filter_types text/css application/javascript text/html;
}
location /kafka-ui/ {
proxy_pass http://10.0.21.36:9000/;
proxy_http_version 1.1;
proxy_set_header Host 10.0.21.36:9000;
proxy_hide_header X-Frame-Options;
proxy_redirect http://10.0.21.36:9000/ /kafka-ui/;
sub_filter_once off;
sub_filter 'href="/' 'href="/kafka-ui/';
sub_filter 'src="/' 'src="/kafka-ui/';
sub_filter_types text/html application/javascript text/css;
}
location /assets/ {
add_header Cache-Control "public, max-age=31536000, immutable";
try_files $uri =404;