The marquee panel only joined 3 region-keyed sources. Add a
"one SQL across every database" reach matrix that fans a single
Trino query out to PostgreSQL, MySQL, MongoDB, Cassandra and the
Hadoop/Iceberg lake in one UNION ALL (telemetry has no region, so
a per-source summary is used instead of a misleading join).
- New MATRIX_SQL + concurrent execution alongside the region
scorecard so total latency stays ~ the slower query
- Federated tab shows the 5-source matrix (records + headline
metric per engine) above the relabelled 3-source region scorecard
Trino Federation tab (3 sub-views):
- Federated: catalog landscape + a single cross-source SQL that joins
PostgreSQL + MySQL + MongoDB (region scorecard) — the federation proof,
computed in the background and cached (large full scans take ~2 min).
- Hadoop Lake: all federated business data materialized as external Iceberg
tables on HDFS (iceberg.hadoop.*_ext, ~120k rows each) with live, fast
business analytics (revenue by region/channel, top customers, HR by
department, supply by type, telemetry averages). Includes a one-click
"rebuild external tables" job.
- Data Dictionary: every business table + column with masked / visible PII
badges and categories.
Backend trino_federated.py: /catalogs, /marquee(+refresh), /lake,
/materialize(+status), /dictionary. Name-based PII detection flags raw PII
in derived/lake tables as visible vs physically-masked curated layer.
LLM context: platform_context now emits a full BUSINESS DATA CATALOG section
(tables, columns, types, source row counts, federated scorecard) with exact
per-column masked/visible status, so the assistant knows the data in detail
and what is masked vs not.
mo