Add a dedicated business-data view (separate from infra/search):
- New "Data Explorer" tab: KPIs (customers, employees, products, source
row totals, sample revenue) + charts driven by Elasticsearch aggregations
(revenue over time, by region/channel/status, top customers & products,
HR by department/role/event, supply chain by type, telemetry averages),
with region + free-text filters.
- Backend /api/search/business endpoint: battery of ES aggregations with
numeric/date index template (so amount sums and date histograms work),
source totals via cheap planner estimates; biased the orders sample to
rows carrying customer names so people are searchable/visible.
Manual data entry on source systems:
- "Insert row" action in the Data Hub browser opens a column-aware form;
POST /api/sql/row/insert writes to Postgres/MySQL/Mongo/Cassandra/Neo4j.
- Inserts into CDC sources (PG/MySQL/Mongo) are captured by Debezium and
streamed to Kafka in real time; UI flags this and pulses the data flow.
When switching source engine, the sample effect fired with the previous
engine selected object (e.g. public.sales_orders against MySQL), surfacing
"Table hr.sales_orders doesn't exist". Track which engine the selected
object belongs to and only sample when it matches the active engine, plus
guard against stale responses overwriting newer ones.
Row-count for the browser used an exact count(*) which full-scanned huge
tables (~30s on 24M rows). Use planner/statistics estimates and only run a
time-bounded exact count for small (<=50k) tables.
Add Graph sub-tab in Data Sources UI for Neo4j: force-directed SVG view of
Product-Supplier nodes and SUPPLIES/RELATED_TO/PART_OF/COMPATIBLE_WITH edges.
Backend GET /api/sql/graph/neo4j with rel_type filter and edge limit.
New "Data Sources UI" tab with per-database Browser (catalog + sample data),
Query Console (SqlWorkbench) and embedded interactive Shell (DbShell via SSH).
Backend:
- Extend sql_console.py with Cassandra (CQL) + Neo4j (Cypher) engines
- Add GET /api/sql/catalog/{engine} and GET /api/sql/sample/{engine}
- ssh_terminal: optional initial_command for auto-launching DB CLIs
Frontend:
- DataSourcesView with 5-DB rail, health dots, Browser/Console/Shell sub-tabs
- DbShell embedded xterm terminal with docker exec CLI per engine
- Deep-link topology DB nodes to Data Sources UI (no SQL dock on platform)
- WorkbenchPanel restricted to agent mode only — frees dashboard space
Clicking agents opens a dedicated terminal panel; topology PostgreSQL/Trino nodes open SQL workbench with ten demo queries and Postgres vs Trino benchmark.
mo