mo
b6d7d3dc74
- etl_offload.py: autonomous agent backfills/tails source DBs (PG/MySQL/ Mongo/Cassandra) to S3 as Parquet in small chunks, accumulates a live federated business matrix (/api/etl/status, /api/etl/business, /run, /config). - storage_s3.py: buffer generated CDC + masked curated rows to S3, overlay live last-write into analytics; put_object_bytes for Parquet parts. - trino_federated.py: capture generated rows + archive to S3; generator_active. - dataflow.py: pulse generate + kafka/spark->S3 archive edges when active. - StorageView: realtime ETL ingest panel; TrinoFederationView: realtime business KPIs/charts from /api/etl/business. - ChangesView: top KPIs/charts now overlay the live WS stream on server stats so they update in lock-step with the bottom feed; faster 2.5s refresh. - useCommandCenter: retain 800 live CDC changes.
20 lines
343 B
Plaintext
20 lines
343 B
Plaintext
fastapi==0.115.6
|
|
uvicorn[standard]==0.34.0
|
|
redis==5.2.1
|
|
httpx==0.28.1
|
|
sqlalchemy==2.0.36
|
|
aiosqlite==0.20.0
|
|
psycopg2-binary==2.9.10
|
|
pydantic==2.10.4
|
|
python-multipart==0.0.20
|
|
websockets==14.1
|
|
pymysql==1.1.1
|
|
pymongo==4.10.1
|
|
cassandra-driver==3.29.2
|
|
neo4j==5.26.0
|
|
python-pptx==1.0.2
|
|
boto3==1.35.99
|
|
paramiko==3.5.0
|
|
aiokafka==0.12.0
|
|
pyarrow==18.1.0
|