Commit Graph

2 Commits

Author SHA1 Message Date
mo c3cacc141d feat(masking): mask_to_curated DAG + OpenMetadata lineage + versioned OM artifacts
- deploy/airflow/mask_to_curated_dag.py: Trino-SQL masking (hash/redact/generalize)
  of PII from postgres/mysql sources into iceberg.curated_masked.* (verified:
  14k+11k masked rows, all PII masked).
- OM source->masked lineage edges created; curated tables cataloged.
- Version OM compose + ingestion configs + om_api helper under deploy/openmetadata.
2026-06-27 03:13:16 +02:00
mo 944635ffe1 feat(masking): add PII columns to generators + backfill (Faker-style, no deps)
Source generators (postgres/mysql/mongodb) now emit realistic PII
(name/email/phone/ip/iban/address/national_id/dob). deploy/mask_pii_setup.py
ALTERs the source tables and backfills a bounded sample so Debezium CDC and
OpenMetadata PII auto-classification see real sensitive values.
2026-06-27 02:50:28 +02:00