mo
c3cacc141d
- deploy/airflow/mask_to_curated_dag.py: Trino-SQL masking (hash/redact/generalize) of PII from postgres/mysql sources into iceberg.curated_masked.* (verified: 14k+11k masked rows, all PII masked). - OM source->masked lineage edges created; curated tables cataloged. - Version OM compose + ingestion configs + om_api helper under deploy/openmetadata.
36 lines
762 B
YAML
36 lines
762 B
YAML
source:
|
|
type: postgres
|
|
serviceName: atc_postgres
|
|
serviceConnection:
|
|
config:
|
|
type: Postgres
|
|
username: mo
|
|
authType:
|
|
password: Dell2026!
|
|
hostPort: 10.0.21.51:5432
|
|
database: postgres
|
|
sourceConfig:
|
|
config:
|
|
type: AutoClassification
|
|
enableAutoClassification: true
|
|
storeSampleData: true
|
|
confidence: 80
|
|
schemaFilterPattern:
|
|
includes:
|
|
- public
|
|
tableFilterPattern:
|
|
includes:
|
|
- sales_orders
|
|
processor:
|
|
type: orm-profiler
|
|
config: {}
|
|
sink:
|
|
type: metadata-rest
|
|
config: {}
|
|
workflowConfig:
|
|
openMetadataServerConfig:
|
|
hostPort: http://openmetadata-server:8585/api
|
|
authProvider: openmetadata
|
|
securityConfig:
|
|
jwtToken: "__JWT__"
|