feat(openmetadata): ingest Elasticsearch (search) + Airflow (pipelines)
- ES: connect over https with verify_certs=false (self-signed CA); 16 indexes incl source_data - Airflow: ingest 8 DAGs as pipelines from SQLite metadata DB snapshot - es_run.py: standalone runner that pins the working ES TLS config - Command Center API now wired with ELASTIC_PASSWORD
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
source:
|
||||
type: airflow
|
||||
serviceName: atc_airflow
|
||||
serviceConnection:
|
||||
config:
|
||||
type: Airflow
|
||||
hostPort: http://10.0.21.55:8080
|
||||
numberOfStatus: 10
|
||||
connection:
|
||||
type: SQLite
|
||||
scheme: sqlite+pysqlite
|
||||
databaseMode: /tmp/airflow.db
|
||||
sourceConfig:
|
||||
config:
|
||||
type: PipelineMetadata
|
||||
includeOwners: true
|
||||
includeLineage: true
|
||||
sink:
|
||||
type: metadata-rest
|
||||
config: {}
|
||||
workflowConfig:
|
||||
openMetadataServerConfig:
|
||||
hostPort: http://openmetadata-server:8585/api
|
||||
authProvider: openmetadata
|
||||
securityConfig:
|
||||
jwtToken: "__JWT__"
|
||||
Reference in New Issue
Block a user