Supported an enterprise data warehouse migration for 17 lines of business, coordinating delivery from offshore.
Migration & scale
- Used BigQuery Migration Service to accelerate assessment and SQL conversion.
- Built and migrated 15K+ BigQuery warehouse objects and automated translation of 10K+ BTEQ/SQL scripts.
- Replicated 100+ TB from mirrored tables into core production tables.
- Designed resilient ingestion for 3,000+ CSV files concurrently.
Orchestration & operations
- Created Cloud Composer DAGs from a shared Python script and YAML configuration.
- Built Python validation routines comparing Teradata and BigQuery data.
- Used Terraform and GitHub Actions for Secret Manager and monitoring infrastructure.
- Integrated PagerDuty with Webex Spaces and built Looker dashboards for pipeline health and BigQuery costs.
Technology: BigQuery, BQMS, Cloud Composer, Airflow, Dataflow, Python, dbt, Dataform, Terraform, GitHub Actions, Looker.