Bhalchandra Kolhe

Designed and implemented diverse ETL pipelines with complex business transformations, employing cloud Dataflow to extract data from various source systems like cloud storage, cloud sql and Pub/Sub, and loading it into destinations such as BigQuery, Pub/Sub, and gcs files. Built Streaming data pipelines using Cloud Pub/Sub and dataflow to load data into Bigquery. Contributed to the migration of MySQL data to SQL Server from the ETL perspective. Additionally, explored Cloud IAM and mastered cross-project communication between GCP projects. Designed and implemented a streamlined process to transfer finance data to Google BigQuery, utilizing AppEngine Service, Google Cloud Storage, Google Dataflow, and Composer. Architected and implemented data pipelines leveraging google cloud dataflow, data fusion, BigQuery, Airflow, Dataproc, GCS, and cloud SQL. Managed a development team and completed the migration of multiple local markets from Onprem to Google Cloud Platform within the established timeframe. Developed Airflow DAGs and custom operators to enable seamless orchestration and enhance team development efficiency. Devised and executed a versatile PySpark-based ingestion framework, managing deduplication, data shifting, and failure handling, ensuring seamless and consistent data loading into BigQuery. Designed and implemented an automated data ingestion framework using GCS, Pub/Sub, CloudFunctions, and Airflow to streamline data pipelines and improve data processing efficiency. Stored data in an optimized and encrypted form using BigQuery denormalization. Collaborated closely with product owners to gather requirements, perform thorough analysis, and plan effective, budget-conscious solutions. Successfully deployed an alerting mechanism that triggers timely notifications to stakeholders on data load issues based on predefined conditions. Efficiently executed varioud POCs to evaluate gcp components for various migration projects, aiding in the selection of optimal solutions