Bala Sourvendra

With data engineering support our client reduced time upto 87% in approvals, business flows Developed scalable ETL/ELT pipelines using ADF and Databricks batch size ranging from 5 - 21 GB/Day Optimized Azure Synapse DW performance, improved query performance, applied security best practices, business logics to secure privacy of 100 million users Solved real world issues such as data skew with broadcast, salting in pyspark and temp tables, materialized views in Synapse resulted in reducing pipeline runtime by 10-17% Monitor, debug production pipelines; created automated reports of pipeline activities, data quality to alert stakeholders reduces the average time taken by developers to fix the pipeline by 11 minutes