Ashish Biradar

Architected and implemented an end-to-end data engineering pipeline for a top medical device manufacturing company, handling 5 TB of claims data to enhance post-market surveillance analysis and customer segmentation. Ingested and stored multi-source data into Azure Data Lake Storage (ADLS) Gen2, streamlining access to over 5 TB of insurance data. Processed and transformed raw data with Azure Databricks. Conducted advanced analytics using Azure Databricks, identifying 5 distinct customer segments for targeted marketing. Automated the data pipeline with Azure Data Factory (ADF), increasing operational efficiency. The project led to increase in targeted marketing effectiveness and reduced customer churn.

Experience in creating ETL pipelines using Azure data factory and Azure synapse, end-to-end project implementation using ADF/Azure synapse. Created Azure data factory pipelines to pull data from on-premises MS-SQL database, and Azure SQL DB to Azure Data Lake storage as a full load daily into corresponding folders. Developed a data quality framework utilizing tools such as Databricks (with Pyspark) and Azure Data Factory. This framework efficiently extracts data from multiple sources (SQL, ADLS, Rest API, etc.), executing check rules. Having good knowledge in Dataset Linked Service, Runtime Integration, Movement / Control Flow activity in ADF. Having experience in transforming and loading data into delta tables in Azure Databricks.