Shubham Madure
@shubhammadure
Microsoft Azure certified data engineer
Pune
Shubham is a Microsoft Azure certified data engineer specializing in PySpark and Scala-Spark development on cloud platforms. His expertise includes Azure Data Factory, Azure Databricks, and data lake management. He has experience developing end-to-end big data solutions and optimizing Spark code for high performance.
Experience
Azure Data Engineer
Letrim intelligence services
Developed end-to-end big data solutions using PySpark & Scala-Spark on cloud platforms. Performed complex ETL operations (data mapping, aggregations, joins) using Python and Spark. Optimized Spark code for performance and throughput. Managed data loading into datalake (ADLS gen 2) from various APIs and developed complex ETL pipelines using Azure Data Factory integrated with Databricks. Used CI/CD pipelines and scheduled data pipelines using Airflow scripts.
Datawarehouse/Business Intelligence Developer
Amdocs
Developed datawarehousing and ETL solutions using Informatica and Teradata. Wrote supporting shell scripts on Linux platform. Performed performance tuning at query level using indexes and partitioning in Teradata, Oracle, and Informatica. Analyzed production data issues by backtracking end-to-end data flow and transferring extracts via SFTP.
Education
The University of Texas at Austin
Post Graduate Program in Artificial Intelligence and Machine Learning
Artificial Intelligence and Machine Learning
University of Pune, India
Bachelor of Engineering
Electronics and Telecommunication Engineering
Licenses & Certifications
Azure Databricks Professional Training
Azure Databricks
Diploma in Advanced computing
Centre for Development of Advanced Computing (C-DAC)