Shubhankar Singhal

Designed and maintained scalable ETL pipelines using Azure Data Factory to ingest and transform large volumes of 2-wheeler and 4-wheeler data from international markets (US, Vietnam). Implemented data ingestion using Azure Blob Storage and Azure Data Lake Storage Gen2 as the central data lake for analytics workloads. Optimized existing data pipelines, achieving ~80% reduction in data processing time and improved data freshness. Performed data transformations and aggregations using SQL and PySpark (Databricks) for downstream analytics. Built SQL-based analytical datasets and views for reporting, reducing manual reporting effort by 60% and saving 200+ hours. Integrated curated datasets with Azure Synapse Analytics for high-performance querying and reporting. Integrated batch and near real-time data ingestion strategies to ensure high data availability. Conducted data profiling, anomaly detection, and data quality validation to improve the reliability of analytics data. Collaborated with analytics and product teams to design data models optimized for business intelligence use cases.

Designed and implemented Azure Data Factory (ADF) batch pipelines to ingest structured data from Azure Blob Storage into Azure Data Lake Storage Gen2 (ADLS). Configured Linked Services, Datasets, and Pipelines in ADF for scheduled batch data ingestion. Implemented basic data transformations using Mapping Data Flows and SQL-based transformations. Loaded curated data into Azure SQL Database / Synapse SQL (serverless) for reporting and analysis. Implemented pipeline scheduling, monitoring, and failure handling using ADF triggers and logs.