Ankit Singh

Designed an end-to-end Streaming data pipeline in Azure Databricks reducing workflow runtime and costs by 12% using Spark and Delta optimizations, along with conditional tasks to avoid executing unnecessary workflows based on incoming data. Automated API-based data enrichment using PySpark, SQL and REST APIs to prevent the loss of critical records during data ingestion.

Led and mentored an offshore team, managing work allocation, progress tracking, defect resolution, while ensuring high-quality deliverables and timely project completion. Served as Subject Matter Expert for the end-to-end automation process, providing guidance and resolving queries for onshore and offshore teams, including developers, QA and team leads, to ensure seamless execution and cross-functional alignment. Designed data validation and transformation workflows using SQL and Python to ensure data quality and reduce ingestion errors by 90%. Developed a SQL-based data validation framework to verify data loads across daily and monthly pipelines, reducing data integrity issues by 70%.

Configured End-to-End SDLC setup: migrated and tested in different environments, resulting in a significant reduction in PROD defects. Identified and worked on critical defects with the development teams, contributing to 90% decrease in bugs.