He possesses a strong foundation in Spark, SQL, Azure, AWS, Python, and Databricks, with a proven track record of success across retail, aviation, hospitality, and life sciences. He excels in developing and implementing robust data solutions and designing the efficient data pipelines that transform raw data into actionable insights. His expertise in optimizing data workflows enables him to drive significant business value across a variety of industries.
Experience
Data Engineer
Nagarro
Architected and implemented scalable data pipelines on Azure for a major airport authority, utilizing medallion architecture. Achieved 20% improvement in data ingestion rates through optimized data flow orchestration. Built and deployed real-time solutions utilizing Azure Event Hubs to handle millions of events per second. Streamlined Spark code within Databricks, achieving a 45% reduction in processing time. Automated notifications with Logic Apps, reducing manual effort by 70%.
Data Engineering Consultant
NTT Data
Executed data transformations within Databricks using Spark SQL and UDFs to cleanse, enrich, and format data. Placed transformed data into Delta tables stored in S3 buckets with appropriate partitioning. Improved query performance by 30% by implementing broadcast joins, caching, and optimize command. Enhanced job stability and reduced overall job execution time by 15% through skew-aware algorithms.
Data Engineer
Tata Consultancy Services
Developed efficient Python scripts for managing S3 data, including file copying, archiving, and deletion. Implemented bucket policies and leveraged cost-effective storage classes to reduce storage costs by 15%. Improved operational efficiency by implementing email notifications for job completion and failures using boto3 and SES. Leveraged Tableau to create visualizations and dashboards featuring various HCP KPIs.
Education
Uttar Pradesh Technical University
B.Tech.
Electronics and Communication Engineering