Siddharth Gupta
@siddharth.gupta
Data Engineer / Software Developer
Kanpur, India
Siddharth Gupta is a Data Engineer and Software Developer skilled in Azure Data Factory, Azure Databricks, and Python. He has experience working with big data streams, migrating notebooks, and ensuring compliance frameworks like GDPR. His background includes developing end-to-end pipelines and working with tools like Spark and Synapse Analytics.
Experience
Digital Specialist Trainee
Infosys
Got trained in big data stream and received training in Spark , HDFS , Hive, SQL Server etc. Built end to end pipeline on Azure Data Factory and DataBricks to ingest and analyze AdventureWorks Dataset. Understood and explored several other tools like Airflow, Kafka etc.
Data Engineer
Infosys
Client - Microsoft. Post Training started working for Microsoft Data engineering team which handles data for Customer and Partners Surveys. Migrated 100+ notebooks from Azure Databricks to Synapse also fixed the errors resulting due to spark version mismatch and other differences in platforms. Found a Key-vault related vulnerability which could have exposed sensitive information to users. Worked on Retention Framework as part of the GDPR compliance.
Education
SRM Institute Of Science and Technology
B.Tech
Computer Science
Database Management Systems, Data Mining, Data Structures and Algorithms, Object Oriented Designs and analysis
Licenses & Certifications
AZ-900 - Azure Fundamentals
DP-900 - Azure Data Fundamentals
Infosys Certified Spark Professional
Infosys