Shivansh Choubey
@Shivansh
Data Engineer at COGNIZANT
Indore, Madhya Pradesh, India
Result-driven Data Engineer with hands-on expertise in Azure Data Factory, Databricks, and SSIS for building robust ETL pipelines in the insurance domain. Proficient in crafting scalable data lakehouse architectures using PySpark, Delta Lake, and Delta Live Tables. Adept at automating workflows, optimizing performance, and streamlining CI/CD deployments via Azure DevOps. Known for delivering production-ready solutions under tight deadlines and collaborating effectively with stakeholders. Certified in AZ-900 and DP-900 with a strong focus on cloud-native data engineering and operational excellence.
Experience
Data Engineer
COGNIZANT
Engineered and maintained 80+ ETL pipelines using MS Fabric, Azure Data Factory (ADF), Azure Databricks, and SSIS to extract, transform, and load critical financial data into SQL Server and Data Lake. Developed and optimized reusable and parameterized ADF pipelines and Databricks notebooks. Collaborated with UK stakeholders to gather ETL requirements. Implemented CI/CD deployments via Azure DevOps. Handled production support tickets and performed RCA for data pipeline failures.
Junior Developer
Goyal Softech Pvt
Optimized SQL queries and created 10+ stored procedures/views to automate reporting workflows for 5+ client dashboards, resulting in a 40% improvement in report generation speed. Supported daily data issue resolution.
Freelance
Aquatrax
Owned end-to-end data solution delivery, including data modelling, ingestion pipelines, transformations, and analytical layer design. Onboarded 15 clients. Designed conceptual, logical, and physical data models. Built a lakehouse architecture with Raw, Silver, and Gold layers.
ADF Trainer
ACTE
Trained 50+ students in Azure Data Factory (ADF) through structured workshops. Conducted both practical hands-on sessions and theoretical training on data integration, pipelines, and ETL concepts.
Education
LNCT, Bhopal
Bachelor of Technology
Electrical Engineering
Licenses & Certifications
AZ-900
DP-900