Harshita Gupta
@harshitagupta
Data Engineer III at Insight
Jaipur, Rajasthan, India
Harshita Gupta is a Data Engineer with over three years of experience building scalable, cloud-native data pipelines across AWS and Azure. She has extensive hands-on experience with Databricks, Spark, Python, and SQL. Currently a Data Engineer III at Insight, she specializes in architectural redesign, performance optimization, and data governance. She is also pursuing an M.Tech. in Data Engineering at the Indian Institute of Technology Jodhpur.
Experience
Data Engineer III
Insight
Led architectural redesign of batch and near-real-time data pipelines using Databricks, Apache Spark, and Azure Data Factory, reducing end-to-end latency by 25% for analytics supporting 500k+ users. Designed optimized Spark transformations and Delta Lake tables to enable scalable consumption for BI and downstream applications. Implemented data quality checks, validation rules, and pipeline monitoring to improve reliability of production datasets. Enforced secure, compliant data access using Unity Catalog, supporting GDPR-aligned governance across 15+ datasets. Collaborated with analytics, product, and platform teams to align data models with evolving business and operational requirements.
Data Engineer II
Insight
Designed and maintained scalable ETL/ELT pipelines using Azure Data Factory, Databricks, Apache Spark, Python, and SQL. Developed Python-based data quality and reconciliation checks prior to merges into curated (Gold) datasets. Implemented CI/CD pipelines and Infrastructure as Code using Terraform and Azure DevOps for data workflows. Supported production operations including pipeline failure analysis, reruns, and SLA adherence.
Data Engineer
Insight
Migrated legacy on-premise data to cloud platforms using ADF, Databricks, Spark. Created analytics-ready datasets for BI and ML teams, improving performance and usability. Built reusable Python components and supported production pipeline debugging.
Education
Indian Institute of Technology Jodhpur
M.Tech.
Data Engineering
Coursework: Big Data Systems, Machine Learning, Deep Learning, Network Science
Swami Keshvanand Institute of Technology, Management and Gramothan
B.Tech.
Computer Science
Coursework: Data Structures & Algorithms, Database Management Systems, Cloud Computing, Big Data Technologies
Licenses & Certifications
Microsoft Certified: Fabric Data Engineer Associate (DP-700)
Microsoft
Databricks Certified Data Engineer Associate
Databricks
AWS Cloud Practitioner
AWS