Sankalp Kumar

Project: Business Systems Integration Platform. Client: 84.51° (Kroger). Developed Azure Databricks (PySpark, Spark SQL) pipelines integrating enterprise data from Salesforce, Dynamics 365, Workday, and Concur into centralized analytics datasets. Built scalable Delta Lake pipelines on ADLS for workforce, and financial data for downstream analytics and reporting. Implemented data validation, schema alignment, and transformation logic across multiple enterprise source systems. Collaborated with client analytics teams to deliver curated datasets supporting marketing analytics, campaign insights, and operational reporting. Managed Git-based development and Databricks Jobs orchestration for production ETL pipelines.

Developed scalable batch and streaming pipelines using PySpark, Spark SQL and Delta Live Tables (DLT), reducing manual processing time by 60%. Managed fine-grained data access and governance using Unity Catalog. Automated workflows with Databricks Jobs API and GitHub Actions, reducing scheduling overhead by 70%. Collaborated with analytics teams to deliver validated, production-grade data assets. Key Projects: Snapdeal – E-commerce Data Platform Migration (Vertica → Databricks). Migrated 35+ TB historical data, 150+ pipelines, 10K+ tables using metadata-driven PySpark frameworks. Improved query performance 3x and reduced infra cost by 40% via Delta optimizations. Implemented automated audit logging & validation for trust and traceability.

Assisted in ongoing Databricks migration and pipeline development projects under senior engineers. Supported teams with data validation, schema alignment, and ETL enhancements in PySpark and SQL. Built foundational skills in Spark optimization, Medallion architecture, and workflow orchestration.