Surendra Kumar
@user.2570551
Senior Data Engineer | Azure, Databricks, Kafka, Delta Lake & Spark Streaming
Greater Noida, India
Surendra Kumar is a Senior Data Engineer with expertise in building and deploying scalable data pipelines using Azure, Databricks, Apache Kafka, Spark Streaming, and Delta Lake medallion architectures. He has reduced pipeline runtime by 90% and costs by 97% through optimised partitioning and parallel processing, and engineered real-time reporting systems replacing legacy SQL Server solutions. A Databricks Certified Professional with progressive experience at CNS and Tata Advanced Systems, Surendra brings deep skills in Python, PySpark, ADF, and orchestration tools.
Experience
Senior Data Engineer
Aggregato
Designed dynamic ingestion pipelines processing 20+ data topics from 40+ sites using Kafka and Azure Event Hub into Delta Tables. Built Delta streaming pipelines for 100+ KPIs. Reduced runtime by 90% and costs by 97% using partitioning and parallel processing.
Data Engineer
Tata Advanced Systems
Ingested structured and semi-structured data from Elastic Search and Kafka into ADLS Gen2 using Spark Streaming. Developed ETL pipelines for Power BI dashboards and APIs.
System Engineer
CNS
Industrialised ML pipelines processing image data with Hive, Spotfire, and Airflow DAGs. Automated pipeline executions and daily thumbnail refresh using S3 and Airflow.
Education
Babu Banarasi Das University
Engineering
Licenses & Certifications
Databricks Certified Professional Data Engineer
Databricks