DLDivyanshu Lawaniya
@divyanshulawaniya
Senior Software Engineer at To The New
Noida, Uttar Pradesh
Divyanshu Lawaniya is an experienced Data Engineer with over 5 years of expertise in ETL, Data Lakes, and Data Pipelines. He is proficient in utilizing Cloud and Big Data tools for designing robust data pipelines and data lake architectures. His skills encompass various technologies necessary for generating meaningful business reports and analytics.
Experience
Senior Software Engineer
To The New
Building Analytics Jobs to achieve KPIs & Reporting etc. Fetching data from n no of sources using Apache NiFi, ES, Flume & Sqoop, and using Spark and Hive for processing that data to generate meaningful business reports. Responsibilities include creating Data Lakes (pulling data from sources like amazon S3, SAP, Oracle into HDFS), maintaining hive data models, migrating MapReduce jobs to Spark, migrating oozie workflows to Apache Airflow, and developing Python scripts for job scheduling.
Associate Hadoop Consultant
DexLab Analytics
Developed data pipelines and automated processes, including migrating oozie jobs into nifi and airflow, and converting java code into nifi flows. Responsibilities included data modeling, building structured data loading into HDFS using Sqoop, and performing data analysis using HQL. Worked on projects such as EDL, Insights Projects, and Customer Life Cycle Management.
Education
Government Engineering College Bharatpur
Bachelor of Engineering / Bachelor of Technology
Information Technology