Abhishek Pandey
@abhishekpandey3703
Consultant at Deloitte
Hyderabad, Telangana
Abhishek Pandey is an experienced Data Engineer skilled in developing scalable ETL pipelines using technologies like Spark, Scala, and Databricks. He has extensive experience in cloud platforms, including Azure and GCP, handling large-scale data migrations (up to 5.6Tb) and ensuring data flow continuity. He has a proven track record of performance optimization, reducing execution times significantly, and leading data engineering teams.
Experience
Consultant
DELOITTE
Leading team of 5 data engineers. Responsible for fine tuning data between two systems and reached up to 95% plus accuracy. Developed ETL pipeline to decrease execution time from 17 hrs. to 3 hrs. Automated ETL pipeline using oozie workflow. Worked on data migration from on premise to Azure, including version changes of spark and Scala. Performed performance optimization on ETL pipelines, reducing execution time from 20 hrs to 6 hrs.
SYSTEM ENGINEER
TATA CONSULTANCY SERVICES
Developed ETL pipelines and monitored data flow. Gained hands-on experience with cloud platforms like Azure and GCP, managing data flow continuity and leading a team of 5 data engineers.
Education
IPS-CTM
Bachelor's of Engineering
Computer Science and Engineering