Abhishek Batade
@abhishekbatade
Data Engineer at E-Mech Solutions Pvt Ltd
Pune, India
Abhishek is a skilled Data Engineer proficient in overseeing end-to-end data pipeline delivery. He specializes in big data processing using Python, PySpark, and the Spark API, coupled with expertise in AWS services (S3, Glue, EMR) and data warehousing solutions like Snowflake and Redshift. He is adept at optimizing data workflows, ensuring data integrity, and utilizing tools like Airflow and Git for robust data solutions.
Experience
Data Engineer
E-Mech Solutions Pvt Ltd
Developed optimized data pipelines using AWS services (S3, Glue, Athena, PySpark) and Snowflake. Key responsibilities included migrating data from SQL Server and AWS RDS to Snowflake, developing PySpark scripts for data extraction and transformation, and building batch processing solutions. Expertise includes applying data cleaning techniques, handling null values, and optimizing storage using Parquet formats. Managed workflows using Airflow and utilized Spark SQL for high-performance data processing. Also analyzed large volumes of financial data, migrating data from RDBMS to HDFS/Hive, and collaborating with BAs to define requirements.
Education
Dr. Babasaheb Ambedkar Technological University (DBATU)
Bachelor of Technology (BTech)
Computer Science & Engineering