Avnish yadav
@avnishyadav
Big Data Engineer at Volkswagen Group Technology Solutions India
Ganesh Nagar Bopkhel, Pune, India
Experienced Data Engineering professional skilled in SQL, Python, Spark, ETL, and ELT. He specializes in fetching data from various source systems to provide business insights. His expertise includes data mining, transformation, and building robust data pipelines.
Experience
Big Data Engineer
Volkswagen Group Technology Solutions India
Imported customer data, product data, and promotion data in relational databases to HDFS raw by using Sqoop. Created a data warehouse in HDFS by defining a schema and creating HIVE tables in HDFS using HIVE queries. Analyzed, cleaned, and transformed data related to finance and sales order using Pyspark and SQL. Developed a data pipeline for combining all business solutions into a single streamline data pipeline & incorporating proper partitions on dataset for optimization process by using python and Hive query(HQL). Developed and implemented ETL data pipelines using AWS services such as S3, Glue. Projects involved the use of AWS Athena and Quicksight to transform and visualize the data.
Customer 360 Project
N/A
Implemented ETL processes for the complete 360 view of the customers dashboard where sales agent can see all the orders purchased by the customer.
Virtual Assistant Alexa Project
N/A
Created a virtual assistant using Python. It is capable of voice interaction, music playback, streaming podcast, playing audiobooks and providing weather, traffic, Sports and other real time information.
Education
Sinhgad college of Engineering
BE
Engineering
KV BEG
HSC
KV NO-1
SSC
Licenses & Certifications
AWS Machine Learning Foundation
Udacity
HTML certificate
Coursera