Sonali Dey is a Data Engineer with hands-on experience in Big Data frameworks including Hadoop, Spark, Hive, and Kafka. She possesses expertise in AWS services, enterprise databases like MySQL and Redshift, and dashboarding tools such as Power BI. She is proficient in Python and has experience building complex data pipelines.
Experience
Data Engineer
Ecom Express Pvt. Ltd.
Created a near real-time ingestion pipeline that imported data from RDBMS (MySQL BinLogs Based CDC) into Amazon S3 using Amazon MSK, Spark, Apache Kafka Connect, Debezium, AWS EMR, Apache Hudi, and AWS Glue. Created a robust big data pipeline that migrated RDBMS data to S3, and Amazon Redshift using EMR – Sqoop - Hudi - Glue which was used for analytics and reporting. Created interactive and highly informative Power BI reports using advanced DAX features. Created a fully automated pipeline using Python for sharing summarized views of data reports through the email used for daily organization performance.
Education
Masters
Specialized in Finance, Economics
Bachelors
Economics