Shivani Sethi
@shivanisethi
Data Engineer at Testbook
Ghaziabad, India
Shivani Sethi is a Data Engineer with experience at Testbook, specializing in building robust ETL pipelines using Apache Airflow and Python. She has hands-on experience with Google Cloud Platform (GCP), including using Pub/Sub and Cloud Run for data processing and creating data lakes. Her technical skills encompass Data Science, Machine Learning, and various databases like PostgreSQL and MongoDB.
Experience
Data Engineer
Testbook
Used google pub/sub workers for creating banner links, pdf files in bulk. Used cloud run for the same before scaling. Created Data Lake using gold and silver storage on GCP bucket, storing data in parquet files. Scheduled Youtube livestream events using diFerent Youtube APIS on Class Booking System (CBS). Created Bookings for daily events on CBS and linked each Youtube channel with CBS. Automated RTMP Keys for each Youtube Event using Youtube APIS. Generated daily OTPs for android and ios teams. Created algorithms for computing strength of users in diFerent chapter questions and providing detailed analysis reports. Completed Airbyte POC to create Data Lake. Created ETL pipeline for analytical purpose and communication with users through mails, IVR calls, WHATSAPP, SMS using Apache-Airflow framework in python. Automated processes using Apache-Airflow, cloud functions, and Appscript.
Programmer Analyst Trainee
Cognizant
Initially interned in 'Automation Testing using Selenium'. Later worked on the Backend as a full-time resource. Profile was promoted from Manual Tester to Functional Automation Tester.
Education
Jaypee Institute Of Information Technology
Bachelors Of Technology with Honours
Electronics and Communication Engineeering
Doon Public School
12th CBSE Board
Licenses & Certifications
Machine Learning Specialization
Coding Blocks
Master Data Structures and Algorithms using C++
Coding Blocks
Representative and Volunteer in Converge