Koushik Thota is a Data Engineer with focused experience in designing, developing, and optimizing scalable batch and streaming data pipelines. He is experienced in data warehousing and ETL frameworks, utilizing technologies across AWS and Azure. His expertise includes building robust analytics platforms and migrating complex data systems.
Experience
Data Engineer
PriceWaterhouseCoopers
▪ Crafted an IOT based analytics platform in AWS right from Ingestion to visualization to get insights of key metrics of Industrial machines which helped the Site managers reduce their dependency on 30+ Floor workers. ▪ Engineered a metadata-driven streaming Data Framework in Azure which provided a reliable, consistent, and structured data processing capability for business logic execution along with an ability to enrich, transform, aggregate and augment data. ▪ Migrated monthly insurance reports from Excel and MS Access to AWS which reduced manual workload by 33%. ▪ Mentored 6 Freshers in helping them build a Processing & Analytics Platform for Web Server Log Data by leveraging open-source Big Data Frameworks. ▪ Translated 10 + business and functional requirements into robust, scalable solutions that work well within the overall data architecture. ▪ Evaluated new technology stacks via quick POCs which helped Architects make the best choices and reduced sprint spillovers by 30%.
Data Engineer
Capgemini
▪ Designed, implemented, and Monitored data warehouse infrastructure to maintain insurance metrics in AWS using Glue & Redshift. ▪ Developed instances and queries to take data from various operational systems and create a unified data model for downstream analytics (AWS Athena) and reporting. ▪ Designed and developed a solution to load files which can't rely on a scheduled run and must be event driven, leveraging AWS S3, Glue, Lambda & DynamoDB. ▪ Carried out major refactoring efforts to improve the performance of some of the flows (Query & Pipeline Level) of the application from unusably slow to extremely responsive by 60%. ▪ Built generic & optimized ingestion pipeline for highly critical & confidential Insurance Data to enforce GDPR. ▪ Engineered data Workflows using Informatica Power Center & Built Process Flows to Handle & Process data from Heterogenous Sources (Oracle, PostgreSQL, Redshift, S3, Flat Files, XML).
Education
V.R. Siddhartha Engineering College
Bachelor of Technology
Electronics and Communications
Licenses & Certifications
Microsoft Certified Azure Data Engineer Associate
AWS Certified Solutions Architect Associate
Oracle Database SQL Certified Associate