Default profile banner
KT

Koushik Thota

@koushikthota

Data Engineer

Hyderabad, India

https://www.linkedin.com/in/koushikt

PriceWaterhouseCoopersV.R. Siddhartha Engineering College

Koushik Thota is a Data Engineer with focused experience in designing, developing, and optimizing scalable batch and streaming data pipelines. He is experienced in data warehousing and ETL frameworks, utilizing technologies across AWS and Azure. His expertise includes building robust analytics platforms and migrating complex data systems.

Experience

Data Engineer

PriceWaterhouseCoopers

Jun 2021 - PresentHyderabad, India

▪ Crafted an IOT based analytics platform in AWS right from Ingestion to visualization to get insights of key metrics of Industrial machines which helped the Site managers reduce their dependency on 30+ Floor workers. ▪ Engineered a metadata-driven streaming Data Framework in Azure which provided a reliable, consistent, and structured data processing capability for business logic execution along with an ability to enrich, transform, aggregate and augment data. ▪ Migrated monthly insurance reports from Excel and MS Access to AWS which reduced manual workload by 33%. ▪ Mentored 6 Freshers in helping them build a Processing & Analytics Platform for Web Server Log Data by leveraging open-source Big Data Frameworks. ▪ Translated 10 + business and functional requirements into robust, scalable solutions that work well within the overall data architecture. ▪ Evaluated new technology stacks via quick POCs which helped Architects make the best choices and reduced sprint spillovers by 30%.

Data Engineer

Capgemini

May 2018 - Jun 2021Hyderabad, India

▪ Designed, implemented, and Monitored data warehouse infrastructure to maintain insurance metrics in AWS using Glue & Redshift. ▪ Developed instances and queries to take data from various operational systems and create a unified data model for downstream analytics (AWS Athena) and reporting. ▪ Designed and developed a solution to load files which can't rely on a scheduled run and must be event driven, leveraging AWS S3, Glue, Lambda & DynamoDB. ▪ Carried out major refactoring efforts to improve the performance of some of the flows (Query & Pipeline Level) of the application from unusably slow to extremely responsive by 60%. ▪ Built generic & optimized ingestion pipeline for highly critical & confidential Insurance Data to enforce GDPR. ▪ Engineered data Workflows using Informatica Power Center & Built Process Flows to Handle & Process data from Heterogenous Sources (Oracle, PostgreSQL, Redshift, S3, Flat Files, XML).

Education

V.R. Siddhartha Engineering College

Bachelor of Technology

Electronics and Communications

Apr 2018Grade: CGPA: 8.81

Licenses & Certifications

Microsoft Certified Azure Data Engineer Associate

• No expiration

AWS Certified Solutions Architect Associate

• No expiration

Oracle Database SQL Certified Associate

• No expiration

Skills

Python
Shell Scripting
SQL
Spark
Kafka
Hive
Hadoop
AWS Glue
AWS Lambda
AWS S3
AWS Redshift
AWS Athena
IOT
AWS DynamoDB
Quick sight
AWS Kinesis
AWS Event Bridge
AWS RDS
Azure Databricks
Azure Data factory
Azure Synapse
ADLS
Blob
Azure Functions
Azure Event Hubs
Azure Stream Analytics
Cosmos DB
SQL Database
Jenkins
SVN
Git
Terraform
Docker
Kubernetes
Airflow
Informatica Power Center
Oracle
Postgres
MySQL
Snowflake
MongoDB
Redis
Elastic Search