Sathwika Kaparthi

Designed and implemented an end-to-end data pipeline using Azure Data Factory for orchestration, Databricks (PySpark) for processing and aggregation of NACH and UPI transaction-level data, and MinIO (S3-compatible) for data lake storage to perform real-time insights. Designed and implemented real-time pipelines for NACH and its sub-products using Kafka, integrated with Apache Flink and Spark Streaming. Built and maintained scalable batch data pipelines using DBT/Python Client, Dagster and SQL, processing large-scale financial transactions over 200M records. Automated data workflows and validations with Python and Pyspark. Implemented Dagster alerting policies and webserver API for automated email reports. Implemented reverse geocode mapping using Python. Optimized SQL queries for performance across distributed systems. Migrated data from Hadoop (HDFS) to Minio (S3-compatible) storage. Containerized data applications using Docker and deployed them on Kubernetes cluster.

Built interactive dashboards using Tableau and Superset to support business intelligence and regulatory reporting (RBI, Ministry of Finance). Led claim and billing data analysis for schemes like NSAP, NREGA, AMC using SQL-driven insights to identify trends and improve cost tracking and automated reconciliation process. Collaborated with business and product teams to implement Python-based data solutions. Used Git for version control and followed CI/CD practices in a Linux-based development environment.