Nishith Barodia

• Experience in building, designing, monitoring, fixing data pipelines using Big data components like Hadoop, Hive, Spark, Pyspark, Sqoop, SQL, Unix, NIFI. • Collaborated with cross-functional teams to identify business requirements and deliver data-driven solutions using Spark. • Optimized Spark jobs for performance by tuning various parameters such as memory usage and parallelism. • Developed a Python-based auditing system for user actions in Cloudera Data Platform's command-line interface (CLI). • Designed Apache Spark workflows for GDPR-compliant data processing, including data masking and hiding. • Created an efficient data flow pipeline in Apache Nifi, using HDFS and Hive for inbound and outbound data deliveries. • Worked on migrating data delivery pipeline infrastructure from On-Premise to Cloud. • Developed a statistical Reconciliation Framework to monitor data deliveries to all the downstream systems. • Owner of 4 Agile releases in development, supported in Test and deployed to production. • Built a MySQL data pipeline for replicating production environment to DR environment in a data warehousing project. • Cross team collaboration with business teams assisting in requirement gathering for downstream.