Prashant Verma

Data Warehouse Design and Architect for Brokerage and Mortgage modules. Design and implement end-to-end big data solutions on cloud platforms, collaborating with cross-functional teams to gather requirements and define architectural strategies. Data Migrations from legacy On-Premise systems to Cloud, Building Data Lake through consuming APIs data, websites Data, Mobile Apps Data, Event Notifications etc. Building Airflow Orchestrations jobs on EC2 using Pyspark for data pipelines in Full and incremental approach. Developing Python utilities to cater file conversions, Schema tracking utilities, Cascade drop and recreate hierarchical views automation and integrating it with GIT hub actions process through YML. Build various system Alert and error mechanism on Slacks, DataDog, Opsgenie using Python, AWS SNS, Webhooks APIs etc.

Gathering Business Requirements, Analysis and Providing Solutions related to DWH and ETL to various Middle East clients. Worked for PPG-AMESA Group which handles. Data migrations from multiple On Prem sources to Snowflke, creating Snow pipes, Procs, integration with AWS S3, Snow SQL etc. Developed and maintained data pipelines and ETL workflows using technologies like Apache Spark, Hive, and Pig for NUPCO. Data Extraction from Mongo DB (Json Format) and ingesting into relational Monet DB target using Pentaho and Python.

Create Data models for new business requirements. Architected a data lake solution on AWS, integrating data from various sources and enabling advanced analytics and machine learning capabilities. Handling huge amount of data in HDFS and querying it through HIVE. Optimize data workflows for performance and cost efficiency by implementing techniques like partitioning, caching, and compression. Migrated slow performing queries and views from NXG to snowflake for faster reporting results. Created DWH in AWS Redshift using AWS Glue, pySpark for one of customer in Travel Domain. Data Science Lead for CoE – Digital Travel Analytics: Video Analytics -Develop Machine Learning model, Train and Improve accuracy using YOLO framework. Flight Turn-Around Analytics –Involved in Development of Framework to automate Flight turnaround time using Video feeds. Used Python, YOLO, OpenCV, Boto3, Image labelling, AWS frameworks. Develop python code/utilities to perform various OS and AWS Cloud Activities.

Data integration between various application(s) and databases –PeopleSoft/Salesforce, Oracle using Informatica Intelligent Cloud Services (IICS). Assisted in the migration of on-premises data infrastructure to cloud platforms, optimizing performance and scalability. Development of new enhancements related to ETL, SQL, and Unix Scripts. Performance tuning in database, SQL queries. Engaging and coordinating business analysis sessions for understanding client business processes and mapping them to required project management standards. Development of new enhancements related to ETL and Unix Scripts.

Development of Incremental and Full load ETL Processes using Informatica. Performance tuning at Database Level (SQL queries optimization), ETL level and Scheduler Level. Create Sessions/ tasks and run Workflows and monitor status and session logs.

Development of ETL Mappings/Mapplets and Configure Sessions/Workflows. Creating and Running Simple, Complex SQL Queries to analyze the data correctness. Performance tuning at mapping, session and database level and troubleshooting. Domain Configuration and Repository backup, Code Migration Activities. Incremental and Full load batch Monitoring and resolving issues in Production and DEV environment. Deploying RPDs, Web catalog in OBIEE in higher environments and testing. Informatica Objects migration using scripts. Documentation- Technical Design Documents and test cases.