Kishlay Dwivedi

Built an automated ADF pipeline that ingests weekly load from Walmart into the Data Lake. Written PySpark and SQL Code in Databricks for large-scale data transformation, improving the quality of over 18 TB of raw data. Created Power BI reports for near real-time data analysis, eliminating the time needed for manual data analysis by up to 90%. Wrote an Azure PowerShell script to automate daily project tasks, eliminating around 10 hours per week of manual work. Used ARM Templates to deploy Azure Data Factory components. Decreased the risk of data discrepancies by 90% by building an automated Azure Data Factory pipeline which creates a data profiling report on the source data. Wrote SQL queries using joins, grouping, aggregation, and nested subqueries, simplifying complex joins to increase overall query performance by 40%.

Built an end-to-end business intelligence solution from the ground up, using raw data containing information about sales, products, customers and locations. Prepared the Data, Model, visualize and Analyze the Data and provide insights about data.