AIK ABDUL SATTAR JAVEED CHISTY

Responsible for creating RDD’s and DFs for required input data and performing data transformations using spark core. Developed Data Bricks pyspark scripts and implemented data analytics use cases to handle large volume of data using spark SQL, Data frames, and Delta format. Performed operations on source data using spark and wrote Pyspark code to transform and load data from staging to curated layer.

Worked as Azure Data Engineer. Key responsibilities included developing data pipelines using Data Factory to ingest data from source systems to Azure Data Lake, creating notifications and alerts in Azure, and developing Spark-SQL scripts in Databricks. Developed Apache Spark Applications using Python and implemented data processing projects to handle data from various Data Lakes.

Involved in writing Hive queries to load and process data as per business requirements in Hadoop File System. Responsibilities included modifying the Data Model and creating SQL queries and performance tuning.