Barath Ranganathan

Designed and orchestrated data ingestion pipelines using Azure Data Factory (ADF) integration run‑times (Azure, self‑ hosted), triggers, activities and pipelines to ingest data from disparate on‑premises data sources (flat‑files, database) to Azure Data Lake (ADL) and Azure Synapse Analytics. Debugged issues related to transformations and connectivity in the deployed ADF pipelines. Assisted Data Architect in designing and deploying the Azure stack module (Azure Storage, Azure Data Lake, Azure ADF Pipelines Azure Synapse Analytics) to import/ migrate the on‑premises data to Azure. Performed AD‑HOC technical analysis on the tables stored in Azure Synapse Analytics and reported anomalies to the upstream team. Developed and enhanced complex SQL queries to extract, join and aggregate data from Azure Synapse Analytics tables and provided the result as a View for the downstream Data Visualization team. Contributed as a team‑player in dveloping and presenting a Proof of Concept for migrating (lifting and shifting) the SSIS jobs to ADF using SSIS integration run‑time. Set‑up sandbox in Azure Portal for kick‑starting Proof Of Concepts and conducted training sessions for freshers on the overview of Azure Data services used in the project.

Developed and provided support for the deployed Hive (queries), Pig (ETL jobs) and SQOOP (incremental load import/export jobs). Implemented Hive optimised joins, partitions, buckets, incremental loads to the Hive external tables. Used Pig commands to perform transformations, event joins and pre‑defined aggregations before loading the data onto the HDFS for specific business requirements. Imported database tables (with latest snapshot date) from MySQL DB to Hadoop in incremental loads using SQOOP IMPORT jobs and provided the script files to the scheduling team for automating the weekly deployments. Managed jobs, work‑flows and files using HUE (HDFS file browser), Ambari Hive view (HiveQL queries) and Cloudera Manager (monitor, administer and report the status of Hadoop services). Monitored and reported the health of Hadoop daemons to the Hadoop administrator in case of any issues/ failures in the Hadoop clusters and tracked until closure (clusters up and running). Collaborated with the application development team in a Proof Of Concept to demonstrate the integration of Apache Kafka, Flume, HDFS and Spark Streaming to perform near real time processing and analytics.

Performed transformations using SAS program modules on the Mainframe DB2 tables to deliver the final tables in the required format for loading into Azure Data lake (or) Azure Synapse Analytics as per the functional specifications. Involved and contributed as a team player in developing Proof Of Concepts for migrating data from legacy systems (IBM z/OS) to Hadoop using talend open Studio.

Designed and developed incremental daily/monthly ETL jobs using SAS DI STUDIO and AD‑HOC non‑ETL jobs using SAS EG. Maintained SAS DI Studio ETL jobs to organise and handle data from different sources (flat files, database), performed the analytical processes (transformations) and delivered the data in the prescribed formats to the targets(flat files, database tables). Monitored and enhanced the time consuming ETL jobs to fix the performance bottlenecks, as part of process enhancement activities. Used SAS/BASE, SAS/PROCS, SAS/SQL, SAS/ODS, SAS/MACROS for accessing, pre‑processing, exploring, cleansing, analyzing, trans‑forming, summarizing, exporting and optimizing in different modules as part of the stand‑alone business requirements. Utilised SQL PASS THROUGH to communicate directly with Microsoft SQL databases to leverage and implement T‑SQL functions. Collaborated with SAS Lead and SAS Data Modeler in managing Metadata objects of the sources. Worked with Microsoft SSIS (ETL tool) in a cross‑functional team, to design the ETL workflows for specific requirements.

Created and modified views, functions and stored procedures in Microsoft SQL Server based on the functional requirements. Developed complex T‑SQL join queries and sub‑queries. Performed Technical Analysis and reported the source table anomalies (missing, null, special characters, inconsistent fields, duplicate records, data type mismatch) prior to the development phase. Executed unit tests and supported bug‑fixes. Monitored query metrics to identify potential performance issues and assisted in optimizing the database queries. participated in the code reviews and unit test case reviews to ensure that the developed code adheres to the requirements. Supported release deployment activities and maintained the technical documentation changes for separate modules.

Developed, modified and deployed COBOL, JCL and DB2 codes based on the functional requirements for each release. Created batch jobs using JCL and monitored the status of the job runs for any performance bottlenecks in each of the production runs. Maintained existing code and provided support for the deployed production code changes. Performed unit tests and provided QA support fixing the reported defects within the estimated turn around time.