Default profile banner
AT

Ankur Tiwari

@user.2462838

Lead Data Migration Engineer at Q2 Software

Gwalior, India

https://github.com/ankurtiwari74

Q2 SoftwareACTS C-DAC, Pune

Experienced Data Engineer with over 9+ years of expertise in designing and implementing robust data solutions. Proficient in Python, Spark, Kafka, SQL, Flink, and cloud platforms like Azure, AWS, and GCP. Skilled in optimizing data pipelines, managing large-scale data migrations, and leveraging cloud technologies to deliver scalable and efficient solutions.

Experience

Lead Data Migration Engineer

Q2 Software

Apr 2022 - Present

Led end-to-end data migration efforts for financial institutions transitioning to Q2’s online banking platform. Directed migration of financial data from competitor platforms using flat files (CSV, TXT, Excel) and transformed data into XML for SQL Server integration. Optimized data migration processes to minimize downtime during go-live, ensuring seamless transitions for end users. Developed and customized base scripts for vendor-specific projects, ensuring compliance with Q2 standards. Conducted comprehensive migration testing, including mock imports and production deployments, ensuring data accuracy and functionality. Improved process efficiency by standardizing reusable components, reducing project timelines by 20%.

Lead Software Engineer

ATYETI Inc

May 2021 - Apr 2022

Worked on the Content Data Lake project, a metadata production platform for collection automation. Developed modules to pre-process data from DAM and upload it to Data Lake using Python, AWS ECS, and Lambda. Configured deployment pipelines using Terraform and resolved customer-reported issues.

Systems Analyst

ValueLabs LLP

Sep 2020 - Apr 2021

Developed Python scripts for BI reporting for WebMD and its subsidiary Medscape. Implemented job scheduling and SQL table mapping for data processing. Utilized Databricks Spark, Vertica, and Azure Data Factory for data transformation and reporting.

Senior Consultant

KPI Partners India Pvt Ltd

Feb 2019 - Sep 2020

Designed and implemented ETL pipelines using Databricks Spark and AWS S3 for data replication. Developed Tableau workbooks for reporting using data populated in Snowflake. Created a custom logging solution to track Spark application jobs and ETL orchestration. Migrated data from Oracle/SQL Server to AWS S3 and Snowflake, including security roles and privileges.

Software Developer

IDC Technologies

Jun 2018 - Feb 2019

Worked on Ericsson Cloud project, automating service migration from physical servers to cloud using YAML. Implemented ANF test cases for cloud migration.

Software Engineer

ATMECS Inc

Oct 2015 - May 2018

Developed a self-healing tool (SEAL) integrated with Nagios for server health monitoring and issue resolution. Created APIs for cloud resource monitoring and cost optimization using AWS and OpenStack. Worked on Eureka, a data migration project for Citrix, involving HDFS, MapReduce, Pig, and Redshift.

Education

ACTS C-DAC, Pune

Post Graduate Diploma in Advanced Computing (PG-DAC)

Feb 2015 - Aug 2015

RustamJi Institute of Technology, Tekanpur

Bachelor of Engineering (BE)

Information Technology

Sep 2010 - Jun 2014

Licenses & Certifications

AWS Certified Developer – Associate

AWS

• No expiration

Introduction to Programming using Python

Microsoft

• No expiration

Skills

Python
NumPy
Pandas
Matplotlib
PySpark
Apache Spark
Databricks
Kafka
HDFS
MapReduce
Pig
Oozie
Sqoop
Azure
Delta Lake
Data Factory
Synapse
AWS
EC2
EMR
Glue
Redshift
Lambda
S3
GCP
BigQuery
SQL
Oracle
SQL Server
MongoDB
Snowflake
Vertica
PyCharm
Jupyter Notebook
Git
Airflow
Terraform
Ubuntu
CentOS
OpenSUSE
Windows
MacOS