Harmanjit Singh
@harmanjitsingh
Lead Data Engineer at Hashmap, an NTT Data Company
Pune, India
Harmanjit is an experienced Data Engineer specializing in large-scale data migration and pipeline development. He has led teams in migrating over 100 terabytes of data from Teradata to Snowflake. His expertise includes building Python automation scripts, integrating ELK stack with Kafka, and developing data ingestion pipelines using Matillion ETL and AWS S3.
Experience
Lead Data Engineer
Hashmap, an NTT Data Company
Led offshore data migration team to deliver the migration of 100+ terabytes of data from Teradata to Snowflake production. Built Python-based automation scripts to enhance migration and validation processes. Integrated ELK stack with Kafka using Snowflake as data warehouse. Developed data ingestion pipeline using Matillion ETL with AWS s3 as source and Snowflake as environment. Mentored and led 10 member offshore team, assisting with technical and analytical development. Managed communication with third-party tool vendors to ensure product compatibility.
Developer
Tata Consultancy Services Ltd.
Developed adaptors in the middleware tier for dataflow across multiple platforms. Created business-oriented services and configuration management using spring MVC. Worked on Webservices and microservices exposed by multiple systems. Delivered and deployed web applications through Jenkins and Docker.
Education
Guru Nanak Dev University
Bachelor of Computer Science and Engineering
Computer Science and Engineering
Licenses & Certifications
SNOW PRO CORE
Azure Fundamentals
Matillion: Build a Datawarehouse