Dedicated software Engineer with experience in achieving tangible results and cross team collaboration. Proactive and excited to partner with like-minded individuals to achieve goals. Innovative individual with proven success evaluating requirements for software development projects to design innovative solutions. Out-of-the-box thinker and problem solver dedicated to improving performance. Work well in team and consistently delivers to deadlines.
Experience
Big Data Engineer - 2
Systango Technologies
Worked on Data generated by Fantasy Game application to clean, store and analyse the data after matches have been played. Worked with Snowflake cloud tool to write the ETL for continuous data integration. Created Data pipeline using pyspark and databricks. Created data pipeline in on local server to clean, transform and update the data into delta tables. Created the data pipeline to get the data from source postgres sever, transforming, merging and updating it into destination tables using JDBC with pyspark. Created internal and external storage, notification and various types of integration objects within snowflake tool. Used AWS s3 and azure blob storage to integrate with external storage in snowflake. Used AWS RDS to host postgres database for fantasy game data.
Software Engineer
Fourtyseven Billion Information Systems pvt ltd.
To clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts. Analyzed the SQL scripts and designed the solution to develop a custom ETL pipeline by using Python/pyspark. Profile structured, unstructured and semi structured data across various sources to identify patterns in data and implement data quality metrics using necessary queries or python scripts based on source. Wrote Scripts in Core python and Advance Python for a component in micro-services architecture.
Education
Devi Ahilya Vishwa Vidyalaya
Bachelor of Computer Applications
Computer Applications