Timings: Mon - Fri; 10:30 AM to 7:30 PM
About the Role

We are seeking a skilled Data Engineer with hands-on experience in building and maintaining scalable data pipelines and analytics solutions. The ideal candidate will be highly proficient in PySpark, Databricks, and Azure, with strong experience in managing large datasets, data warehousing, and modern data stack technologies.

Key Responsibilities

Design, build, and maintain ETL/ELT pipelines for efficient data ingestion, transformation, and storage.
Collaborate with data analysts, data scientists, and stakeholders to understand data requirements and deliver actionable solutions.
Implement and maintain data models and transformation workflows using Databricks.
Optimize performance of large-scale distributed data processing jobs using PySpark.
Ensure data quality, consistency, and integrity through validation and monitoring.
Work with Azure cloud data platforms (e.g., Azure Data Lake, Azure Data Factory, Azure Databricks, Azure Synapse) to manage data storage and pipelines.
Troubleshoot and resolve data-related technical issues.
Participate in code reviews, documentation, and best practice enforcement for data engineering workflows.

Required Skills and Qualifications

Minimum 2 years of experience in Data Engineering or a related role.
Experience writing code in Python language.
Proficiency in PySpark for distributed data processing.
Expertise in Databricks for data engineering, transformation, and workflow development.
Strong knowledge of SQL and relational databases.
Familiarity with data warehousing solutions such as Snowflake, BigQuery, or Redshift.
Experience with cloud platforms, with strong preference for Azure.
Understanding of data pipelines, ETL/ELT processes, and data architecture principles.
Knowledge of version control systems such as Git.
Strong problem-solving skills and attention to detail.

Preferred Skills

Working knowledge of DBT for data modeling and transformation.
Experience with orchestration tools like Airflow or Prefect.
Knowledge of streaming data pipelines such as Kafka or Kinesis.
Familiarity with containerization using Docker or Kubernetes.
Understanding of data governance, security, and compliance practices.

Perks and Benefits of Working at Algoscale

Opportunity to collaborate with leading companies across the globe.
Opportunity to work with the latest and trending technologies.
Competitive salary and performance-based bonuses.
Comprehensive group health insurance.
Flexible working hours and remote work options (for some positions only).
Generous vacation and paid time off.
Professional learning and development programs and certifications.