Data Engineer

Location: Hyderabad / Remote (with quarterly visits)
Notice Period: 30 Days
Experience: 5+ Years

Responsibilities:

• Design and develop business-critical backend systems using high-quality data pipelines.
• Assemble large, complex datasets that meet functional and non-functional business requirements.
• Build infrastructure for optimal data extraction, transformation, and loading (ETL) from various data sources using Azure Databricks and Azure Data Factory (ADF).
• Develop a cloud-native, real-time stream processing and data lake platform that scales to massive data volumes.
• Construct data pipelines that deliver actionable insights into customer acquisition, operational efficiency, and key business performance metrics.
• Automate data pipelines using Logic Apps and DevOps practices.
• Conduct root cause analysis on external and internal data processes to identify improvement opportunities.
• Embrace continuous improvement, always seeking to enhance existing systems with better solutions.
• Collaborate with stakeholders to design and code large-scale batch and real-time data pipelines on Azure.
• Perform code reviews, emphasizing readability, extensibility, and testability.

Requirements:

• Bachelor’s degree in Computer Science or a related field.
• 5+ years of experience as a Data Engineer.
• Proficiency in Azure big data tools, including Azure Databricks, Azure Synapse, ADLS, and Azure Data Factory (ADF).
• Strong knowledge of relational SQL and Azure Synapse.
• Excellent problem-solving and analytical skills, especially in handling structured and unstructured datasets using Azure big data tools.
• Hands-on experience with data pipeline and workflow management tools such as ADF and Logic Apps.
• Familiarity with Azure cloud services, including Azure SQL DB, Azure Synapse, and Azure Databricks.
• Proficiency in Python for scripting and automation.