DATA ENGINEER - AZURE DATABRICKS & PYSPARK
Duncan & Ross
Duncan & Ross is hiring a Data Engineer for a leading client in Dubai. You'll design, build and run scalable data pipelines and data warehouse solutions on Azure and Databricks, turning raw data from across the business into trusted, analytics-ready data.
What you'll do
- Design, build and maintain ETL/ELT pipelines that ingest, transform and load data from multiple source systems into Azure and Databricks.
- Develop data transformations in Python and PySpark, optimising Spark jobs for performance, scalability and cost.
- Write efficient, well-structured SQL for data modelling, transformation and analysis across relational databases.
- Build and manage data warehouse and lakehouse solutions on Azure and Databricks, including Delta Lake and medallion (bronze, silver, gold) architectures.
- Orchestrate and schedule data workflows using tools such as Azure Data Factory, Databricks Workflows or Apache Airflow, with monitoring and alerting.
- Put data quality checks, validation rules and reconciliation in place so data is accurate, complete and reliable.
- Apply data governance standards, including cataloguing, lineage, access control and security, using tools such as Unity Catalog or Microsoft Purview.
- Work with data analysts, data scientists and business stakeholders to understand requirements and deliver data products.
- Troubleshoot pipeline failures and performance issues, and continuously improve reliability.
- Document pipelines, data models and processes, and follow CI/CD and version control best practices.
What you'll bring
- 3-5+ years of hands-on data engineering experience.
- Data pipelines: proven experience building ETL/ELT pipelines for batch and incremental loads.
- SQL and relational databases: advanced SQL and strong experience with databases such as SQL Server, Azure SQL, PostgreSQL or Oracle.
- Big data: hands-on Apache Spark experience, including performance tuning and handling large datasets.
- Cloud platforms: strong experience with Microsoft Azure (Data Lake Storage, Data Factory, Synapse) and Databricks.
- Data warehousing: experience designing data warehouses or lakehouses on Azure and Databricks, including dimensional modelling (star and snowflake schemas) and Delta Lake.
- Programming: strong Python and PySpark skills.
- Orchestration: experience with Azure Data Factory, Databricks Workflows, Airflow or similar.
- Data quality and governance: experience implementing data quality frameworks, data lineage, metadata management and access controls.
- Familiarity with Git, CI/CD pipelines (Azure DevOps or GitHub Actions) and Agile delivery.
- Bachelor's degree in computer science, Engineering, Information Systems or a related field.
- Fluent written and spoken English.
Nice to have
- Databricks certification (Data Engineer Associate or Professional) or Microsoft Azure Data Engineer Associate (DP-203).
- Experience with streaming data (Kafka, Event Hubs, Spark Structured Streaming).
- Experience with Unity Catalog, Microsoft Purview or Microsoft Fabric.
- Infrastructure as code (Terraform or Bicep).
- Exposure to BI tools such as Power BI.
- Previous experience in the UAE or wider GCC.
Vertical:
Technology
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the DATA ENGINEER - AZURE DATABRICKS & PYSPARK in Dubai vacancy
- ...Artefact is a new generation of data service providers, specialising in data consulting... ...Additionally, you will lead technical teams, mentor engineers, and contribute to Artefact’s growth by... ...and implementing cloud architectures on Azure at scale Strong expertise in cloud...
- ...Azure Consultant (L2) Role summary The Azure Consultant (L2) designs, deploys and supports Azure infrastructure for enterprise... ...AZ-305 Azure Solutions Architect Expert, AZ-500 Azure Security Engineer, or AZ-700 Azure Network Engineer. Nice to have •...
- ...Akshaya IT Solutions is hiring an experienced Data Engineer for a full-time onsite opportunity in Dubai. This role is ideal for professionals with strong expertise in big data technologies, data pipeline development, and cloud-based data engineering who are ready to work...
- ...responsible for designing and implementing innovative solutions using Azure AI and Azure OpenAI services. This role is critical in... ...AI solutions into existing systems to optimize communication and data processing Establish and maintain a CI/CD pipeline for...
- ...their financial goals. Role Overview We are seeking a Senior Data Analyst to join our Quantitative Dealing team and take ownership... ...MT5 databases and trading data structures. Experience with PySpark and large-scale data processing. Familiarity with AWS Glue...
- ...Description • Lead technical discovery with client business, data, security and technology stakeholders; convert business... ...Qualifications • 7+ years of professional experience across software engineering, cloud architecture, AI/ML engineering or enterprise solution...
- ...About CORE Emirates CORE Emirates is a specialist data centre consultancy and commissioning partner supporting the delivery of complex... ...We are looking for an experienced Senior Extra Low Voltage Engineer to take a lead role in the commissioning and validation of Extra...
- ...Process Automation Expert (M365 & Azure) Experience: 5+ years Location: Dubai,UAE Contract: 4-6 months Pay: Competitive salary (varies with experience) Job Summary: Our client is looking for a skilled Process Automation Expert to drive automation...
- ...Position: Data Center Engineer (Mid-Level) Date Posted: 14 October 2026 Industry: Information Technology / Data Center Infrastructure Employment Type: Full-time Experience: Minimum 4 years Qualification: Diploma or Bachelor’s Degree in Information Technology...
- ...In this role, you'll turn data into decisions by identifying high-value business problems, framing them analytically, and delivering... ..., and machine learning, while partnering closely with product, engineering, sales, operations, or finance stakeholders. Your work will not...
- ...Job Title: Palantir Foundry Engineer Location: Dubai, UAE Employment type: Full-time... ...directly with client teams to understand their data and operational challenges, then design... ...platform. - Proficiency in Python and PySpark for building and troubleshooting data...
- ...is a hands-on leadership role for someone who can lead a strong Data Science team while remaining close to the technical work —... ...from concept through to production • Work closely with Product, Engineering and senior business stakeholders • Define the Data Science roadmap...
- ...An excellent opportunity is available for a Data Entry Analyst to manage, verify, and analyze business information while supporting accurate reporting and record management. The role is well suited to a detail-oriented individual who enjoys working with data, spreadsheets...
$30 - $50 per hour
...Position: AI/ML Data Annotation Engineer Date Posted: September 9, 2026 Industry: Artificial Intelligence / Machine Learning / Data Operations Employment Type: Full Time Experience: 3+ Years in Data Operations, ML Operations, Evaluation, or Annotation Engineering...- ...Job Description WSP in the Middle East is currently hiring for Associate / Principal Electrical Design Engineer - Data Centres & Mission Critical who will lead multidisciplinary design, advisory, and delivery of mission-critical data centre projects across the region...
- ...tier banks. And now we need to move fast to secure leadership in this $300Bn+ industry . About the Role We're looking for a Data Analyst to turn every transaction into the reliable analytics. You'll work directly with the co-founders and will own the metrics,...
- ...A data-focused opportunity is available for a Logistics Data Analyst to help improve supply chain performance through accurate reporting, data analysis, and operational insights. The role is ideal for someone who enjoys working with numbers, reports, and logistics operations...
- ...In this role, you design, develop, and apply advanced data science, statistical, and machine learning methods to solve complex, business... ...high-value problems, framing them analytically, and engineering robust features and insights. Design, develop, train, and implement...
- ...About The Team: The Personalization team sits within Careem's Data Science organization and owns the AI systems that decide what... ...quantitative discipline such as Physics, Statistics, Mathematics, Engineering, or Computer Science. ~ Solid experience with deep learning...
- ...A reputable company is seeking a detail-oriented data entry operator to compile, capture, and maintain our digital database. As a data entry operator, you will be required to compile, merge, and format documents for data entry, check documents for errors, input data,...
- ...A structured business environment in Dubai offers an opportunity for a detail-focused Data Quality Assistant to review information, maintain accurate records, identify data inconsistencies, and support reliable day-to-day data processing activities. Key Responsibilities...
- 1inch is the DeFi ecosystem building financial freedom for everyone. 1inch products help users and builders trade, hold and track digital assets - with the self-custody, comprehensive security and the intuitive user experience they need to unlock the potential of true crypto...
- ...agile services driven organization, committed to enabling future thinkers. Job Description Lead and manage end-to-end data and AI projects , ensuring alignment with business objectives. Oversee data collection, processing, modeling, and AI-driven...
- ...Job Title – Data & Business Intelligence Analyst Location: Dubai, UAE About the Role We are seeking a detail-oriented and analytical Data & Business Intelligence Analyst to transform complex datasets into actionable business insights that support strategic...
- ...A reputable company is seeking a detail-oriented data entry operator to compile, capture, and maintain our digital database. As a data entry operator, you will be required to compile, merge, and format documents for data entry, check documents for errors, input data,...
- ...Talent One is seeking a Data Integration Officer to develop, monitor, and support reliable data interfaces across enterprise systems... ...procedures, and incident records. • Coordinate with data engineers, application teams, vendors, analysts, and business users....
- ...is seeking an experienced Senior IT Field Engineer to join its technical services team in... ...Server, Active Directory, Microsoft 365, and Azure environments. • Configure and... ...and disaster recovery solutions to ensure data protection. • Perform system maintenance...
- ...employees including more than 250 certified engineers. We offer in-depth expertise and a range... ...the design, deployment and migration of data-centre infrastructure for CNS customers -... .... Migrate on-prem workloads to Azure, AWS or local UAE cloud, and Exchange to...
- ...will own end-to-end project delivery for large, multidisciplinary data centre and mission-critical facility projects, responsible for... ...stakeholder relationships ~ Proven ability to mentor engineers and develop high-performing teams Qualifications Technical...
- Who are we? Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet. A place where bold ideas are welcomed, human connection is valued, and everyone has the opportunity...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to DATA ENGINEER - AZURE DATABRICKS & PYSPARK. Be the first to apply!
