Sign up to access all features of our service
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Data Engineer (Investment Data)

Luxoft

We’re looking for an experienced, hands-on Data Engineer who is capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office. The ideal candidate has financial markets familiarity (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments—proactively shaping solutions, not waiting for tickets. You’ll own data end-to-end: from ingesting vendor and internal sources, to modeling in our lakehouse, to making data discoverable, reliable, and cost-efficient. You’ll partner closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.

Responsibilities

Participate in requirements clarification and sprint planning sessions.

Design technical solutions and implement them, inc ETL Pipelines – Build robust data pipelines in PySpark to extract, transform, using PySpark

Optimize ETL Processes – Enhance and tune existing ETL processes for better performance, scalability, and reliability

Writing unit and integration tests.

Support QA teammates in the acceptance process.

Resolving PROD incidents as a 3rd line engineer.

Skills

Must have

Bachelor’s degree (Computer Science, Engineering, Information Systems, or related discipline).

5+ years experience in data engineering roles (flexible based on depth of capability).

Strong hands-on experience with Databricks in production environments (prerequisite).

Strong programming experience with PySpark (must) and strong SQL (must).

Proven experience with Declarative Pipelines / pipeline orchestration on Databricks (prerequisite).

Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.

Experience delivering production-ready datasets with appropriate operational controls (monitoring, troubleshooting, reliability patterns).

Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).

Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.

Experience working in enterprise environments with CI/CD pipelines and structured release processes.

Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.

Nice to have

Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.

Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.

Familiarity with Dynatrace or Datadog for system observability and monitoring.

Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.

Other

Languages

English: C1 Advanced

Seniority

Senior

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer (Investment Data) in Abu Dhabi vacancy
  •  ...Partners is working with one of the world’s largest private investment firms, which is building its data team in Abu Dhabi. This seat sits upstream of...  ...ll be the person researchers, investment teams and AI engineers trust to tell them the data is right, and to tell them... 

    Durlston Partners

    Abu Dhabi
    8 days ago
  •  ...About the Role We are looking for a Senior Data Engineer to join our growing data team and take ownership of our financial data warehouse, real-time data platform and data governance initiatives. You will work across DWH architecture, real-time streaming, data... 

    Astra Tech

    Abu Dhabi
    5 days ago
  •  ...motivated Databricks Platform Engineer to join its core Lakehouse...  ...integrations that enable ADIA’s Data & AI ecosystem at scale....  ...Understanding of Investment Data domain (desired)....  ...Seniority Lead... 

    Luxoft

    Abu Dhabi
    1 day ago
  •  ...Only applications submitted through the portal will be considered. Thank you. About AgentixLake AgentixLake GmbH is a Data and AI engineering consultancy based in Hamburg, Germany. We design, build and operate production-grade data platforms and AI systems in our... 
    Remote job

    AgentixLake GmbH

    Abu Dhabi
    1 day ago
  •  ...Dhabi) Contract: 6 months DQ Engineer The organization's analytics,...  ...reporting and AI ambitions depend on data that decisionmakers can trust, and the Senior Data Quality Engineer is...  ...and using the Microsoft Purview investment as the governance and master data... 

    Datamatics Technologies

    Abu Dhabi
    1 day ago
  •  ...Travel required) Duration: 6-month assignment Summary  (Data Engineer) Timely, reliable data pipelines are what turn a platform...  ...and reconciliation into pipelines in partnership with the Senior Data Quality Engineer. Ensure that pipelines publish metadata... 

    Datamatics Technologies

    Abu Dhabi
    2 days ago
  •  ...We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode. This role involves working on a large-scale data platform migration using modern data technologies and building production-ready pipelines in a dynamic... 

    EPAM Systems

    Abu Dhabi
    a month ago
  •  ...DeepLight AI is a specialist AI and data consultancy dedicated to transforming the regional corporate landscape through bespoke, high...  ...-impact intelligent systems. We combine deep expertise in data engineering, cloud architecture, AI/ML platforms, and systems integration... 

    DeepLight AI

    Abu Dhabi
    10 days ago
  •  ...Deeplight is a specialist AI and data consultancy dedicated to transforming the regional...  ...cloud architecture, and modern MLOps to engineer secure, scalable, and resilient platforms...  ...banking engagements in Dubai. Supported by senior data engineers embedded within dedicated... 

    DeepLight

    Abu Dhabi
    22 hours ago
  •  ...and P2P partner performance — conversion, cost, failure reasons, settlement reconciliation — and support commercial negotiations with data Deliver data support for regional entities and regulatory requirements, where accuracy standards are externally auditable... 

    Bybit

    Abu Dhabi
    3 days ago
  •  ...Make an impact with NTT DATA Join a company that is pushing the boundaries of what...  ...day at NTT DATA The Network & Security Engineer supports the availability, performance and...  ...partners. NTT DATA is part of NTT Group, which invests over $3 billion each year in R&D.... 

    NTT DATA

    Abu Dhabi
    2 days ago
  •  ...industrial digitalization, building AI and data solutions that solve the world’s hardest,...  ...make impactful business decisions. As a Senior Data Scientist at Cognite, you will...  ...cross-functional teams that include data engineers, solution architects, and project managers... 

    Cognite

    Abu Dhabi
    5 days ago
  •  ...have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking... 

    IFM

    Abu Dhabi
    10 days ago
  • AED17000 - 20000 per month

     ...Urgent requirement for Senior Data Engineer with experience in Redshift Spectrum & ( Retail Banking ,Corporate Banking & SME Banking) in banking domain is required for our banking clients in Abu Dhabi ,UAE Design and develop enterprise data warehouses, data marts, and... 

    Talent Arabia

    Abu Dhabi
    2 days ago
  •  ...Senior Data Governance Consultant – Azure Experience: 7–9 Years Employment Type: Full-Time Role Overview We are looking...  ...and compliance requirements. Collaborate with Azure Data Engineering and Architecture teams to embed governance-by-design into data... 

    NorthBay Solutions

    Abu Dhabi
    17 days ago
  •  ...accelerating infrastructure investment worldwide, our services are in...  ...50,000 planners, designers, engineers, scientists, digital innovators...  ...We're looking for a Senior Engineer - Planning to join our...  ...feasibility studies and provide data-driven recommendations to senior... 

    AECOM

    Abu Dhabi
    2 days ago
  •  ...operating system that turns thousands of data streams into a realtime, 3D command and...  ...order of 50 systems representing a decisive investment that anchors production and accelerates...  ...ABOUT THE JOB We need a Mechatronics Engineer based in Abu Dhabi to support... 

    Anduril Industries

    Abu Dhabi
    3 days ago
  •  ...generation of financial services. (Suitable candidate has to relocate to KL, Malaysia or Abu Dhabi) Job Overview We are seeking a data-driven professional with extensive hands-on experience in big data analytics to join our team. The ideal candidate will be... 

    Bybit

    Abu Dhabi
    3 days ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...HALCON is seeking an experienced Senior Software Engineer to join our...  ...thrive. At EDGE, the real investment goes beyond compensation. Through...  ...accordance with applicable data protection and privacy... 

    EDGE Group

    Abu Dhabi
    6 days ago
  •  ...the Role Orbitworks is seeking a Senior Industrial Engineer to lead the industrialization of our...  ...assembly and test flows, and manufacturing data needs, driving design for manufacture...  ..., system-level results. Develop investment and resource options, lead delivery... 

    Loft Orbital

    Abu Dhabi
    6 days ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...RED is seeking an experienced Senior Linux Systems Engineer to ensure...  ...cyber investigations through data processing, maintaining...  ...thrive. At EDGE, the real investment goes beyond compensation. Through... 

    EDGE Group

    Abu Dhabi
    7 days ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...compliance awareness. Production data analysis and reporting....  ...thrive. At EDGE, the real investment goes beyond compensation. Through...  ..., depending on your role and seniority, benefits can include family... 

    EDGE Group

    Abu Dhabi
    9 days ago
  •  ...Job Description Overview The Senior Project Manager will be responsible for the end-to...  ...logistics, manufacturing, and mission-critical data centre projects across the UAE. The role...  ...documentation. Coordinate architects, engineers, consultants, and specialist vendors.... 

    AtkinsRéalis

    Abu Dhabi
    12 hours ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...performance. o Analyze test data to correlate with computational...  ...thrive. At EDGE, the real investment goes beyond compensation. Through...  ..., depending on your role and seniority, benefits can include family... 

    EDGE Group

    Abu Dhabi
    16 days ago
  •  ...Black Pearl Consult is hiring for a Junior Investment Analyst in Abu Dhabi, UAE. This opportunity suits a finance professional with strong analytical and research skills to support investment decisions within a family office environment. The role focuses on evaluating... 

    Black Pearl Consult

    Abu Dhabi
    1 day ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...ROLE The Special Processes Senior Engineer is a senior technical...  ...thrive. At EDGE, the real investment goes beyond compensation. Through...  ...accordance with applicable data protection and privacy regulations... 

    EDGE Group

    Abu Dhabi
    5 days ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  .... Support test campaigns, data reduction, and post-fire evaluation...  ...thrive. At EDGE, the real investment goes beyond compensation....  ..., depending on your role and seniority, benefits can include family... 

    EDGE Group

    Abu Dhabi
    16 days ago
  •  ...what’s possible. Job Description: Senior Utilities Coordination Engineer Abu Dhabi, UAE Parsons is looking...  ...assess Notice of Information (NOI) data and existing utility records to establish...  ...or genetic information. We truly invest and care about our employee’s wellbeing... 

    Parsons

    Abu Dhabi
    8 days ago
  •  ...EDGE At EDGE bold ideas are engineered into technologies that...  ...ABOUT THE ROLE The Senior Avionics Systems Engineer is...  ...thrive. At EDGE, the real investment goes beyond compensation. Through...  ...in accordance with applicable data protection and privacy regulations... 

    EDGE Group

    Abu Dhabi
    7 days ago
  •  ...troubleshooting, and problem-solving abilities ✅ Strong communication and stakeholder management skills ✅ Experience working in Agile and Waterfall delivery environments ✅ Domain expertise of the following: Capital Markets Investment Banking Equity.... 

    QKTech

    Abu Dhabi
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Data Engineer (Investment Data). Be the first to apply!