Senior Data Engineer (Investment Data)
Luxoft
We’re looking for an experienced, hands-on Data Engineer who is capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office. The ideal candidate has financial markets familiarity (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments—proactively shaping solutions, not waiting for tickets. You’ll own data end-to-end: from ingesting vendor and internal sources, to modeling in our lakehouse, to making data discoverable, reliable, and cost-efficient. You’ll partner closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.
Responsibilities
Participate in requirements clarification and sprint planning sessions.
Design technical solutions and implement them, inc ETL Pipelines – Build robust data pipelines in PySpark to extract, transform, using PySpark
Optimize ETL Processes – Enhance and tune existing ETL processes for better performance, scalability, and reliability
Writing unit and integration tests.
Support QA teammates in the acceptance process.
Resolving PROD incidents as a 3rd line engineer.
Skills
Must have
Bachelor’s degree (Computer Science, Engineering, Information Systems, or related discipline).
5+ years experience in data engineering roles (flexible based on depth of capability).
Strong hands-on experience with Databricks in production environments (prerequisite).
Strong programming experience with PySpark (must) and strong SQL (must).
Proven experience with Declarative Pipelines / pipeline orchestration on Databricks (prerequisite).
Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.
Experience delivering production-ready datasets with appropriate operational controls (monitoring, troubleshooting, reliability patterns).
Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).
Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.
Experience working in enterprise environments with CI/CD pipelines and structured release processes.
Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.
Nice to have
Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.
Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.
Familiarity with Dynatrace or Datadog for system observability and monitoring.
Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.
Other
Languages
English: C1 Advanced
Seniority
Senior
- ...Partners is working with one of the world’s largest private investment firms, which is building its data team in Abu Dhabi. This seat sits upstream of... ...ll be the person researchers, investment teams and AI engineers trust to tell them the data is right, and to tell them...
- ...About the Role We are looking for a Senior Data Engineer to join our growing data team and take ownership of our financial data warehouse, real-time data platform and data governance initiatives. You will work across DWH architecture, real-time streaming, data...
- ...motivated Databricks Platform Engineer to join its core Lakehouse... ...integrations that enable ADIA’s Data & AI ecosystem at scale.... ...Understanding of Investment Data domain (desired).... ...Seniority Lead...
- ...Only applications submitted through the portal will be considered. Thank you. About AgentixLake AgentixLake GmbH is a Data and AI engineering consultancy based in Hamburg, Germany. We design, build and operate production-grade data platforms and AI systems in our...Remote job
- ...Dhabi) Contract: 6 months DQ Engineer The organization's analytics,... ...reporting and AI ambitions depend on data that decisionmakers can trust, and the Senior Data Quality Engineer is... ...and using the Microsoft Purview investment as the governance and master data...
- ...Travel required) Duration: 6-month assignment Summary (Data Engineer) Timely, reliable data pipelines are what turn a platform... ...and reconciliation into pipelines in partnership with the Senior Data Quality Engineer. Ensure that pipelines publish metadata...
- ...We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode. This role involves working on a large-scale data platform migration using modern data technologies and building production-ready pipelines in a dynamic...
- ...DeepLight AI is a specialist AI and data consultancy dedicated to transforming the regional corporate landscape through bespoke, high... ...-impact intelligent systems. We combine deep expertise in data engineering, cloud architecture, AI/ML platforms, and systems integration...
- ...Deeplight is a specialist AI and data consultancy dedicated to transforming the regional... ...cloud architecture, and modern MLOps to engineer secure, scalable, and resilient platforms... ...banking engagements in Dubai. Supported by senior data engineers embedded within dedicated...
- ...and P2P partner performance — conversion, cost, failure reasons, settlement reconciliation — and support commercial negotiations with data Deliver data support for regional entities and regulatory requirements, where accuracy standards are externally auditable...
- ...Make an impact with NTT DATA Join a company that is pushing the boundaries of what... ...day at NTT DATA The Network & Security Engineer supports the availability, performance and... ...partners. NTT DATA is part of NTT Group, which invests over $3 billion each year in R&D....
- ...industrial digitalization, building AI and data solutions that solve the world’s hardest,... ...make impactful business decisions. As a Senior Data Scientist at Cognite, you will... ...cross-functional teams that include data engineers, solution architects, and project managers...
- ...have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking...
AED17000 - 20000 per month
...Urgent requirement for Senior Data Engineer with experience in Redshift Spectrum & ( Retail Banking ,Corporate Banking & SME Banking) in banking domain is required for our banking clients in Abu Dhabi ,UAE Design and develop enterprise data warehouses, data marts, and...- ...Senior Data Governance Consultant – Azure Experience: 7–9 Years Employment Type: Full-Time Role Overview We are looking... ...and compliance requirements. Collaborate with Azure Data Engineering and Architecture teams to embed governance-by-design into data...
- ...accelerating infrastructure investment worldwide, our services are in... ...50,000 planners, designers, engineers, scientists, digital innovators... ...We're looking for a Senior Engineer - Planning to join our... ...feasibility studies and provide data-driven recommendations to senior...
- ...operating system that turns thousands of data streams into a realtime, 3D command and... ...order of 50 systems representing a decisive investment that anchors production and accelerates... ...ABOUT THE JOB We need a Mechatronics Engineer based in Abu Dhabi to support...
- ...generation of financial services. (Suitable candidate has to relocate to KL, Malaysia or Abu Dhabi) Job Overview We are seeking a data-driven professional with extensive hands-on experience in big data analytics to join our team. The ideal candidate will be...
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...HALCON is seeking an experienced Senior Software Engineer to join our... ...thrive. At EDGE, the real investment goes beyond compensation. Through... ...accordance with applicable data protection and privacy...
- ...the Role Orbitworks is seeking a Senior Industrial Engineer to lead the industrialization of our... ...assembly and test flows, and manufacturing data needs, driving design for manufacture... ..., system-level results. Develop investment and resource options, lead delivery...
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...RED is seeking an experienced Senior Linux Systems Engineer to ensure... ...cyber investigations through data processing, maintaining... ...thrive. At EDGE, the real investment goes beyond compensation. Through...
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...compliance awareness. Production data analysis and reporting.... ...thrive. At EDGE, the real investment goes beyond compensation. Through... ..., depending on your role and seniority, benefits can include family...
- ...Job Description Overview The Senior Project Manager will be responsible for the end-to... ...logistics, manufacturing, and mission-critical data centre projects across the UAE. The role... ...documentation. Coordinate architects, engineers, consultants, and specialist vendors....
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...performance. o Analyze test data to correlate with computational... ...thrive. At EDGE, the real investment goes beyond compensation. Through... ..., depending on your role and seniority, benefits can include family...
- ...Black Pearl Consult is hiring for a Junior Investment Analyst in Abu Dhabi, UAE. This opportunity suits a finance professional with strong analytical and research skills to support investment decisions within a family office environment. The role focuses on evaluating...
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...ROLE The Special Processes Senior Engineer is a senior technical... ...thrive. At EDGE, the real investment goes beyond compensation. Through... ...accordance with applicable data protection and privacy regulations...
- ...EDGE At EDGE bold ideas are engineered into technologies that... .... Support test campaigns, data reduction, and post-fire evaluation... ...thrive. At EDGE, the real investment goes beyond compensation.... ..., depending on your role and seniority, benefits can include family...
- ...what’s possible. Job Description: Senior Utilities Coordination Engineer Abu Dhabi, UAE Parsons is looking... ...assess Notice of Information (NOI) data and existing utility records to establish... ...or genetic information. We truly invest and care about our employee’s wellbeing...
- ...EDGE At EDGE bold ideas are engineered into technologies that... ...ABOUT THE ROLE The Senior Avionics Systems Engineer is... ...thrive. At EDGE, the real investment goes beyond compensation. Through... ...in accordance with applicable data protection and privacy regulations...
- ...troubleshooting, and problem-solving abilities ✅ Strong communication and stakeholder management skills ✅ Experience working in Agile and Waterfall delivery environments ✅ Domain expertise of the following: Capital Markets Investment Banking Equity....
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer (Investment Data). Be the first to apply!
