Senior Data Engineer (Investment Data)
Luxoft
We’re looking for an experienced, hands-on Data Engineer who is capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office. The ideal candidate has financial markets familiarity (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments—proactively shaping solutions, not waiting for tickets. You’ll own data end-to-end: from ingesting vendor and internal sources, to modeling in our lakehouse, to making data discoverable, reliable, and cost-efficient. You’ll partner closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.
Responsibilities
Participate in requirements clarification and sprint planning sessions.
Design technical solutions and implement them, inc ETL Pipelines – Build robust data pipelines in PySpark to extract, transform, using PySpark
Optimize ETL Processes – Enhance and tune existing ETL processes for better performance, scalability, and reliability
Writing unit and integration tests.
Support QA teammates in the acceptance process.
Resolving PROD incidents as a 3rd line engineer.
Skills
Must have
Bachelor’s degree (Computer Science, Engineering, Information Systems, or related discipline).
5+ years experience in data engineering roles (flexible based on depth of capability).
Strong hands-on experience with Databricks in production environments (prerequisite).
Strong programming experience with PySpark (must) and strong SQL (must).
Proven experience with Declarative Pipelines / pipeline orchestration on Databricks (prerequisite).
Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.
Experience delivering production-ready datasets with appropriate operational controls (monitoring, troubleshooting, reliability patterns).
Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).
Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.
Experience working in enterprise environments with CI/CD pipelines and structured release processes.
Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.
Nice to have
Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.
Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.
Familiarity with Dynatrace or Datadog for system observability and monitoring.
Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.
Other
Languages
English: C1 Advanced
Seniority
Senior
- ...disciplines to push the frontier of quantitative investing. They blend deep expertise with novel... ..., backed by rigorous research and engineering. The team's core mission is to accelerate... ...by delivering high-quality, reliable data and the scalable technology systems that...
- ...Location: Onsite – Abu Dhabi, UAE Employment Type: Full-Time, Permanent Job Summary We are seeking an experienced Senior Data Engineer with 8–10+ years of experience in designing, developing, and managing modern data platforms, data pipelines, and cloud-...
- ...We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode. This role involves working on a large-scale data platform migration using modern data technologies and building production-ready pipelines in a dynamic...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...Description We are looking for Graduate - Data Engineering Specialist to join our AECOM...
- Full Time Posted 2 days ago ~Develop and maintain Conceptual, Logical, and Physical DataModels. ~Perform quality assurance reviews of DRS, DDM, Physical DataModels, Erwin submissions, and Authoritative Register models. ~Validate datamodeling standards, naming conventions...
- ...have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking...
- ...standards, CORE Emirates supports the delivery of complex, mission-critical infrastructure across one of the world’s fastest-growing data centre markets. Established in 2011, CORE Emirates is a specialist data centre consultancy and commissioning partner, supporting...
- ...Position: Data Classification Engineer Location: Abu Dhabi (Onsite) Required Experience: 6+ years Salary Range: AED 15,000-16,000 Summary: We are looking for a Data Classification Engineer to help manage how we organize and secure our data. In this role,...
- Confidential
- ...Job Purpose ~ The Senior Materials Engineer is responsible for overseeing the selection, evaluation, testing, approval, and quality control of... .... Analyze pavement construction quality and performance data. Support pavement management initiatives through material...
- ...que hacemos. En estos momentos buscamos incorporar un/a Data Scientist Senior con disponibilidad para trabajar presencialmente en Abu... ...testing y validación. Colaborarás de forma estrecha con Data Engineers, Software Engineers, Analysts y otros Data Scientists del...
- ...We are seeking a Junior Commissioning Engineer (Electrical) to support the commissioning of a 200MW data center project. This role focuses on verifying that electrical distribution and power systems meet design intent, code requirements, and the Owner's Project Requirements...
- ...We are seeking a Junior Commissioning Engineer (Mechanical) to support the commissioning of a 200MW data center project. This role assists in the verification, testing, and documentation of mechanical systems to confirm they are designed, installed, and performing in...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...Join us. Job Description The Senior BIM Engineer is responsible for managing...
- ...Description Description of the Role Provide structural engineering services, including but not limited to design review services,... ..., Accountant and Managing Director in analyzing the financial data associated with your projects. Duties and Responsibilities...
- ...Join our UAE Defence Services team and help transform complex military data into actionable insights that support mission-critical decision-making. You will work at the intersection of data engineering, Artificial Intelligence (AI) including agentic, and system...
- ...foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in... ...the next generation of AI pioneers. The Role As a Senior Quality Assurance Engineer , you will play a critical...
- ...real-world applications. As part of our engineering team, you will operate at the... ...multimodal systems at scale. The Role As a Senior MLOps Engineer , you will design, build,... ...ownership of the model lifecycle — from data ingestion to real-time serving — and ensure...
- ...Corporate, investments and M&A lawyers Are you an experienced M&A and corporate transactions lawyer looking to work on some of the most... ...related governance matters. Partner closely with senior executives and business leaders to deliver pragmatic, commercially...
- ...We are seeking a detail-oriented and highly skilled Senior QA Engineer with 6–8 years of experience to ensure the delivery of high-quality software... ..., and security. Conduct database testing to verify data integrity, consistency, and backend validation using SQL...
- ...Description The Sr. Electrical Estimation Engineer will take on a pivotal role in the... ...HV/MV/LV equipment and systems. The Senior Electrical Estimation Engineer will be responsible... ...and update the suppliers/manufacturers' data bank records, ensuring the information...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...Description We are seeking an experienced Senior BIM Engineer to join our Construction...
- ...Job Purpose The Asset Data & Reliability Engineering Specialist is responsible for the integrity, coordination, and governance of data across all... ...Any other duties that may be assigned by the reporting Senior Data Intelligence Manager. Qualifications & Experience...
AED20001 - 25000 per month
Looking for Senior Procurement Engineer having FM Industry Experience – Job location is Abu Dhabi- ...We are seeking a Junior Commissioning Engineer (ELV/FLS) to support the commissioning of a 200MW data center project. This role covers Extra Low Voltage (ELV) systems and Fire Life Safety (FLS) systems, ensuring they are installed, integrated, and functioning per design...
- ...have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking...
AED5000 - 7000 per month
...Job Description: Video Engineer (×1) VIDEO / LED DEPARTMENT Reports to Video / LED HOD Department Video / LED Direct... ...Al Khatim, Abu Dhabi (heavily on-site) Role summary Senior on-show operator and second-in-command in the department. Runs...- ...Employment Type: Full-Time, Permanent Work Model: Hybrid Location: UAE Role Overview We are seeking a highly skilled Senior Full Stack Engineer (React.js & Python/Node.js) to contribute to the design, development, and delivery of scalable, cloud-native...
- ...EIS Technology Solutions LLC is seeking an experienced Senior Audio-Visual Engineer to provide on-site AV engineering support for a landmark ADNOC facility in Abu Dhabi. This long-term project-based opportunity is ideal for professionals with advanced AV system expertise...
- ...My client a leading financial institution is looking for an AI engineer who has experience in banking. Great opportunity to join the Innovation... ..., and product managers on refining features Collaborate with Data Scientist and Data Engineers to improve the Data Quality and...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer (Investment Data). Be the first to apply!
