Senior Data Engineer (Investment Data)
Luxoft
We’re looking for an experienced, hands-on Data Engineer who is capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office. The ideal candidate has financial markets familiarity (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments—proactively shaping solutions, not waiting for tickets. You’ll own data end-to-end: from ingesting vendor and internal sources, to modeling in our lakehouse, to making data discoverable, reliable, and cost-efficient. You’ll partner closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.
Responsibilities
Participate in requirements clarification and sprint planning sessions.
Design technical solutions and implement them, inc ETL Pipelines – Build robust data pipelines in PySpark to extract, transform, using PySpark
Optimize ETL Processes – Enhance and tune existing ETL processes for better performance, scalability, and reliability
Writing unit and integration tests.
Support QA teammates in the acceptance process.
Resolving PROD incidents as a 3rd line engineer.
Skills
Must have
Bachelor’s degree (Computer Science, Engineering, Information Systems, or related discipline).
5+ years experience in data engineering roles (flexible based on depth of capability).
Strong hands-on experience with Databricks in production environments (prerequisite).
Strong programming experience with PySpark (must) and strong SQL (must).
Proven experience with Declarative Pipelines / pipeline orchestration on Databricks (prerequisite).
Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.
Experience delivering production-ready datasets with appropriate operational controls (monitoring, troubleshooting, reliability patterns).
Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).
Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.
Experience working in enterprise environments with CI/CD pipelines and structured release processes.
Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.
Nice to have
Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.
Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.
Familiarity with Dynatrace or Datadog for system observability and monitoring.
Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.
Other
Languages
English: C1 Advanced
Seniority
Senior
- ...We are seeking an experienced Senior Investment Administrator to oversee fund accounting, investment administration and investor transactions within an ADGM environment. The successful candidate will bring extensive experience across fund accounting, fund administration...
- ...We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode. This role involves working on a large-scale data platform migration using modern data technologies and building production-ready pipelines in a dynamic...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...Description We are looking for Graduate - Data Engineering Specialist to join our AECOM...
- ...accelerating infrastructure investment worldwide, our services are in... ...50,000 planners, designers, engineers, scientists, digital innovators... ...We're looking for a Senior Engineer - Planning to join our... ...feasibility studies and provide data-driven recommendations to senior...
- ...are seeking an experienced and pragmatic Senior AI Engineer to lead the end-to-end delivery of high... ...Domains: Pensions administration, investment operations, and corporate support functions... ...intelligence platforms, automated data reconciliation, and enterprise AI governance...
- ...operating system that turns thousands of data streams into a realtime, 3D command and... ...order of 50 systems representing a decisive investment that anchors production and accelerates... ...ABOUT THE JOB We need a Mechatronics Engineer based in Abu Dhabi to support...
- ...We are exclusively retained to appoint a founding Investment Director / Managing Director for a newly established, FOAK independent private... ...a true strategic partner to the CEO and Managing Partner, this senior hire will shape the firm’s investment approach, portfolio...
- ...An established investment platform is seeking an Associate to support direct investments and strategic transactions across infrastructure... ...investment memoranda, financial analysis and presentation materials for senior management and Investment Committee review. Candidate...
- ...of the probationary period. Qualification (minimum) Master of Science in Data Analytics or bachelor's degree in computer science/engineering or equal . Experience (minimum) 10+ Years experience in Information Technology...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...We are looking for an experienced Senior Mechanical Engineer to join our infrastructure...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...Requirements: We are looking for an experienced Senior Engineer – Roads to join our...
- ...AECOM. With accelerating infrastructure investment worldwide, our services are in great demand... ...of over 50,000 planners, designers, engineers, scientists, digital innovators, program... ...world. Join us. Job Description Senior Mechanical Engineer with strong...
- ...Corporate, investments and M&A lawyers Are you an experienced M&A and corporate transactions lawyer looking to work on some of the... ...transaction related governance matters. Partner closely with senior executives and business leaders to deliver pragmatic, commercially...
- ...in architecture, consulting, construction engineering and mobility services. We create and... ...Description Position Overview The Senior ELV Engineer will be responsible for supervising... ...Structured Cabling Systems. Data Network Infrastructure. IPTV and SMATV...
- ...We need a Mechatronics Engineer based in Abu Dhabi to support integration of electromechanical actuation, sensing, and control subsystems... ...Python, MATLAB, C++, Bash or Rust) to automate testing, collect data, or interface with onboard software. ~ Proficiency across a meaningful...
- ...accelerating infrastructure investment worldwide, our services are in... ...50,000 planners, designers, engineers, scientists, digital innovators... ...in the UAE. Reporting to the Senior Planning Engineer or Planning... ...Value Management (EVM) data, including SPI, CPI, earned value...
- ...We are seeking an experienced Data Center Commissioning Engineer – ELV to support ELV commissioning activities. The candidate will coordinate ELV commissioning activities, system preparation, testing, start-up and handover in accordance with approved procedures and project...
- ...Attend vendor kick-off meetings, inspections, FAT, and expedite as required. Manage vendor data reviews and close out technical queries. Provide site engineering support during construction, installation, and hook-up, including resolving RFIs, NCRs, and field...
- ...AirQ is looking for an experienced Senior Systems Engineer to take technical ownership of the Cargo 200 Programme and 40 Programme , a VTOL... ...cards, support flight envelope expansion, review flight-test data and lead anomaly resolution. Airworthiness & Technical Coordination...
- ...better time to be at AECOM. With accelerating infrastructure investment worldwide, our services are in great demand. We invite you to... ...become part of a global team of over 50,000 planners, designers, engineers, scientists, digital innovators, program and construction...
- ...across the life of their assets. We specialize in consulting, engineering, procurement and construction across the project lifecycle, with... ...Specifications and design calculations. Designs, drawings and data by other Contractors' or by Vendors. Engineering studies and...
- ...Position: Data Classification Engineer Location: Abu Dhabi (Onsite) Required Experience: 6+ years Salary Range: AED 15,000-16,000 Summary: We are looking for a Data Classification Engineer to help manage how we organize and secure our data. In this role,...
- ...their assets. We specialize in consulting, engineering, procurement and construction across the... ...An experienced and highly skilled Senior Instrumentation and Control Engineer to lead... ...packages. Development of I&C specifications, data sheets, Instrumentation & Control System...
- Full Time Posted 3 weeks ago ~Develop and maintain Conceptual, Logical, and Physical DataModels. ~Perform quality assurance reviews of DRS, DDM, Physical DataModels, Erwin submissions, and Authoritative Register models. ~Validate datamodeling standards, naming conventions...
- ...have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking...
- ...Join our UAE Defence Services team and help transform complex military data into actionable insights that support mission-critical decision-making. You will work at the intersection of data engineering, Artificial Intelligence (AI) including agentic, and system...
- Senior engineer delivering features end to end: React / Next.js front end, Python (FastAPI / Django) back end, database integration and... ...Experience in financial services, sovereign wealth / investment holding or other regulated enterprise environments....
- ...We are looking for an experienced Electrical QA/QC Engineer to oversee and ensure the quality of electrical installation works for a Data Center / Mission Critical project. The role will be responsible for implementing project QA/QC procedures, conducting inspections...
- ...We are looking for an experienced Mechanical QA/QC Engineer to oversee and ensure the quality of mechanical installation works for a Data Center project for our client JLW. The role will be responsible for implementing project quality requirements, conducting inspections...
- ...international player active in architecture, consulting, construction engineering and mobility services. We create and operate intelligent... .... Job Description Position Overview The Senior Mechanical Engineer will lead the supervision and inspection of...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer (Investment Data). Be the first to apply!
