Project description

We are seeking a highly skilled and motivated Databricks Platform Engineer to join its core Lakehouse Platform team.
This role focuses on engineering and evolving the Databricks-based Lakehouse platform itself, building advanced platform capabilities, reusable services, automation frameworks, and enterprise integrations that enable ADIA’s Data & AI ecosystem at scale.
While the platform supports data pipelines and analytics workloads, this is not a traditional data engineering role focused on building ETL jobs or delivering datasets. Instead, the role is centred on:
• Designing and developing shared platform capabilities
• Building internal libraries and APIs
• Implementing governance, security, and integration patterns
• Ensuring scalability, reliability, and operational excellence
The ideal candidate combines strong Python software engineering expertise with solid infrastructure understanding and experience integrating enterprise systems. Clear, precise, and articulate communication is essential.
The primary focus of this role is building and enhancing platform capabilities, not developing business ETL pipelines or reporting datasets.

Responsibilities

Platform Capability Engineering

Collaborate with stakeholders during requirements clarification and sprint planning to ensure alignment with platform strategy and enterprise standards.

Design and implement technical solutions on ADIA’s Lakehouse platform (Databricks), including:

Prototyping and productising new platform capabilities.

Developing reusable Python libraries, SDKs, and internal frameworks.

Exposing platform capabilities to support ADIA’s Data Products strategy and broader Data & AI ecosystem.

Build automation for provisioning, governance controls, configuration management, and operational standards.

Enterprise Integration & APIs

Integrate the Lakehouse platform with enterprise systems, including:

Incident and monitoring systems (e.g., ServiceNow).

Identity and access management solutions.

Observability platforms (e.g., Dynatrace, Prometheus, Grafana).

Design and implement secure API-based integrations using enterprise authentication and authorisation mechanisms.

Reliability, Quality & Operations

Develop and maintain unit and integration tests to ensure quality and resilience of platform components.

Support QA teams during acceptance testing of platform features.

Act as a third-line engineer for complex production incidents affecting platform stability.

Collaborate with cloud and infrastructure teams to deliver secure, scalable, and reliable platform architecture.

Skills

Must have

Minimum 10 Years of experience in IT/Data.

Minimum 3 years of expertise in Databricks from a platform enablement perspective.

3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management

Advanced Python skills for production-grade library and service development.

Solid understanding of distributed processing concepts (PySpark knowledge beneficial, but not limited to ETL use cases).

Experience designing and integrating REST APIs.

Advanced SQL skills for troubleshooting and performance optimisation (e.g., Oracle, MS SQL).

Experience with Git and CI/CD workflows.

Familiarity with monitoring and observability tools (e.g., ServiceNow, Prometheus, Grafana).

Knowledge of scheduling and orchestration tools (e.g., Stonebranch, Control-M, Airflow).

Understanding of data quality and governance frameworks (e.g., Great Expectations).

Strong grasp of cloud infrastructure fundamentals:

DNS

Certificates

Identity & access management

Networking concepts

Load balancing

Nice to have

Containerization & Orchestration: Docker, Kubernetes.

Infrastructure as Code (IaC): Terraform.

Understanding of Investment Data domain (desired).

Other

Languages

English: C1 Advanced

Seniority

Lead

Related jobs

View all vacancies

Lead Data Architect

DWH Development

United States of America

Chicago

Abu Dhabi, United Arab Emirates

Req. VR-125848

DWH Development

BCM Industry

30/10/2026

Req. VR-125848

Apply for Senior Databricks Data Platform Engineer in Abu Dhabi

*Indicates a required field

First name

Last name

Your country

Your City

Phone

Specialization

Attach your resume (max file size 1MB; allowed extensions: doc, txt, pdf, docx) Upload resume

Number of years of experience in the specialization you are applying for

What are your monthly gross salary expectations?

Read Explanatory Notes

Under the terms of your specific consent or to perform our obligations under a contract with you, as applicable, we, Luxoft Holding Inc. will manually and electronically process your personal data, specifically your first name, last name, phone number, e-mail address and other data you provide us through this form.

Within this context, we process personal data only for the specific purpose(s) indicated in the individual consent language or other notices provided below.

We will – insofar as reasonably necessary for the purpose you have agreed to and within the scope of applicable laws – transfer your personal data to other entities within the Luxoft Group and to the group of third party recipients listed in our Privacy Notice. Such Recipients can be located outside the European Union (EU) and/or the European Economic Area (EEA) (“Third Countries”). The Third Countries concerned, e.g. the USA, may not have the level of data protection that you enjoy e.g. under the GDPR. This can result in disadvantages such as an impeded enforcement of data subjects’ rights, a lack of control over further processing and access by state authorities. You may only have limited legal remedies against this. Insofar our transfer of your personal data to recipients in Third Countries is not covered by an adequacy decision of the EU Commission, we achieve an adequate level of data protection as further detailed out in our Privacy Notice.

With your consent, we personalise marketing communications to you by way of carrying out marketing research analysis, analysing the surfing-behaviour of our website visitors and to adjust it to their detected tendencies, as well as to plan more efficient future marketing activities. This personalised marketing does not include any automated decision-making activities.

Further information on how we process personal data in general is available in our Privacy Notice. You may withdraw any given consent at any time. The withdrawal of your consent(s) will not affect the lawfulness of processing before its withdrawal. For any request in this context, please e-mail us at: DPO@luxoft.com.

Before uploading CV or any other information to this website, to learn more about your obligations and restrictions arising from the use of this website, please read our Terms of Use.

By clicking the Agree and Send button, you consent that Luxoft Holding Inc. can process your provided personal data for the purpose of processing your application to establish a potential employment, internship or relocation with Luxoft Holding Inc., a relevant affiliate of the Luxoft Group or Luxoft Group’s clients, as applicable. You consent to the retention of your personal data, with the purpose of contacting you for potential future employment opportunities. If you refer a friend, we will process your provided personal data of you and your referral to award you with a bonus for such recommendation. You can withdraw your consent at any time with effect for the future. Before you consent, please click on „Read these Explanatory Notes“ and view our Privacy Notice (e.g. for your data subject rights).

Note: Our system automatically converts each CV into our standardized format, enabling our HR teams to process applications more efficiently.

I consent that Luxoft Holding Inc. and the affiliates of the Luxoft Group can process my personal data for the purpose of sending me regular emails with information about Luxoft as an employer including job vacancies and (ii) marketing communications with information about recruitment-related events, organized webinars, conferences, as well as the Logeek Magazine and other similar publications. I can withdraw my consent at any time with effect for the future. These communications are personalised to my interests in accordance with our Privacy Notice.

Related Jobs

All jobs