Back to jobs
Databricks Data Platform Engineer/Architect
Successfully
Req. VR-118910
We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering. The candidate will have a deep understanding of both data platforms and software engineering, enabling them to effectively integrate and operate the platform within a broader IT ecosystem.
This role requires a hands-on individual contributor who takes full ownership of deliverables end-to-end, including design, development, testing, deployment, and ongoing support.
Manage and optimize Databricks data platform including workspace setup, cluster policies, job orchestration, Unity Catalog, cost controls, multi-tenancy.
Design, write and maintain APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
Ensure high availability, security, and performance of data systems which includes access control, secrets management, RBAC, monitoring, alerting, RLS, incident handling, performance tuning.
Provide valuable insights about the data platform (Databricks) usage which includes cost attribution, usage analytics, workload patterns, telemetry.
Aware of new features of Databricks including serverless, Declarative Pipelines, Agents, lakebase , etc.
Design and maintain system libraries (Python) used in ETL pipelines and platform governance (Databricks).
Optimize ETL Processes
Enhance and tune existing ETL processes for better performance, scalability, and reliability.
Must have
Minimum 10 Years of experience in IT/Data.
Minimum 3 years of experience as a Databricks Data Platform Engineer.
3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
Bachelor's in IT or related field.
Infrastructure & Cloud: Azure, AWS (expertise in storage, networking, compute).
Programming: Proficiency in PySpark for distributed computing.
minimum 4 years of Python experience for ETL development.
SQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
Data Warehousing: Experience working with data warehousing concepts and Databricks platform.
ETL Tools: Familiarity with ETL tools & processes
Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
Data Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo.
Nice to have
Containerization & Orchestration: Docker, Kubernetes.
Infrastructure as Code (IaC): Terraform.
Understanding of Investment Data domain (desired).
Languages
English: C1 Advanced
Seniority
Lead
Abu Dhabi, United Arab Emirates
Req. VR-118910
DWH Development
BCM Industry
11/02/2026
Req. VR-118910
Apply for Databricks Data Platform Engineer/Architect in Abu Dhabi
*Indicates a required field