Back to jobs
Senior Data Engineer
Successfully
Req. VR-122986
We are looking for a highly skilled Senior/Lead Data Engineer to design, build, and manage scalable data pipelines and data lake solutions. You need to have strong expertise in big data technologies, cloud platforms, and data engineering best practices, along with experience leading teams and delivering high-impact data solutions.
Key Responsibilities:
Design and develop end-to-end ETL/data pipelines using PySpark, Databricks, Delta Lake, and AWS S3.
Integrate data from APIs, mainframe systems, HDFS, Hive, Delta tables, and AWS S3 into enterprise data lakes.
Perform Exploratory Data Analysis (EDA) and Data Quality Analysis (DQA).
Implement Delta Lake best practices including ACID transactions, schema enforcement, and incremental loads.
Build and optimize scalable, reliable, and fault-tolerant pipelines.
Orchestrate workflows using Databricks Workflows, Airflow (EMR), and Control-M.
Collaborate with business stakeholders to translate requirements into technical solutions.
Ensure data governance, data quality, and performance optimization.
Troubleshoot production issues and provide timely resolution. Leadership Responsibilities:
Lead and mentor a team of data engineers.
Conduct code reviews and enforce best practices.
Drive solution design and architecture decisions.
Provide technical guidance and delivery support.
Manage production support and ensure SLA adherence.
Work hours are 11 am/12.00 pm
8.00 pm/9.00 pm IST
You should be able to do hybrid role 2-3 days a week from office, have very strong preference
Must have
Strong experience in PySpark, Spark SQL, and Python
Hands-on experience with Databricks and Delta Lake
Strong understanding of data lake and data warehouse concepts
Experience with AWS services (S3, EMR)
Strong SQL and Hive/HDFS experience
Experience with Airflow, Control-M, or similar tools
Knowledge of data quality frameworks
Strong problem-solving and analytical skills
Excellent communication and stakeholder management
Ability to work in a fast-paced environment
Nice to have
Experience in Insurance domain
Exposure to ML data pipelines
CI/CD and DevOps knowledge
Experience handling large-scale data (100M+ records/day)
Languages
English: C1 Advanced
Seniority
Senior
Chennai, India
Req. VR-122986
Data Modeling
BCM Industry
21/05/2026
Req. VR-122986
Apply for Senior Data Engineer in Chennai
*Indicates a required field