IN-Senior Associate_ Data Engineer Databricks_Data and Analytics_Advisory_ Bangalore

Description

& Summary: We are seeking an experienced Data Engineer – Databricks to design, build, and operate scalable, high-performance data pipelines on the Databricks Lakehouse Platform. The role involves hands-on development using Apache Spark, Databricks notebooks, Delta Lake, and cloud-native services, along with close collaboration with analytics, AI/ML, and business teams

Responsibilities:

· Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL) • Implement batch and incremental data processing using Delta Lake and multi-hop architecture • Develop and optimize Databricks notebooks, jobs, and workflows • Ingest, transform, and curate large-scale structured and semi-structured datasets • Support analytics, reporting, and downstream data consumption use cases • Ensure data quality, reliability, lineage, and governance • Collaborate with data scientists, analysts, and architects on AI/ML workloads • Optimize Spark j...