Data Engineer

Hello There,

Wish you a Happy Monday,

My name is Yashmita, and I am a Technical Recruiter at Kanak IT Services LLC. I am reaching out to you regarding the following job opportunity. If you are interested, kindly reply to this email with your updated resume.

Position : Databricks Data Engineer

Location : [Bay Area, CA]

Duration : 12+ Months

Need experience with Databricks

Job Summary :

We are seeking a skilled Data Engineer with hands-on Databricks experience to design, build, and optimize large-scale data pipelines and analytics solutions. You will work with cross-functional teams to enable scalable data processing using the Databricks Lakehouse Platform on Azure.

Key Responsibilities :

Design and implement ETL / ELT pipelines using Databricks, Delta Lake, and Apache Spark

Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and well-modeled data

Build and manage data workflows with Databricks Jobs, Notebooks, and Workflows

Optimize Spark jobs for performance, reliability, and cost-efficiency

Maintain and monitor data pipelines, ensuring availability and data quality

Implement CI / CD practices for Databricks notebooks and infrastructure-as-code (e.g., Terraform, Databricks CLI)

Document data pipelines, datasets, and operational processes

Ensure compliance with data governance, privacy, and security policies

Qualifications :

Bachelor's or Master's in Computer Science, Data Engineering, or a related field

5+ years of experience in data engineering or a similar role

Strong hands-on experience with Databricks and Apache Spark (Python, Scala, or SQL)

Proficiency with Delta Lake, Unity Catalog, and data lake architectures

Experience with cloud platforms (Azure, AWS, or GCP), especially data services (e.g., S3, ADLS, BigQuery)

Familiarity with CI / CD pipelines, version control (Git), and job orchestration tools (Airflow, DB Workflows)

Strong understanding of data warehousing concepts, performance tuning, and big data processing

Preferred Skills :

Experience with MLflow, Feature Store, or other machine learning tools in Databricks

Knowledge of data governance tools like Unity Catalog or Purview

Experience integrating BI tools (Power BI, Tableau) with Databricks

Databricks certification(s) (Data Engineer Associate / Professional, Machine Learning, etc.)

Feel free to reach out yashmita@kanakits.com