Data Engineer

Purpose of Position :

We are looking for an experienced and highly skilled Data Engineer to join our Data Governance Team. The ideal candidate will have a strong background in data engineering with specific experience in Databricks. You will be responsible for designing building and maintaining scalable data pipelines and architectures to enable advanced analytics and data-driven decision-making across the organization.

Essential Job Functions :

• Design and develop scalable data pipelines using Databricks ensuring efficient data processing and integration from various sources.
• Collaborate with data scientists analysts and business stakeholders to understand data requirements and deliver solutions that meet their needs.
• Implement best practices for data management storage and retrieval focusing on optimizing performance and ensuring data integrity.
• Work with large datasets and perform data cleansing transformation and aggregation to support analytics and reporting functions.
• Monitor and troubleshoot data pipeline performance ensuring high availability and reliability.
• Stay updated with the latest advancements in data engineering and Databricks technologies to continuously improve our data infrastructure.
• Document data processes and workflows ensuring transparency and knowledge sharing across the team.
• Ensure data security and compliance with regulatory requirements and industry best practices.
• Perform other duties as directed.

Knowledge Skills and Experience Requirements :

Working Conditions :

This is a full-time position operating on a hybrid schedule with three days per week in the 275 7th Ave NY office.

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full Time

Experience : years

Vacancy : 1

Yearly Salary Salary : 120000 - 135000