Data Engineer - HadoopGTN Technical Staffing • New York, NY, United States
Data Engineer - Hadoop
GTN Technical Staffing • New York, NY, United States
job_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.permanent
job_description.job_card.job_descriptionStrong hands-on experience with Hadoop administration , ideally within Cloudera environments. Proficiency with Python , particularly for automation and data workflows. Experience with Apache Spark (supporting jobs, tuning performance, understanding resource usage). Solid understanding of Linux / Unix systems administration , shell scripting, permissions, networking basics, and OS-level troubleshooting. Experience supporting distributed compute environments or large-scale data platforms. Familiarity with DevOps collaboration (patching, upgrades, deployments, incident response, etc.). Excellent communication skills with the ability to work directly with data scientists and technical end users. Ability to coordinate with multiple technical teams (DevOps, Infrastructure, ML Ops). Strong troubleshooting and problem-solving capabilities. Ability to manage multiple priorities in a fast-moving environment. Experience with ML Ops environments or supporting machine learning workflows. Experience with cluster performance optimization and capacity planning. Background in distributed systems or data engineering.
Data Engineer – Hadoop Administrator
HIGHLIGHTS
Location : Chicago, IL / New York, NY / Phoenix, AZ (Hybrid)
Position Type : Direct Hire
Compensation : BOE
Overview
We are seeking a Data Engineer to support Newton , our Data Science R&D compute cluster. This role functions as a Hadoop Administrator embedded within the ML Ops organization, providing hands-on operational support for the platform while partnering directly with data scientists, DevOps, and infrastructure teams. This individual will ensure the health, stability, performance, and usability of the Newton cluster, acting as the primary point of contact for platform support, troubleshooting, and environment optimization.
This is a highly collaborative and technical role with room for long-term career progression.
Key Responsibilities
- Serve as the primary administrator for the Newton Hadoop / Cloudera cluster.
- Provide direct support to data scientists experiencing issues with jobs, workloads, dependencies, cluster resources, or environment performance.
- Troubleshoot complex Hadoop, Spark, Python, and OS-level issues; drive root cause analysis and implement permanent fixes.
- Coordinate closely with DevOps to ensure patching, upgrades, infrastructure changes, and system reliability activities are completed on schedule.
- Monitor cluster performance, capacity, and resource utilization; tune and optimize for efficiency and cost.
- Manage Hadoop and Cloudera configurations, services, security, policies, and operational health.
- Implement automation and scripting to improve operational workflows and reduce manual intervention.
- Validate vendor patches, updates, and upgrades and coordinate deployments with DevOps and infrastructure teams.
- Maintain documentation, operational runbooks, troubleshooting guides, and environment standards.
- Serve as a liaison between Data Science, ML Ops, Infrastructure, and DevOps teams to ensure seamless platform operations.
- Support the organization’s commitment to protecting the integrity, availability, and confidentiality of systems and data.
Required Technical Skills
Required Soft Skills & Competencies
Preferred Skills (Nice to Have)
serp_jobs.job_alerts.create_a_job
Data Engineer • New York, NY, United States
Job_description.internal_linking.related_jobs
Resonance is transforming the fashion industry by building a more sustainable and valuable ecosystem for designers, brands, manufacturers, consumers, and the planet.
Our AI-powered operating system,...serp_jobs.internal_linking.show_more
Water Street, 7th Floor, New York, NY 10004 .New Yorkers by uniting communities through care.We believe that Health care is a right, not a privilege.
If you have compassion and a collaborative sp...serp_jobs.internal_linking.show_more
Enterprise Data Management – Data Cloud, Senior Developer I.The Data Engineering team oversees the organization's central data infrastructure, which powers enterprise-wide data products and advance...serp_jobs.internal_linking.show_more
DL Software produces Godel, a financial information and trading terminal.This is a full-time, on-site role based in New York, NY, for a Data Engineer.
The Data Engineer will design, build, and maint...serp_jobs.internal_linking.show_more
Location : Jersey City , NJ(Hybrid).We are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-base...serp_jobs.internal_linking.show_more
Haptiq is a leader in AI-powered enterprise operations, delivering digital solutions and consulting services that drive value and transform businesses.
We specialize in using advanced technology to ...serp_jobs.internal_linking.show_more
Chubb is looking for a Azure Platform Engineering Professional with a Bachelor’s Degree to join our Global Data Platform team.
This is a permanent full-time position and a compelling opportunity to ...serp_jobs.internal_linking.show_more
KINESSO is the technology-driven performance marketing agency that sits at the very heart of IPG Mediabrands, providing actionable growth for both our agency partners and clients.We turn 'action' i...serp_jobs.internal_linking.show_more
We are looking for a highly skilled and technical Data Architect to lead the design and implementation of our data ecosystem.
In this role, you will be responsible for the end-to-end architecture of...serp_jobs.internal_linking.show_more
Data Engineer (Contract) Sports Tech & Entertainment | NYC (Hybrid).Negotiable, depends on experience).Were seeking a Data Engineer to support our Customer Experience team at a leading sports tech...serp_jobs.internal_linking.show_more
A leading AI data company is seeking a Forward Deployed Engineer to drive impactful solutions in the advancement of AI.You will collaborate with technical customers to deliver high-quality data sol...serp_jobs.internal_linking.show_more
Senior Lead Data Engineer (Snowflake, Databricks, Apache Iceberg, Spark + SQL workload optimizations).Do you love building and pioneering in the technology space? Do you enjoy solving complex busin...serp_jobs.internal_linking.show_more
This role combines technical leadership with hands-on development, focusing on data architecture, pipeline design, and governance for high-quality, reliable financial data.You will be acting as a l...serp_jobs.internal_linking.show_more
Job Title : Azure Data Engineer.Location : Weehawken, NJ ( 5 days WFO).Expert level skills writing and optimizing complex SQL.
Experience with complex data modelling, ETL design, and using large datab...serp_jobs.internal_linking.show_more
Be part of a global consulting powerhouse, partnering with clients on their most critical strategic transformations.Energetic, solution-driven experts who focus as much on people as on performance ...serp_jobs.internal_linking.show_more
Location : Jersey City , NJ(Hybrid).We are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-base...serp_jobs.internal_linking.show_more
Job title : Lead Software Engineer.Duration : Fulltime / Contract to Hire.Location : New York, NY (Hybrid).The successful candidate will be a key member of the HR Technology team, responsible for develo...serp_jobs.internal_linking.show_more
A leading AI research company is seeking an Analytics Engineer to join their Data Science & Analytics team in New York.The ideal candidate will have at least 5 years of relevant experience and stro...serp_jobs.internal_linking.show_more
Data and Analytics Engineer
Resonance • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
Data Engineer
MetroPlus Health Plan • New York, NY, United States
serp_jobs.job_card.full_time +1
serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
Cloud Data Engineer
Gotham Technology Group • New York, NY, United States
serp_jobs.job_card.permanent
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Data Engineer
DL Software Inc. • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Lead Data Engineer (Jersey City)
EXL • Jersey City, NJ, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Data Engineer
Haptiq • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Cloud Data Platform Engineer
Chubb • Jersey City, New Jersey, United States
serp_jobs.job_card.full_time +1
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Senior Data Engineer
IPG Mediabrands • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
Data Architect
Macmillan Learning • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
Data Engineer (New York)
Brooksource • New York, NY, US
serp_jobs.job_card.part_time +1
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
GenAI Forward-Deployed Engineer : Data Infra Impact
Scale AI • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted
Senior Lead Data Engineer (Snowflake, Databricks, Apache Iceberg, Spark + SQL workload optimizations
Capital One • New York, NY, US
serp_jobs.job_card.full_time +1
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Lead Data Engineer
Northbound Executive Search • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Azure Data Engineer (Weehawken)
Programmers.io • Weehawken, NJ, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Data Engineer Manager (New York)
Wavestone • New York, NY, US
serp_jobs.job_card.full_time +1
serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
Lead Data Engineer
EXL • Jersey City, NJ, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Lead Data Engineer
APN Consulting, Inc. • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
Analytics Data Engineer : Scale Data Pipelines & Dashboards
anthropic • New York, NY, United States
serp_jobs.job_card.full_time
serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted