Data Platform Engineer Cloud Ops + Data Ops R1012521YASMESOFT INC • Plano, Texas, USA

Data Platform Engineer Cloud Ops + Data Ops R1012521

YASMESOFT INC • Plano, Texas, USA

job_description.job_card.variable_days_ago

serp_jobs.job_preview.job_type

serp_jobs.job_card.full_time

serp_jobs.job_card.temporary

job_description.job_card.job_description

Industry Group : Automotive.

Job Title : Data Platform Engineer Cloud Ops Data Ops - R1012521

Location : Plano TX (Local to Dallas area in client office 3days / wk.)

Duration : 12 Months Contract (Potential for extension)

Pay Rate : $70 - $75

Custom Skill Requirements :

Data Platform Engineer : Cloud Ops Data Ops
PySpark
AWS
Cloud
DevOps : CI / CD
Databricks administration

Qualifying Questions :

Have you worked on Kubernetes

Do you have PySpark

Do you have Cloud AWS experience

Are you able to work with offshore

Job Description :

As a Data Platform Engineer you will be responsible for the design development and maintenance of our high-scale cloud-based data platform treating data as a strategic product. You will lead the implementation of robust optimized data pipelines using PySpark and the Databricks Unified Analytics Platform-leveraging its full ecosystem for Data Engineering Data Science and ML workflows. You will also establish best-in-class DevOps practices using CI / CD and GitHub Actions to ensure automated deployment and reliability. This role demands expertise in large-scale data processing and a commitment to modern scalable data engineering and AWS cloud infrastructure practices.

Key Responsibilities :

Platform Development : Design build and maintain scalable efficient and reliable ETL / ELT data pipelines to support data ingestion transformation and integration across diverse sources.

Big Data Implementation : Serve as the subject matter expert for the Databricks environment developing high-performance data transformation logic primarily using PySpark and Python. This includes utilizing Delta Live Tables (DLT) for declarative pipeline construction and ensuring governance through Unity Catalog.

Cloud Infrastructure Management : Configure maintain and secure the underlying AWS cloud infrastructure required to run the Databricks platform including virtual private clouds (VPCs) network endpoints storage (S3) and cross-account access mechanisms.

DevOps & Automation (CI / CD) : Own and enforce Continuous Integration / Continuous Deployment (CI / CD) practices for the data platform. Specifically design and implement automated deployment workflows using GitHub Actions and modern infrastructure-as-code concepts to deploy Databricks assets (Notebooks Jobs DLT Pipelines and Repos).

Data Quality & Testing : Design and implement automated unit integration and performance testing frameworks to ensure data quality reliability and compliance with architectural standards.

Performance Optimization : Optimize data workflows and cluster configurations for performance cost efficiency and scalability across massive datasets.

Technical Leadership : Provide technical guidance on data principles patterns and best practices (e.g. Medallion Architecture ACID compliance) to promote team capabilities and maturity. This includes leveraging Databricks SQL for high-performance analytics.

Documentation & Review : Draft and review architectural diagrams design documents and interface specifications to ensure clear communication of data solutions and technical requirements.

Required Qualifications :

Experience : 5 years of professional experience in Data Engineering focusing on building scalable data platforms and production pipelines.

Big Data Expertise : Minimum 3 years of hands-on experience developing deploying and optimizing solutions within the Databricks ecosystem.

Deep expertise required in :

Delta Lake (ACID transactions time travel optimization).

Unity Catalog (data governance access control metadata management).

Delta Live Tables (DLT) (declarative pipeline development).

Databricks Workspaces Repos and Jobs.

Databricks SQL for analytics and warehouse operations.

AWS Infrastructure & Security : Proven hands-on experience (3 years) with core AWS services and infrastructure components including :

Networking : Configuring and securing VPCs VPC Endpoints Subnets and Route Tables for private connectivity.

Security & Access : Defining and managing IAM Roles and Policies for secure cross-account access and least privilege access to data.

Storage : Deep knowledge of Amazon S3 for data lake implementation and governance.

Programming : Expert proficiency (4 years) in Python for data manipulation scripting and pipeline development.

Spark & SQL : Deep understanding of distributed computing and extensive experience (3 years) with PySpark and advanced SQL for complex data transformation and querying.

DevOps & CI / CD : Proven experience (2 years) designing and implementing CI / CD pipelines including proficiency with GitHub Actions or similar tools (e.g. GitLab CI Jenkins) for automated testing and deployment.

Data Concepts : Full understanding of ETL / ELT Data Warehousing and Data Lake concepts.

Methodology : Strong grasp of Agile principles (Scrum).

Version Control : Proficiency with Git for version control.

Preferred Qualifications :

AWS Data Ecosystem Experience : Familiarity and experience with AWS cloud-native data services such as AWS Glue Amazon Athena Amazon Redshift Amazon RDS and Amazon DynamoDB.

Knowledge of real-time or near-real-time streaming technologies (e.g. Kafka Spark Structured Streaming).

Experience in developing feature engineering pipelines for machine learning (ML) consumption.

Background in performance tuning and capacity planning for large Spark clusters.

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full Time

Experience : years

Vacancy : 1

serp_jobs.job_alerts.create_a_job

Data Engineer Data • Plano, Texas, USA

Job_description.internal_linking.related_jobs

SR Data Engineer

Alliance Technical Group • Dallas, TX, United States

serp_jobs.job_card.full_time

Alliance Technical Group is seeking an experienced Senior Data Engineer to design, build, and optimize the databases and data systems that power our business intelligence and analytics platforms.In...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Cloud Infrastructure Engineer - AWS & ML Tooling

Zelis Healthcare, LLC • Plano, TX, United States

serp_jobs.job_card.full_time

Zelis is modernizing the healthcare financial experience for all by providing a connected platform that bridges the gaps and aligns interests across payers, providers, and healthcare consumers.This...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted

Data Engineer

Protagona • Dallas, TX, US

serp_jobs.job_card.full_time

serp_jobs.filters_job_card.quick_apply

As a Data Engineer, you will be part of a talented team of engineers responsible for the deployment and configuration of cloud resources to meet individual client business needs in AWS.Client engag...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_30

GCP Data Engineer (Richardson)

Infosys • Richardson, TX, US

serp_jobs.job_card.part_time

Infosys is seeking a Google Cloud (GCP) data engineer with experience in Github and python.In this role, you will enable digital transformation for our clients in a global delivery model, research ...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Sr. Data Engineer (Dallas)

Trinity Industries, Inc. • Dallas, TX, US

serp_jobs.job_card.part_time

The successful candidate will work with the Trinity Rail teams to develop and maintain data pipelines in Azure utilizing Databricks, Python and SQL. Join our team today and be a part of.Delivering G...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Lead Data Engineer - Capital One Software (Remote)

Capital One • Plano, TX, US

serp_jobs.filters.remote

serp_jobs.job_card.full_time +1

Lead Data Engineer - Capital One Software (Remote).Capital One Software is seeking a Lead Data Engineer who is passionate about marrying innovation with emerging technologies.In 2022, we publicly a...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new

Lead Data Engineer

Capital One • Plano, TX, US

serp_jobs.job_card.full_time +1

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,. At Capital One, you'll be part of a big group of makers, ...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new

Data Engineer (Dallas)

BeaconFire Inc. • Dallas, TX, US

serp_jobs.job_card.part_time

BeaconFire is based in Central NJ, specializing in Software Development, Web Development, and Business Intelligence; looking for candidates who are good communicators and self-motivated.You will pl...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Snowflake Data Engineer (Dallas)

Atos • Dallas, TX, US

serp_jobs.job_card.part_time

Minimum 7+ years of experience in designing, implementing, and supporting Data Warehousing and Business Intelligence solutions on Snowflake. Expertise in Snowflake : Snowpipe, Azure Notifications, St...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Cloud Engineer (Plano)

Optomi • Plano, TX, US

serp_jobs.job_card.full_time +1

Optomi, in partnership with one of our premier clients, is seeking a Senior Cloud Engineer to lead the design, automation, and security of large-scale AWS networking environments.This role blends h...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Data Engineer-2

eTeam Inc • Plano, Texas, United States

serp_jobs.job_card.full_time

serp_jobs.filters_job_card.quick_apply

Extensive experience in designing, configuring, deploying, managing and automating AWS Core Services like S3, IAM, EC2, Route53, SNS, SQS, ELB, CloudWatch, Lambda and VPC.Experience in automating c...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_30

Data Platform Engineer

Hitachi Digital Services • Dallas, Texas, United States

serp_jobs.job_card.full_time

This job is with Hitachi Digital Services, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted

Cloud Engineer

ABC Co, • Dallas, TX, United States

serp_jobs.job_card.full_time

Experience with overseeing Azure and AWS infrastructures.We are seeking a skilled and motivated Cloud Engineer to join our IT team. The ideal candidate will have hands-on experience managing both Az...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Lead Data Engineer – Big Data (Cloud & Palantir)

DRC Systems • Dallas, TX, United States

serp_jobs.job_card.temporary

serp_jobs.filters_job_card.quick_apply

Only W2 (No H1B, No OPT, No C2C, No 1099) Duration : 6 Months Contract with possible extension Title : L...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.new

Palantir Data Engineer

InfoVision Inc. • Dallas, TX, United States

serp_jobs.job_card.full_time

Palantir Data Engineer – Dallas, TX Hybrid.We are seeking a skilled Palantir Data Engineer to join our data and AI team in Dallas, TX. In this role, you will design and deploy scalable data pipeline...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Azure Data Engineer (Local to TX-Onsite Interview) (Dallas)

TekValue IT Solutions • Dallas, TX, US

serp_jobs.job_card.part_time

Cloud data engineering experience on azure.Deep understanding of azure services like data factory, data bricks, and machine learning. Strong proficiency in SQL, python and pyspark.Proven ability to ...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted

Data Engineer (Dallas)

EXL • Dallas, TX, US

serp_jobs.job_card.part_time

Location : Location : Hybrid (3 days onsite, 2 days work from Home) Pittsburgh / Cleveland / Dallas / Phoenix.Design, manage, and optimize HDFS directories, tables, and partitioning strategies.Implement ...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted

Lead Data Engineer - AWS

Tiger Analytics • Dallas, TX, US

serp_jobs.filters.remote

serp_jobs.job_card.full_time

serp_jobs.filters_job_card.quick_apply

Tiger Analytics is a fast-growing advanced analytics consulting firm.Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for multiple ...serp_jobs.internal_linking.show_more

serp_jobs.last_updated.last_updated_30