Talent.com
Data Platform Engineer Cloud Ops + Data Ops R1012521
Data Platform Engineer Cloud Ops + Data Ops R1012521YASMESOFT INC • Plano, Texas, USA
Data Platform Engineer Cloud Ops + Data Ops R1012521

Data Platform Engineer Cloud Ops + Data Ops R1012521

YASMESOFT INC • Plano, Texas, USA
job_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
  • serp_jobs.job_card.temporary
job_description.job_card.job_description

Industry Group : Automotive.

Job Title : Data Platform Engineer Cloud Ops Data Ops - R1012521

Location : Plano TX (Local to Dallas area in client office 3days / wk.)

Duration : 12 Months Contract (Potential for extension)

Pay Rate : $70 - $75

Custom Skill Requirements :

  • Data Platform Engineer : Cloud Ops Data Ops
  • PySpark
  • AWS
  • Cloud
  • DevOps : CI / CD
  • Databricks administration

Qualifying Questions :

  • Have you worked on Kubernetes
  • Do you have PySpark
  • Do you have Cloud AWS experience
  • Are you able to work with offshore

    • Job Description :

    As a Data Platform Engineer you will be responsible for the design development and maintenance of our high-scale cloud-based data platform treating data as a strategic product. You will lead the implementation of robust optimized data pipelines using PySpark and the Databricks Unified Analytics Platform-leveraging its full ecosystem for Data Engineering Data Science and ML workflows. You will also establish best-in-class DevOps practices using CI / CD and GitHub Actions to ensure automated deployment and reliability. This role demands expertise in large-scale data processing and a commitment to modern scalable data engineering and AWS cloud infrastructure practices.

    Key Responsibilities :

  • Platform Development : Design build and maintain scalable efficient and reliable ETL / ELT data pipelines to support data ingestion transformation and integration across diverse sources.
  • Big Data Implementation : Serve as the subject matter expert for the Databricks environment developing high-performance data transformation logic primarily using PySpark and Python. This includes utilizing Delta Live Tables (DLT) for declarative pipeline construction and ensuring governance through Unity Catalog.
  • Cloud Infrastructure Management : Configure maintain and secure the underlying AWS cloud infrastructure required to run the Databricks platform including virtual private clouds (VPCs) network endpoints storage (S3) and cross-account access mechanisms.
  • DevOps & Automation (CI / CD) : Own and enforce Continuous Integration / Continuous Deployment (CI / CD) practices for the data platform. Specifically design and implement automated deployment workflows using GitHub Actions and modern infrastructure-as-code concepts to deploy Databricks assets (Notebooks Jobs DLT Pipelines and Repos).
  • Data Quality & Testing : Design and implement automated unit integration and performance testing frameworks to ensure data quality reliability and compliance with architectural standards.
  • Performance Optimization : Optimize data workflows and cluster configurations for performance cost efficiency and scalability across massive datasets.
  • Technical Leadership : Provide technical guidance on data principles patterns and best practices (e.g. Medallion Architecture ACID compliance) to promote team capabilities and maturity. This includes leveraging Databricks SQL for high-performance analytics.
  • Documentation & Review : Draft and review architectural diagrams design documents and interface specifications to ensure clear communication of data solutions and technical requirements.

    • Required Qualifications :

  • Experience : 5 years of professional experience in Data Engineering focusing on building scalable data platforms and production pipelines.
  • Big Data Expertise : Minimum 3 years of hands-on experience developing deploying and optimizing solutions within the Databricks ecosystem.
  • Deep expertise required in :
  • Delta Lake (ACID transactions time travel optimization).
  • Unity Catalog (data governance access control metadata management).
  • Delta Live Tables (DLT) (declarative pipeline development).
  • Databricks Workspaces Repos and Jobs.
  • Databricks SQL for analytics and warehouse operations.
  • AWS Infrastructure & Security : Proven hands-on experience (3 years) with core AWS services and infrastructure components including :
  • Networking : Configuring and securing VPCs VPC Endpoints Subnets and Route Tables for private connectivity.
  • Security & Access : Defining and managing IAM Roles and Policies for secure cross-account access and least privilege access to data.
  • Storage : Deep knowledge of Amazon S3 for data lake implementation and governance.
  • Programming : Expert proficiency (4 years) in Python for data manipulation scripting and pipeline development.
  • Spark & SQL : Deep understanding of distributed computing and extensive experience (3 years) with PySpark and advanced SQL for complex data transformation and querying.
  • DevOps & CI / CD : Proven experience (2 years) designing and implementing CI / CD pipelines including proficiency with GitHub Actions or similar tools (e.g. GitLab CI Jenkins) for automated testing and deployment.
  • Data Concepts : Full understanding of ETL / ELT Data Warehousing and Data Lake concepts.
  • Methodology : Strong grasp of Agile principles (Scrum).
  • Version Control : Proficiency with Git for version control.

    • Preferred Qualifications :

  • AWS Data Ecosystem Experience : Familiarity and experience with AWS cloud-native data services such as AWS Glue Amazon Athena Amazon Redshift Amazon RDS and Amazon DynamoDB.
  • Knowledge of real-time or near-real-time streaming technologies (e.g. Kafka Spark Structured Streaming).
  • Experience in developing feature engineering pipelines for machine learning (ML) consumption.
  • Background in performance tuning and capacity planning for large Spark clusters.

    • Key Skills

    Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

    Employment Type : Full Time

    Experience : years

    Vacancy : 1

    serp_jobs.job_alerts.create_a_job

    Data Engineer Data • Plano, Texas, USA

    Job_description.internal_linking.related_jobs
    SR Data Engineer

    SR Data Engineer

    Alliance Technical Group • Dallas, TX, United States
    serp_jobs.job_card.full_time
    Alliance Technical Group is seeking an experienced Senior Data Engineer to design, build, and optimize the databases and data systems that power our business intelligence and analytics platforms.In...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Senior GCP Data Engineer

    Senior GCP Data Engineer

    Diligent Tec Inc • Plano, Texas, USA
    serp_jobs.job_card.full_time
    Senior Data Engineer - GCP Native Platform.Fully Remote Long-Term Contract C2C / W2 / 1099 Min 12 Years Any Visa.Seeking a Senior Data Engineer with strong hands-on experience across the GCP Na...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    Protagona • Dallas, TX, US
    serp_jobs.job_card.full_time
    serp_jobs.filters_job_card.quick_apply
    As a Data Engineer, you will be part of a talented team of engineers responsible for the deployment and configuration of cloud resources to meet individual client business needs in AWS.Client engag...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30
    Data Engineer

    Data Engineer

    Ascentt • Plano, Texas, United States
    serp_jobs.job_card.full_time
    Describe the role and team the candidate will be joining.Describe the specific responsibilities and job functions of the role. Describe the experience and attributes of the ideal candidate.serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    Tanium • Addison, Texas, United States
    serp_jobs.job_card.full_time
    Tanium is expanding rapidly and is seeking a skilled and motivated Data Engineer with a strong focus on data integrations and ETL pipeline development. This role will play a critical part in designi...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    Cg Infinity • Plano, Texas, United States
    serp_jobs.job_card.full_time
    Data Engineer (Full Time Position) Dallas, TX.We offer solutions that are tailored to the needs of each individual cli...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Palantir Data Engineer (Dallas)

    Palantir Data Engineer (Dallas)

    InfoVision Inc. • Dallas, TX, United States
    serp_jobs.job_card.full_time
    We are seeking a skilled Palantir Data Engineer to join our data and AI team in Dallas, TX.In this role, you will design and deploy scalable data pipelines, ontologies, and machine learning models ...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted
    Cloud Engineer

    Cloud Engineer

    Talent Portus • Plano, Texas, USA
    serp_jobs.job_card.full_time
    Plano TX (Need within 30-40 min distance no flex) -.Banking experience is a required in recent 2-3 years in total.The role involves supporting critical data systems collaborating with engineeri...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted
    Data Platform Engineer

    Data Platform Engineer

    Hitachi Digital Services • Dallas, Texas, United States
    serp_jobs.job_card.full_time
    This job is with Hitachi Digital Services, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new
    Cloud Engineer

    Cloud Engineer

    ABC Co, • Dallas, TX, United States
    serp_jobs.job_card.full_time
    Experience with overseeing Azure and AWS infrastructures.We are seeking a skilled and motivated Cloud Engineer to join our IT team. The ideal candidate will have hands-on experience managing both Az...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted
    Lead Data Engineer Databricks

    Lead Data Engineer Databricks

    Allata • Dallas, Texas, United States
    serp_jobs.filters.remote
    serp_jobs.job_card.full_time
    Allata is a global consulting and technology services firm with offices in the US, India, and Argentina.We help organizations accelerate growth, drive innovation, and solve complex challenges by co...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Snowflake Data Engineer (Dallas)

    Snowflake Data Engineer (Dallas)

    Atos • Dallas, TX, United States
    serp_jobs.job_card.full_time
    Minimum 7+ years of experience in designing, implementing, and supporting Data Warehousing and Business Intelligence solutions on Snowflake. Expertise in Snowflake : Snowpipe, Azure Notifications, St...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_1_day • serp_jobs.job_card.promoted
    Senior AWS Data Engineer

    Senior AWS Data Engineer

    Enable Data Incorporated • Plano, Texas, United States
    serp_jobs.job_card.full_time
    At Enable Data Incorporated, we are excited to welcome a talented Senior AWS Data Engineer to join our dynamic team! With our extensive knowledge in application, data, and cloud engineering service...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_hours • serp_jobs.job_card.promoted • serp_jobs.job_card.new
    Lead Data Engineer - Hybrid (Houston or Dallas, TX)

    Lead Data Engineer - Hybrid (Houston or Dallas, TX)

    Aecom • Dallas, Texas, United States
    serp_jobs.job_card.full_time
    At AECOM, we're delivering a better world.Whether improving your commute, keeping the lights on, providing access to clean water, or transforming skylines, our work helps people and communities thr...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    (US) Fullstack Data Engineer - Databricks

    (US) Fullstack Data Engineer - Databricks

    Codvo.ai • Dallas, Texas, United States
    serp_jobs.job_card.full_time
    We are looking for a highly skilled Full Stack Data Engineer with expertise in Databricks to design, develop, and optimize end-to-end data pipelines, data platforms, and analytics solutions.This ro...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Sr. Data Engineer I

    Sr. Data Engineer I

    Axs • Frisco, Texas, United States
    serp_jobs.job_card.full_time
    AXS connects fans with the artists and teams they love.Each year we sell millions of tickets to thousands of incredible events – from concerts and festivals to sports and theater – at some of the m...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    Spearhead Technology • Plano, Texas, United States
    serp_jobs.job_card.full_time
    Spearhead Technology — where every challenge is an.As a full-lifecycle IT company, we transcend mere delivery; we.Be it planning, analysis, design,. Spearhead Technology, quality isn't a mere.We rec...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_30 • serp_jobs.job_card.promoted
    Data Engineer III AWS Databricks

    Data Engineer III AWS Databricks

    JPMorganChase • Plano, Texas, USA
    serp_jobs.job_card.full_time
    Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team.As a Data Engineer III at JPMorgan Chase within the Enterprise Technology - Core Data Platforms...serp_jobs.internal_linking.show_more
    serp_jobs.last_updated.last_updated_variable_days • serp_jobs.job_card.promoted