Talent.com
Sr. Data Engineer

Sr. Data Engineer

IT Vision GroupPortland, OR, US
job_description.job_card.30_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
job_description.job_card.job_description

Job Description

Job Description

Title : Sr. Data Engineer

Location :   Oregon

Duration : 12 Months+

Roles & Responsibilities

  • Perform data analysis according to business needs
  • Translate functional business requirements into high-level and low-level technical designs
  • Design and implement distributed data processing pipelines using Apache Spark, Apache Hive, Python, and other tools and languages prevalent in a modern analytics platform
  • Create and schedule workflows using Apache Airflow or similar job orchestration tooling
  • Build utilities, functions, and frameworks to better enable high-volume data processing
  • Define and build data acquisitions and consumption strategies
  • Build and incorporate automated unit tests, participate in integration testing efforts
  • Work with teams to resolve operational & performance issues
  • Work with architecture / engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and followed.

Tech Stack

  • Apache Spark
  • Apache Spark Streaming using Apache Kafka
  • Apache Hive
  • Apache Airflow
  • Python
  • AWS EMR and S3
  • Snowflake
  • Other Tools & Technologies : : PyCharm, Jenkin, Github.
  • Apache Nifi (Optional)
  • Scala (Optional)
  • serp_jobs.job_alerts.create_a_job

    Sr Data Engineer • Portland, OR, US