Senior Data Engineer
Pierce Technology CorpLargo, Florida, United Statesjob_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
- serp_jobs.job_card.full_time
job_description.job_card.job_descriptionStrategy Creation : Collaborate with cross-functional teams to define the data engineering strategy aligned to business objectives, including data modeling that unifies data assets across a range of source systems used to manage the operations of our partnering hospitals. Pipeline Development : Define and execute processes needed to develop, test, deploy, and maintain high quality data pipelines. Oversee the end-to-end development of data pipelines from source data extraction through to production-grade analytical dataset delivery, ensuring data quality and security throughout the pipeline. Performance Optimization : Continuously monitor and optimize data processing performance and efficiency. Identify and address bottlenecks, optimize query performance, and improve overall system stability. Data Governance : Establish and enforce data quality management policies, data access controls, and data privacy standards. Technical Leadership : Stay abreast of the latest developments in engineering tools and best practices. Provide guidance to the team about technical challenges. Documentation : Maintain clear and comprehensive documentation of data pipelines, architecture, and processes to ensure knowledge sharing and team continuity. Third-party Management : Evaluate and manage relationships with third-party vendors and tools, making informed decisions about when to leverage external solutions. 3+ years in data engineering roles in a production environment Advanced proficiency in Python and SQL for data engineering Up-to-date knowledge of and 1+ years of experience using Databricks for Lakehouse management Deep understanding of data modeling, data architecture, and data integration best practices Strong hands-on experience with Apache Spark Familiarity with data governance, security, and privacy principles Comfort using git or equivalent to manage the software development life cycle Exceptional ability to learn and use new software development techniques and tools Ability to manage multiple projects simultaneously High energy, humble team player with “get it done” attitude, seeking collaboration with colleagues
Preferred Qualifications
- Experience with the Azure cloud ecosystem
- Experience developing production-ready, real-time machine learning model serving pipelines
- Comfort developing in the Apache Spark Structured Streaming paradigm
- Experience working in a private equity-backed services company
- Experience deploying machine learning models with MLFlow or equivalent
- Experience developing CI / CD pipelines
serp_jobs.job_alerts.create_a_job
Senior Data Engineer • Largo, Florida, United States