Position Summary
The primary focus of this role will be to support internal organizations in implementing and delivering their strategy and results. The Senior Data Solutions Engineer is responsible for building, managing, and optimizing reusable enterprise data pipelines effectively and in a timely manner through the development lifecycle to be used my internal consumers, such as business / data analysts and data scientists. The engineer would use both technical and analytical skills to understand and solve business problems using available resources and current technology stack, while ensuring data governance and data security compliance. They will mentor junior engineers in finding optimal and efficient solutions for designing, preparing, and storing data for analytical and operational use cases.
Essential Duties and Responsibilities
- Create and maintain technical design documentation
- Participate in requirements gathering, data mapping and designing
- Create, build, and maintain data pipelines from disparate sources that meet functional / non-functional business requirements
- Create, maintain and reuse existing ETL processes, employing a variety of data integration and data preparation tools
- Identify, design, and implement internal process improvements automating manual processes, optimizing data delivery, re-designing pipelines for greater scalability, etc.
- Reverse engineering existing processes to improve and / or optimize.
- Work with stakeholders including Product, Data and Business teams to assist with data-related technical issues and support their data needs
- Create datasets for : (1) operational reports, key performance indicators / metrics, or other insights into current organizational activities, (2) analytics and data science to provide the ability to uncover the answers to major questions that help organizations make objective decisions and / or gain a competitive edge
- Write, debug and implement complex queries involving multiple tables or databases across platform(s)
- Collaborate with the Enterprise Architecture team to ensure alignment on data standards and processes
- Work with data and analytics experts to strive for greater functionality in data systems
- Position requires on-call and off-hours support
Qualifications, Knowledge, and Skills
Bachelor of Science in Computer Science, Information Technology or equivalent5+ years of experience in a data / cloud engineering role5+ years of experience working and creating datasets for a data warehouse3+ years of experience with Databricks preferredClear understanding of data modeling patterns (relational and dimensional) and ability to create model recommendations with limited guidance from a modeler5+ years of experience with a scripting language : Python, Java, Scala, etc.5+ years of experience with ETL development tools, Informatica or Azure Data Factory (ADF) preferred3+ years of cloud experience, Azure preferredExperienced in using best practices in designing, building and managing data pipelines that require data transformations as well as metadata and workload managementExperienced in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional and new data integration technologies (such as ETL, ELT, data replication, change data captures, message-oriented data movement, API design, stream data integration and data virtualization)Experienced in performing root cause analysis on internal and external data and processes to identify issues and opportunities for improvementExpert level knowledge with programming languages including SQL, PL / SQL, T-SQLExpert level knowledge with relational SQL databases such as Oracle and SQL ServerExperience with NoSQL databases are a plusExperience supporting and working with cross-functional teams in a dynamic environmentProven ability to collaborate with technical peersCapable of working independently as well as part of a teamStrive to provide orientation and direction to junior engineers requiring their expertiseExperienced with continuous integration and continuous deployment practicesAbility to approach complex problems with creativity and display analytical and problem-solving skillsDisplay curiosity in understanding the data for the specific area of responsibilityKnowledge and experience working with agile methodologies and tools (such as Jira) a plus