Data Engineer

A company is looking for a Data Engineer to design and maintain data ingestion pipelines. Key Responsibilities Design, build, and maintain high-volume data ingestion pipelines for processing merchant transactions Implement measures to ensure data quality, reliability, and optimal pipeline performance Share knowledge with team members and foster a culture of agile, data-driven decision-making Required Qualifications Professional experience as a Data Engineer with a focus on high-volume data ingestion Proficiency in Python for building data pipelines Extensive experience with Apache Spark, including Spark SQL and PySpark Strong understanding of the AWS ecosystem, particularly S3 and Lambdas Familiarity with AWS Glue for data processing and transformations