Sr Data Engineer

As a Sr. Data Engineer, you will create and maintain optimal data pipeline architecture, Infrastucture automation(IAC) and also will assemble large, complex data sets that meet functional / non-functional business requirements across Data Solutions.

You will also identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability etc.

Implement foundational and advanced analytics methodologies to address key metrics to improve customer experience, network performance, quality of service, profitability of products, and personalized offerings

Delivering technical functionality developed against the Global Lockton Cloud Data Platform's with a Product Mindset.

Use data, analysis, and automation to generate value within the company through advanced data modeling and other advanced analytics principles

Building the infrastructure required for optimal ETL / ELT of data from a wide variety of data sources using Cloud Technologies and OnPrem.

Responsible for helping an organization deliver high data availability and quality throughout the entire data life cycle from ingestion to end products : dashboards, machine learning models, and production datasets.

The Successful Applicant

Technical Knowledge, Key skills and experience required for the successful candidate :

Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, ADLS-Delta lake, Azure functions, Logic aps, Synapse, Azure Logic Apps, Azure Storage, Azure Search, Purview, Azure SQL server, Scala / Spark / SQL / PYTHON etc.

Azure Database development experience with Infra as a Code, Azure CI / CD, Azure Classic Pipelines, Azure YAML, ARM / Terraforms / Ansible, AKS Maintenance, Azure ML, Pipeline integrations, Azure PaaS and IaaS, Azure Infra Experience (Spinning VM's, Databases and Cosmos)

Expertise in Data warehousing - data modelling, relational and Dimension modelling, Data quality, data catalog and Data Governance

Design and build highly scalable data pipelines using new generation tools and technologies like Spark, Kafka to induct data from various systems

Experience with data security and data access controls and design across digital transformation / cloud migration programs.

Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modelling)

Ensure proper requirement traceability and Quality delivery during development and integration testing

A strong background in the Microsoft SQL Server stack (including SSIS, SSRS, Power BI, and SSAS).

Ability to build processes supporting data transformation, data structures, metadata, dependency, and workload management.

Strong analytic skills related to working with structured and unstructured datasets

Should support the Production live application by maintaining % Data accuracy and data availability in all 24

• Minimum of 7+ years of increasingly responsible experience in Data Platforms, Data Engineering, Software Engineering at high impactful and changing environments
• 4+ years of working experience in Azure or cloud platforms as Solution / Data Architect / Data Engineer
• 4+ years of hands-on experience with one or more cloud technologies such as Azure, AWS, Google.
• Experience in collaborating with a high-profile team of Data Engineers, data scientists, Business Analytics and Data Analysts
• Knowledge of Software development agile methodologies and worked in Scrum teams
• Excellent communication & presentation skills with a track record of engaging with senior business leaders
• Experience in Finance / Insurance would be a distinct advantage
• Should have worked in a dynamic and fast-changing environment
• High affinity with AI-powered insight tools and engines and application of real-time solutions to analytics problems
• Experience with GEN AI and LLM's will be added advantage
• Exposure to COGS / KTLO, Operations and Command Center activity

Soft Skills Required

Academics

Career progression :