Lead Consultant - Azure data pipelines Engineer

Career Opportunities > Lead Consultant - Azure data pipelines Engineer

Contract: Full-Time

Location: United Kingdom, GB

Apply now
Use my indeed resume
Apply through Linkedin

Job Description

Positon: Lead Consultant - Azure data pipelines Engineer


Job Title: Lead Consultant - Azure data pipelines Engineer

Location: Flexible

Salary: Competitive, plus benefits

Contract type: Permanent, Full-time

Responsibilities:

  • Construct and manage a sophisticated data platform on Azure, tailored to manage extensive and intricate data sets.
  • Develop solutions for Data Lakes and Data Warehouses.
  • Employ Data Vault and Dimensional modelling techniques for data model construction.
  • Establish both batch and streaming data pipelines that are automated, efficient, and
  • Engage collaboratively with Data Science teams to fulfill their data-related needs.
  • Coordinate with teams focused on Reporting and Data Visualization to deliver comprehensive data visualization solutions.
  • Devise strategies for managing Personal and Sensitive data securely.
  • Adopt Agile methodologies and DevOps practices in daily work.

Experience:

  • Proficiency with Azure Data Lake Storage (ADLS Gen2).
  • Experience with Synapse Dedicated SQL Pool, including table creation, stored procedures, and more.
  • Familiarity with Azure Data Factory, including the creation and management of metadata-driven pipelines.
  • Skilled in using Azure Databricks for both batch and streaming data processing.
  • Understanding of various Data Lake storage formats such as Parquet and Delta Lake.
  • Capability to process complex data types, including JSON and XML.
  • Experience with Azure Event Hubs and Event Grids.
  • Proficiency in Azure Functions and Cognitive Search.
  • Programming skills in a structured language, such as Python or C#.

Desirable:

  • Knowledge of implementing CI/CD pipelines.
  • Experience in feature engineering and ML Ops.
  • Capability to develop reporting solutions using Power BI.
  • Familiarity with tools within the Synapse Analytics Workspace.
  • Understanding of SQL and Data Streaming technologies.


Agile Solutions is focused on deriving true value from its customers data. We help them to modernize, manage and monetize it.

We provide advice, support and delivery services across various industry sectors covering a multitude of areas, ranging from Data Strategy, Governance and Security to Data Platform Modernization, Cloud and Customer Intelligence.

We do everything with a view to creating tangible business benefits through intelligently introducing new data-related technology, delivering with agility and proving real customer value that our Enterprise and SME clients expect and deserve.