Job Description
Senior Data Engineer (Databricks, Spark, Python)
Remote (UK based)
3 - 6 months rolling
£550 - £650 per day
Outside IR35
If the following job requirements and experience match your skills, please ensure you apply promptly.
A Senior Data Engineer with broad knowledge of Databricks is needed to join a leading consultancy renowned for its innovative approach and expertise in delivering state-of-the-art data solutions.
As a proud Databricks Partner, our client is at the forefront of driving transformation and excellence in data strategies for businesses across various sectors. This contract role is an exceptional opportunity for a Senior data engineering professional looking to make a substantial impact while enjoying the flexibility of a fully remote, outside IR35 contract.
As the Senior Data Engineer, you will play a pivotal role in designing and implementing cutting-edge data solutions utilising Databricks technologies.
Key Responsibilities:
- Architect and execute efficient data solutions leveraging Azure Data Factory, Synapse, Databricks, and Azure DevOps.
- Develop, maintain, and optimise data models and ETL pipelines for premier data processing efficiency.
- Design and implement processes to ingest data from various sources into Databricks.
- Use Databricks to integrate, consolidate, and cleanse data to ensure it is accurate and ready for analysis.
- Develop, test, and maintain robust, scalable data pipelines using Databricks.
- Utilise Databricks' support for languages like Python, Scala, SQL, and R to process and transform data.
- Design and implement data models that effectively represent business processes and requirements.
- Optimise data storage in Databricks for efficient querying and analysis, leveraging Delta Lake for ACID transactions and versioned data management.
- Translate complex business requirements into technical specifications in collaboration with various teams.
- Implement and maintain data governance policies and practices, ensuring data quality and consistency.
- Manage data access and security within Databricks, aligning with organizational policies and compliance requirements.
- Engage with clients to support architectural designs and solve technical queries.
- Remain abreast of the latest developments and Databricks technologies, ensuring best practices in data governance and security compliance.
Essential Skills and Qualifications:
- A minimum 5 years' experience in data engineering
- Minimum 2-3 projects delivered with hands-on experience in Databricks
- Expertise in data modeling, ETL development, and data warehousing.
- Familiarity with Databricks medallion architecture
- Experience working with Delta Live Tables
- Familiarity with Unity Catalog
- Strong programming experience using Python
- Experience with distributed computing with Spark
- Streaming experience using technologies such as Kafka
- Strong problem-solving abilities, capable of adapting quickly in a dynamic environment.
- Excellent communication skills, with experience in mentoring and collaborative teamwork.
- Prior experience in client-facing roles is highly advantageous.
- Proficiency in Agile and DevOps methodologies.
- Databricks Data Engineering Professional certification is highly desirable
Contract Highlights:
- Competitive day rate of £550 - £650.
- Fully remote work, offering unparalleled flexibility.
- A 6-month contract outside IR35, providing freedom and independence in your professional engagements.
- An opportunity to engage with innovative projects and leadership roles within a high-growth, value-driven consultancy.
Application Process:
To apply for this opportunity, please click apply or send your CV to CV@primus-connect.com
Primus Connect champions diversity and inclusion, welcoming applications from all qualified individuals. Join us for an enriching contract experience that promises professional growth and a collaborative working environment.