AWS Data Engineer (SC Clearance) AWS Data Engineer (SC) - Contract - Hybrid working (London) - £competitive Synergize Consulting are now hiring for a Data Engineer to work at a leading consultancy client on behalf of a large public sector body, on a contract basis. The successful candidate will be working within a high-profile programme of work, and will work alongside Project Managers and Consultants to support the delivery of end-to-end Data solutions. Responsibilities include: - Analysis of existing data products and the scoping of future requirements - Providing expert support to the delivery of end-to-end data products - Review of Data pipelines - Engage with customers and stakeholders within the client to ascertain needs and requirements - Work with other SME's to collaboratively scope new requirements - Apply data knowledge and business knowledge to represent data plans and pipelines Experience required: - Strong background in data engineering at a senior level within a complex organisation - Excellent AWS experience including S3, Lambda and Athena - Python programming experience would be preferred - Tableau experience would be highly desirable - DevOps experience including Terraform and Docker would be beneficial - Strong consultative experience, and the ability to scope product designs and provide support to delivery - Excellent stakeholder engagement skills - Strong communication skills, both written and verbal - Due to the nature of the client, it would be an advantage for candidates to hold, or be eligible and willing to undergo, a certain level of security clearance (SC). If you are looking to take the next exciting step in your IT career, please send in your CV for immediate consideration.

Methods Analytics exists to improve society by helping people make better decisions with data. Combining passionate people, sector-specific insight and technical excellence to provide our customers an end-to-end data service. We use a collaborative, creative and user centric approach data to do good and solve difficult problems. We ensure that our outputs are transparent, robust and transformative.We value discussion and debate as part of our approach. We will question assumptions, ambition and process – but do so with respect and humility. We relish difficult problems, and overcome them with innovation, creativity and technical freedom to help us design optimum solutions. Ethics, privacy and quality are at the heart of our work and we will not sacrifice these for outcomes. We treat data with respect and use it only for the right purpose. Our people are positive, dedicated and relentless. Data is a vast topic, but we strive for interactions that are engaging, informative and fun in equal measure.Methods Analytics was acquired by the Alten Group in early 2022. Purpose of the Role:Methods Analytics (MA) is recruiting for a Data Engineer to join our team within the Public Sector Business unit on a permanent basis. Location:This role will be mainly remote but require flexibility to travel to client sites, and our offices based in London, Sheffield, and Bristol. Requirements:<ul><li>Work closely with cross-functional teams, translating complex technical concepts into clear, accessible language for non-technical audiences and aligning data solutions with business needs.</li><li>Collaborate with a dynamic delivery team on innovative projects, transforming raw data into powerful insights that shape strategic decisions and drive business transformation.</li><li>Utilise platforms and tools such as Microsoft Fabric, Azure Data Factory, Azure Synapse, Databricks, and PowerBI to build robust, scalable, and future-proof end-to-end data solutions.</li><li>Design and implement efficient ETL and ELT pipelines, ensuring seamless integration and transformation of data from various sources to deliver clean, reliable data.</li><li>Develop and maintain sophisticated data models, employing dimensional modelling techniques to support comprehensive data analysis and reporting.</li><li>Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, Unity Catalog, and Apache Atlas to maintain data integrity and trust.</li><li>Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data queries for performance and cost-efficiency.</li><li>Develop intuitive and visually compelling Power BI dashboards that provide actionable insights to stakeholders across the organisation.</li><li>Monitor and tune solution performance, identifying opportunities for optimisation to enhance the reliability, speed, and functionality of data systems.</li><li>Stay ahead of industry trends and advancements, continuously enhancing your skills and incorporating the latest Data Engineering tools, languages, and methodologies into your work.</li></ul> Essential Skills and Experience:<ul><li>Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease.</li><li>Understanding of Data Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture.</li><li>Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets.</li><li>Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ETL and ELT pipelines, mostly in Azure, utilising Azure Data Factory and Spark-based solutions to ensure efficient data flow and transformation.</li><li>Efficiency in Query Writing: You can craft and optimise queries to be both cost-effective and high-performing, ensuring fast and reliable data retrieval.</li><li>Experience in Power BI Dashboard Development: You possess experience in creating insightful and interactive Power BI dashboards that drive business decisions.</li><li>Proficiency in Dimensional Modelling: You are adept at applying dimensional modelling techniques, creating efficient and effective data models tailored to business needs.</li><li>CI/CD Mindset: You naturally work within Continuous Integration and Continuous Deployment (CI/CD) environments, ensuring automated builds, deployments, and unit testing are integral parts of your development workflow.</li><li>Business Requirements Translation: You have a knack for understanding business requirements and translating them into precise technical specifications that guide data solutions.</li><li>Strong Communication Skills: Ability to effectively translate complex technical topics into clear, accessible language for non-technical audiences</li><li>Continuous Learning and Development: Commitment to continuous learning and professional development, staying up to date with the latest industry trends, tools, and technologies.</li></ul> Your Impact:<ul><li>Enable business leaders to make informed decisions with confidence by providing them with timely, accurate, and actionable data insights.</li><li>Be at the forefront of data innovation, driving the adoption and understanding of modern tooling, architectures, and platforms.</li><li>Deliver seamless and intuitive data solutions that enhance the user experience, from real-time streaming data services to interactive dashboards.</li><li>Play a key role in cultivating a data-driven culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice.</li></ul> Desirable Skills and Experience:<ul><li>Exposure to Microsoft Fabric: Familiarity with Microsoft Fabric and its capabilities would be a significant advantage.</li><li>Experience with High-Performance Data Systems: Handling large-scale data systems with high performance and low latency, such as managing 1 billion+ records or terabyte-sized databases.</li><li>Knowledge of Delta Tables or Apache Iceberg: Understanding and experience with Delta Tables or Apache Iceberg for managing large-scale data lakes efficiently.</li><li>Knowledge of Data Governance Tools: Experience with data governance tools like Azure Purview, Unity Catalog, or Apache Atlas to ensure data integrity and compliance.</li><li>Exposure to Streaming/Event-Based Technologies: Experience with technologies such as Kafka, Azure Event Hub, and Spark Streaming for real-time data processing and event-driven architectures.</li><li>Understanding of SOLID Principles: Familiarity with the SOLID principles of object-oriented programming.</li><li>Understanding of Agile Development Methodologies: Familiarity with iterative and agile development methodologies such as SCRUM, contributing to a flexible and responsive development environment.</li><li>Familiarity with Recent Innovations: Knowledge of recent innovations such as GenAI, RAG, and Microsoft Copilot, as well as certifications with leading cloud providers and in areas of data science, AI, and ML.</li><li>Experience with Data for Data Science/AI/ML: Experience working with data tailored for data science, AI, and ML applications,</li><li>Experience with Public Sector Clients: Experience working with public sector clients and understanding their specific needs and requirements.</li></ul> This role will require you to have or be willing to go through Security Clearance. As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard; details of the evidence required to apply may be found on the government website Gov.UK. If you are unable to meet this and any associated criteria, then your employment may be delayed, or rejected . Details of this will be discussed with you at interview Benefits:Methods Analytics is passionate about its people; we want our colleagues to develop the things they are good at and enjoy.By joining us you can expect<ul><li>Autonomy to develop and grow your skills and experience</li><li>Be part of exciting project work that is making a difference in society</li><li>Strong, inspiring and thought-provoking leadership</li><li>A supportive and collaborative environment</li></ul>As well as this, we offer:<ul><li>Development access to LinkedIn Learning, a management development programme and training</li><li>Wellness 24/7 Confidential employee assistance programme</li><li>Social – Breakfast Tuesdays, Thirsty Thursdays and Pizza on the last Thursday of each month as well as commitment to charitable causes</li><li>Time off 25 days a year</li><li>Pension Salary Exchange Scheme with 4% employer contribution and 5% employee contribution</li><li>Discretionary Company Bonus based on company and individual performance</li><li>Life Assurance of 4 times base salary</li><li>Private Medical Insurance which is non-contributory (spouse and dependants included)</li><li>Worldwide Travel Insurance which is non-contributory (spouse and dependants included)</li></ul>

About Birlasoft Ltd.Birlasoft combines the power of domain, enterprise, and digital technologies to reimagine business processes for customers and their ecosystem. Its consultative and design thinking approach makes societies more productive by helping customers run businesses. As part of the multibillion-dollar diversified <a>CK Birla Group</a>, Birlasoft with its 12,500+ professionals, is committed to continuing the Group’s 161-year heritage of building sustainable communities. Requirements<ol><li>Good Knowledge of Microsoft T-SQL query writing.</li><li>Good Knowledge of Document writing</li><li>Good Knowledge of MSBI (SSIS) ETL tool.</li><li>Minimum experience should be 7 years.</li><li>Azure Data Factory (Must)</li><li>Azure Data Lake (Must)</li><li>Azure Data Bricks (Must) </li><li>Must also have the knowledge of Spark and Python</li></ol>

Lead Data Engineer (x2) – London – Permanent – Hybrid This role requires hybrid working based in Moorgate, London The salary for this role ranges between £65,000 – £74,000 Lead Data Engineer required for a leading client based in Moorgate, London. Our customer is seeking a well experienced Lead Data Engineer to come on board to spearhead agile data engineering initiatives, delivering data and insights products. This role involves understanding the business and IT strategy to align technical solutions with desired outcomes, developing APIs and integrations for data transfer between systems, and providing aggregated data solutions. Key Responsibilities: <ul><li>Ability to lead complex data engineering projects</li><li>Design, code, test, and document data applications</li><li>Develop and manage scalable APIs, machine learning endpoints, data integration pipelines, and data storage solutions.</li><li>Shape the information strategy and platform architecture</li><li>Introduce new data engineering tools, frameworks, standards, and processes</li><li>Ensure the application of agreed standards and tools</li><li>Ensure technical solutions support the accessibility, retrievability, security, quality, retention, and ethical handling of data.</li><li>Experience with strategies and design approaches for APIs and data pipelines.</li><li>Experience in designing and implementing data-driven tools and solutions.</li><li>Proficiency in developing scalable APIs in Python using frameworks like FastAPI or Flask.</li><li>Proficiency in developing scalable data pipelines in Spark/Python using tools such as Airflow.</li><li>Experience with SQL, Python, and other scripting/programming languages</li><li>Experience with diverse data infrastructure and systems, such as PostgreSQL, SQL Server, Azure Data Lakes, Databricks, and Kubernetes.</li><li>Strong stakeholder management and communication skills at varying levels</li></ul> Interested? Please submit your updated CV to lewis.rushton@crimson.co.uk for immediate consideration. Not interested? Do you know someone who might be a perfect fit for this role? Refer a friend and earn £250 worth of vouchers! Crimson is acting as an employment agency regarding this vacancy

Get stuck in immediately to a database migration using Snowflake. This company work across various energy and commodities markets across the world. We appreciate that not everyone wants to work within Oil and Gas trading, however, part of the role of the data team is to look at more sustainable options for trading all kinds of commodities products. You’re going to be getting involved with a number of newly launched data projects, with your initial project being to work on this migration. You’ll face off with the business (Heads of Desk, Traders, Analysts), understanding what they need, discussing solutions with the Data Science team, then building out the best solution possible, whether it be with an off the shelf product, or building it completely from scratch using primarily Python and SQL. The data team has grown over the past 12-18 months, with data engineering still being built out in London. There’s a strong opportunity to take on leadership responsibilities, so if management is in your sights and ambitions, you’ll be able to achieve that here. The team are using more advanced technology as time goes on and you’ll be able to suggest potential tools to use. Snowflake is one of the examples of this, as it’s recently been brought into the team on suggestion of one of the team and is now being widely used. Alternatively, if there’s a ready customised tool within AWS that you feel is a better option, then you can use that. There really is plenty of technical freedom here. In terms of your technical experience you’ll need to have worked in a commercial data engineering role for a few years, this is a mid-level position. Strong Python, Snowflake and SQL experience will be required and any experience of working with tools like Docker/Kubernetes and AWS would be a huge preference. Commodities experience/knowledge is not required but would be a plus. This is a global commodities firm with a strong history of performance and revenue. Your starting salary will be up to £130,000 plus a performance related bonus. Benefits include medical, dental and life insurance, wellness programs, pension, generous parental leave and various other perks. Want to make sure data has an impact on the future of commodities trading? Get in touch. No up-to-date CV required.

Job Title: Data EngineerClient Location: UK RemoteStarting: Dec 2024 Salary/Pay Rate: £77.000-83.9000 per annum, pro rata PAYEHours: Full-timeDuration: 12 months We are seeking a skilled and motivated Data Engineer to join our social media client to develop, evaluate, and maintain data architectures and solutions that drive critical initiatives across our organization. As a Data Engineer, you’ll play a crucial role in executing strategies that control, protect, and maximize the value of our data assets. This is a 12 months contract, offering up to £77.000-83.9000 per annum, pro rata (PAYE). Key Responsibilities:<ul><li>Oversee the end-to-end lifecycle of data engineering projects, ensuring robust planning, execution, and delivery.</li><li>Collaborate with business and technical stakeholders to translate key business needs into actionable data engineering solutions.</li><li>Design, build, test, and maintain scalable data management and ETL systems aligned with industry best practices and business requirements.</li><li>Develop high-performance algorithms, prototypes, predictive models, and proof of concepts to enhance data insights.</li><li>Implement and manage enterprise-level data ETL processes to streamline data accessibility and accuracy.</li></ul> Key Qualifications:<ul><li>Bachelor’s degree in Computer Science, Engineering, or a related technical field.</li><li>Proven experience in data engineering, with a strong focus on ETL processes, data architecture, and scalable systems.</li><li>Expertise in:</li><li>Python and SQL for data handling, scripting, and manipulation.</li><li>Airflow, Presto, Spark, or similar tools for ETL and big data management.</li><li>Data visualization to present insights effectively to stakeholders.</li></ul> Skills & Competencies:<ul><li>Team-oriented with the ability to work both independently and collaboratively.</li><li>Strong communication skills for effective written and verbal reporting and presentations.</li><li>Proven ability to work alongside Software Engineers, Data Scientists, and Product Managers to drive project goals.</li></ul> Apply today to become a valued member of our data engineering team and help shape the future of our data solutions! The next steps will be shared with shortlisted candidates by COB on Monday 4th November 2024. Client Description:Our Client is the largest social media company in the world. They have substantial B2B and B2C advertising and media platforms, as well as a nonprofit initiative. With the mission of bringing people together, they now boast over 2 billion users, and are rapidly developing as they influence the world around us. Aquent is dedicated to improving inclusivity & is proudly an equal opportunities employer. We encourage applications from under-represented groups & are committed to providing support to applicants with disabilities. We aim to provide reasonable accommodation for any part of the employment process, to those with a medical condition, disability or neurodivergence.

Senior Data engineer (Contract)- up to £600 inside ir35 Want to help save lives?*Hybrid in Central London**2 stage interview* Did you know a frontline worker loses their life every 10 seconds? Imagine leading a team in developing a risk management platform to help save lives.Your role is crucial in creating and maintaining an innovative mobile application that will be used to create detailed real time safety alerts for frontline workers operating on wind turbines and solar farms. We are looking for a Data Engineer who has experience in: <ul><li>3+ years within Snowflake and DBT</li><li>Experience in Azure or AWS</li><li>Solid experience and clear understanding of one ETL tool (For eg: Matillion, SNP Glue, Fivetran, Informatica)</li></ul>

Junior Data Engineer – Data & Tech Team Key Responsibilities<ul><li>Data Pipeline Development: Assist in building, maintaining, and optimizing data pipelines for extracting, transforming, and loading (ETL/ELT) data from multiple sources.</li><li>Data Quality & Integrity: Ensure the accuracy, consistency, and integrity of data within pipelines and systems.</li><li>Cloud Support: Assist in managing and deploying cloud-based data solutions, including basic troubleshooting and performance optimization.</li><li>Collaboration: Work closely with data engineer, scientists & analysts to understand business needs and develop data models that address key problems.</li><li>Automation & Process Improvement: Help identify and implement opportunities for automating manual data processes to improve overall efficiency.</li><li>Documentation: Contribute to clear and comprehensive documentation of data processes, pipelines, and architecture to support team knowledge sharing.</li><li>Learning & Growth: Stay up-to-date with the latest trends and technologies in data engineering, and actively contribute to team learning and improvement efforts.</li></ul> Skills & Qualifications (Not all essential)<ul><li>Relevant Degree: Bachelor’s degree in an associated field, such as computer science, software engineering, IT.</li><li>Basic SQL Knowledge: Ability to write and understand basic SQL queries for data retrieval and manipulation.</li><li>Python Familiarity: Some experience using Python for data manipulation and scripting.</li><li>Cloud Awareness: Exposure to cloud platforms (AWS, Google Cloud, or Azure) through prior experience, coursework or internships.</li><li>Problem-Solving Mindset: Has a proven background solving data related problems and identifying opportunities for optimisation.</li><li>Attention to Detail: Strong organisational skills with a focus on accuracy and the ability to manage multiple tasks effectively.</li><li>Team-Oriented: Excellent communication and collaboration skills, with a desire to work in a matrix structure in our agency.</li><li>Eager to Learn: A passion for learning new technologies, tools, and methodologies in the field of data engineering.</li></ul> Preferred Qualifications (Nice-to-Have)<ul><li>Experience with ETL tools such as Apache Airflow or dbt.</li><li>Familiarity with version control systems like Git/GitHub.</li><li>Prior experience in data engineering or data analytics.</li><li>Awareness of CI/CD practices and DevOps principles.</li></ul> Be Your Best Statement We want everyone to make the most of the opportunity to shine and showcase your talents and we are happy to make adjustments in the recruitment process so you can be your best. Please discuss any specific requirements with your dedicated Talent Acquisition team member or if you would feel more comfortable, you can email us confidentially at beyourbest@omnicommediagroup.com to let us know how we can support you. DE&I at OMG UKOur vision is to be an agency group where difference is valued and everyone is able to thrive in a culture of equality, inclusion and belonging. We are committed to providing a truly inclusive environment that reflects today’s society where everyone is able to bring their true selves to work and where diverse voices and backgrounds are valued, heard and well-represented.

Data Engineer Jobs in Greater London, England

Data Engineer
November 18, 2024

Data Engineer
November 18, 2024

Azure Data Engineer
November 12, 2024

Lead Data Engineer X2 London Permanent Hybrid
November 11, 2024

Data Engineer
November 7, 2024

Senior Data Engineer Pythonsnowflake Commodities Trading 130000 Salary Bonus
November 6, 2024

Data Engineer
November 6, 2024

Senior Data Engineer Contract
November 6, 2024

Junior Data Engineer
October 24, 2024

Quick filters

Top companies

Data Engineer Jobs in Greater London, England

Data EngineerNovember 18, 2024

Data EngineerNovember 18, 2024

Azure Data EngineerNovember 12, 2024

Lead Data Engineer X2 London Permanent HybridNovember 11, 2024

Data EngineerNovember 7, 2024

Senior Data Engineer Pythonsnowflake Commodities Trading 130000 Salary BonusNovember 6, 2024

Data EngineerNovember 6, 2024

Senior Data Engineer ContractNovember 6, 2024

Junior Data EngineerOctober 24, 2024

Quick filters

Top companies

Data Engineer
November 18, 2024

Data Engineer
November 18, 2024

Azure Data Engineer
November 12, 2024

Lead Data Engineer X2 London Permanent Hybrid
November 11, 2024

Data Engineer
November 7, 2024

Senior Data Engineer Pythonsnowflake Commodities Trading 130000 Salary Bonus
November 6, 2024

Data Engineer
November 6, 2024

Senior Data Engineer Contract
November 6, 2024

Junior Data Engineer
October 24, 2024