Senior Data Engineer
2 days ago
Newcastle upon Tyne
As a Senior Data Engineer, you are responsible for leading, managing and delivering all aspects of data development activities to meet customer needs to agreed timescales. You will own the overall Data Engineering Development process within Data Engineering along with setting the strategies for products and the wider community. As a Senior Data Engineer, you will provide technical leadership to a team of Data Engineers along with providing oversight to the products they develop. Youll lead a small team of Data Engineers and collaborate closely with Software Engineers, DevOps Engineers, and Service Management teams to build and maintain high-quality ETL pipelines and data strategies. Your work will directly support the delivery of cutting-edge solutions using technologies such as artificial intelligence, machine learning, and robotic process automation. You will have line management responsibilities, mentoring and coaching others, helping to build a strong Data community. To help you really make a difference, youll be using the best tools and equipment for the job. We use Elastic Stack to conduct log analytics as a search and real-time visualisation tool. We use Qlik Sense as our primary BI tool for data integration and dashboard creation. We plan and track work in Jira, we document in SharePoint, we communicate using Teams. We do it all on MacBook Pro devices. You will also have access to fantastic learning and development resources; conferences with industry expert speakers, comprehensive training, and subscriptions to digital training solutions. Your roles and responsibilities include: • Lead the redevelopment of existing data journeys, providing leadership to improve, performance, resilience, or to account for system, tooling or platform change on code frameworks and practices to meet all user needs., • Develop, own, maintain and ensure adherence to standards across the Data Engineering community in proactively preventing issues and problems with data, outputs, tools and pipelines., • Provide technical leadership, setting engineering team and community direction to ensure the products that are developed conform to data standards and strategic principles, • Responsible for translating user requirements and data designs into effective, robust data ETL solutions in line with build standards and ensure code is production ready and properly assured., • Accountability and ownership for BI reporting strategy and ensure continuous improvement, maintenance and deletion/archiving activities are undertaken. Works with other disciplines to ensure best practice is embedded., • Design and undertake complex analysis to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers., • Communicating with stakeholders at all levels: managing expectations around delivery of data, products and data pipelines; facilitating discussions across high risk and complex projects or under constrained delivery timescales., • Design and undertake complex analysis and build capability to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers., • Lead the redevelopment of existing data journeys, providing leadership to improve, performance, resilience, or to account for system, tooling or platform change on code frameworks and practices to meet all user needs., • Develop, own, maintain and ensure adherence to standards across the Data Engineering community in proactively preventing issues and problems with data, outputs, tools and pipelines., • Provide technical leadership, setting engineering team and community direction to ensure the products that are developed conform to data standards and strategic principals., • Responsible for translating user requirements and data designs into effective, robust data ETL solutions in line with build standards and ensure code is production ready and properly assured., • Accountability and ownership for BI reporting strategy and ensure continuous improvement, maintenance and deletion/archiving activities are undertaken. Works with other disciplines to ensure best practice is embedded., • Design and undertake complex analysis to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers., • Communicating with stakeholders at all levels: managing expectations around delivery of data, products and data pipelines; facilitating discussions across high risk and complex projects or under constrained delivery timescales., • Design and undertake complex analysis and build capability to resolve data and pipeline problems, issues; ensuring quality outputs can be supplied to researchers and other customers. Person specification See selection process for further details. If you would like to learn more about the role, please contact . Technical skills We'll assess you against these technical skills during the selection process: • Business Intelligence and Data Visualisation (Expert) Benefits Alongside your salary of 57,946, Department for Work and Pensions contributes 16,786 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides. We also have a broad benefits package built around your work-life balance which includes: • Working patterns to support work/life balance such as job sharing, term-time working, flexi-time and compressed hours., • Generous annual leave at least 25 days on entry, increasing up to 30 days over time (prorata for part time employees), plus 9 days public and privilege leave., • Support for financial wellbeing, including interest-free season ticket loans for travel, a cycle to work scheme and an employee discount scheme., • Health and wellbeing support including our Employee Assistance Programme for specialist advice and counselling and the opportunity to join HASSRA a first-class programme of competitions, activities and benefits for its members (subscription payable monthly)., • Family friendly policies including enhanced maternity and shared parental leave pay after 1 years continuous service., • Funded learning and development to support progress in your role and career. This includes industry recognised qualifications and accreditations, coaching, mentoring and talent development programmes. This role may be suitable for hybrid working, which is where an employee works part of the week in their DWP office and part of the week from home. This is a voluntary, non-contractual arrangement and your office will be your contractual place of work. If a hybrid working arrangement is suitable for the role and for you, you will normally be required to spend a minimum of 60% of your contracted working hours from your DWP office. If you have a disability, caring responsibilities, or other circumstances that may affect your ability to meet the minimum office attendance requirement, please discuss this with us using the contact details in this advert. Salary Information Pay for this role is from 57,946 to 73,517. The maximum salary for the grade is 68,205, however a Digital Allowance of up to 5,312 per annum is available for exceptional candidates. Digital Allowance is non-salary, and non-pensionable, and is classed as a temporary allowance. It is reviewed annually where it could be retained, amended, or removed. Our offer to successful candidates will be based on an assessment of your skills and experience as demonstrated at interview. Existing Civil Servants who secure a new role on lateral transfer will maintain their current salary. Existing Civil Servants who gain promotion will either move to the bottom of the grade pay scale or get a 10% increase to their current salary whichever would be the greater. Things you need to know Artificial intelligence Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use. Selection process details This vacancy is using Success Profiles (opens in a new window) , and will assess your Experience and Technical skills. Stage 1: Application Your application will consist of four parts: 1. A Personal Details application form., 2. Employment history - this should contain your work experience and any skills, qualifications and accomplishments relevant to the jobs you have completed., 3. Personal statement - up to 750 words. This statement should be used to provide examples of how you meet the essential criteria below: • Data Engineering Design and Delivery (Practitioner): Ability to design and oversee the implementation of complex, enterprise-scale data processes, and ETL pipelines. Skilled in translating complex customer requirements into robust data processing solutions., • Data Standards and Governance (Expert): Deep understanding of how to establish and maintain data engineering standards across teams. Ability to lead adherence to wider community and organisational standards, ensuring consistency and compliance., • Problem Solving and Incident Management (Practitioner): Proven ability to lead investigations into complex data issues, including ETL pipeline failures and platform-level problems. Understands the strategic, tactical, and operational impact of data-related issues and apply appropriate solutions., • Leadership and Team Development (Practitioner/Expert): Experience in leading data engineering teams, mentoring junior engineers, and promoting continuous improvement. Ability to drive capability uplift and foster a culture of best practice within the team. 4. Technical statement (up to 250 words). The following statement is aligned to the required technical skill of Business Intelligence and Data Visualisation (Expert): describe in 250 words, how you would: • Describe the process you would follow to design and implement a Business Intelligence solution, including the development and delivery of dashboards for large volumes of operational and performance data across multiple systems for senior stakeholders, who require timely, accurate, and an approach for selection of KPIs that are visually compelling dashboards to support strategic decision-making. Ensure that all examples provided in your statements are taken directly from your own experience and that you describe the examples in your own words. If you choose to use gen-AI to support your statements, you must follow the guidelines outlined in the Artificial intelligence and recruitment guide. The sift panel will use the information in your employment history, personal statement, and technical statement to assess your experience, skills and knowledge. An initial sift will be conducted using the technical statement. Candidates who pass the initial sift will be progressed to a full sift. If you do progress to a full sift, you will be provided with one combined overall assessment score for both your employment history and Personal Statement. For Hints and Tips on completing your application visit Applying for jobs at DWP Digital. Applications will be sifted at regular intervals from the date the posts are advertised. Please apply as soon as you can, do not wait until the end of the campaign. Important Information • You will be asked to complete your employment history. Any information that you would customarily share on a CV should therefore be entered onto the application form., • Personal details that could be used to identify you including your name, contact details and address must be removed for your application to be considered. If youre successful at stage 1 you will be invited to a video interview via Microsoft Teams. There, you will be assessed against your response to the technical statement. Stage 3: Interview 2 If youre successful at stage 2 you will be invited to a video interview via Microsoft Teams. There, you will be assessed against the remaining experiences listed in the essential criteria. Interviews will take place from mid-January 2026. Interview dates to be confirmed. Other information: If high application volumes are received, the benchmark for candidates to proceed to the next stage may be raised. In line with our commitment to the Disability Confident Scheme (DCS), we aim to advance all candidates applying under the DCS who meet the minimum standard. However, we may only progress those candidates who best meet the required standards. All successful candidates and those placed on reserve will be posted in merit list order by location. For this vacancy, we strongly recommend that applicants consult with an immigration specialist or qualified advisor to assess their eligibility for Visa Sponsorship before deciding to apply. Please note that while we consider sponsorship requests in accordance with current DWP guidance and Home Office policy, sponsorship cannot be guaranteed. Security Clearance Requirement You must meet the security requirements before you can be appointed. The level of security needed is security check. Applicants for this post must have lived in the UK for 5 years or more during their lifetime for DWP to accept National Security Vetting applications at the Security Check (SC) level. Requests to apply for SC will always be supported where people have lived in the UK for the last 5 years but will be subject to a pre-assessment if they have lived outside the UK during this period. For further information on National Security Vetting please visit the Demystifying Vetting website . Reasonable Adjustment At DWP we value diversity and inclusion and actively encourage and welcome applications from everyone, including those that are underrepresented in our workforce. We consider visible and non-visible disabilities, neurodiversity or learning differences, chronic medical conditions, or mental ill health. Examples include dyslexia, epilepsy, autism, chronic fatigue, or schizophrenia. If you need a change to be made so that you can make your application, you should: Contact Government Recruitment Service via as soon as possible before the closing date to discuss your needs. Complete the Reasonable Adjustments section in the Additional requirements page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if youre deaf, a Language Service Professional. For further information on reasonable adjustments, terms and conditions and how we recruit visit the How we recruit page. Find out more about Working for DWP A reserve list may be held for a period of 6 months from which further appointments can be made. Reserve list candidates will be posted in merit order by location. The Civil Service values honesty and integrity and expects all candidates to abide by these principles. Please ensure that all examples provided in your application are taken directly from your own experience and that you describe the examples in your own words. Applications will be screened and if evidence of plagiarism or copying examples/answers from other sources is found, your application will be withdrawn. Internal DWP candidates may also face disciplinary action. Feedback will only be provided if you attend an interview or assessment. Security Successful candidates must undergo a criminal record check. Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window) . See our vetting charter (opens in a new window) . People working with government assets must complete baseline personnel security standard (opens in new window) checks. Nationality requirements This job is broadly open to the following groups: • UK nationals, • nationals of the Republic of Ireland, • nationals of Commonwealth countries who have the right to work in the UK, • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window), • nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS), • individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020, • Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service Further information on nationality requirements (opens in a new window) Working for the Civil Service The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants. We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window) . The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy. Diversity and Inclusion The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window) . Apply and further information This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative. The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window) . Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records. Contact point for applicants Job contact : • Name : Caicy Sandford Appointment to the Civil Service is governed by the Civil Service Commission Recruitment Principles. If you wish to make a complaint, please find further details here: . Attachments DWP Terms and Conditions January 2024 Opens in new window (docx, 17kB) Success-Profiles-Candidate-Overview (18) Opens in new window (pdf, 635kB)