ML Data Engineer
2 days ago
City of London
Reports to: Director of Data Science & Engineering Job Purpose We are looking for a Data Engineer with Machine Learning experience to join our team. They will be responsible for maintaining and supporting the existing data infrastructure used to underpin our data analytics and reporting. The Data Engineer will own the company's drive into Machine Learning and AI adoption. They will be accountable for building as well as owning new Machine Learning and AI solutions across the team and the wider business. These solutions must align with business needs and be delivered with ROI in mind. These must complement our current, scalable data architecture. They will support the customer analytics function by developing and deploying machine learning models and techniques to deliver value around our consumer (B2C) data. The Data Engineer will be expected to optimize the architecture of our data pipelines and ensure that data flows support various cross functional teams across the business. The ideal candidate will have a self-directed, innovative mindset who is comfortable supporting the data needs of multiple teams. The right candidate will be proactive in identifying and implementing improvements for our systems contributing constructively to the current data eco-system. As the business continues to invest in cloud solutions, particularly Google Cloud Platform, you will be excited by the prospect of owning new projects, propelling our data initiatives and capabilities. Lead Data Engineer will be exposed to best practice methods with the current framework. The Data & Marketing Department The Data & Marketing department is the intelligence powerhouse of the business, representing the customer. Our goal is to inspire, engage and grow a loyal readership by harnessing data-driven insights and innovative marketing to deliver personalised, impactful content experiences that builds lasting trust, diverse revenue streams, and a thriving future for quality journalism. The department consists of four teams: Data Science & Engineering, Data Intelligence & Monetisation, Research & Insight and Customer Marketing. About You • An accomplished ML/Data Engineer with proven experience deploying and managing ML and AI based services in production., • Strong SQL skills and hands-on experience with both relational and non-relational databases, supporting data needs in fast-paced, content-driven environments., • Strong expertise in designing and maintaining scalable data pipelines and architectures—integrating data from web analytics, content management systems (CMS), subscription platforms, ad tech, and social media., • Proven ability to automate and optimise data workflows, using modern ETL/ELT tools (e.g., Airflow, dbt, Apache Spark) to ensure timely and reliable delivery of data., • Experience building robust data models and reporting layers to support performance dashboards, user engagement analytics, ad revenue tracking, and A/B testing frameworks., • Skilled in cloud-based data platforms and infrastructure (e.g., AWS, GCP), ensuring scalability and security for large volumes of streaming and batch data. Additionally, experience with data warehouses such as BigQuery or Snowflake., • Adept in Python and/or Java for developing data services and integrating APIs to bring in diverse sources of media data., • Exposure to utilising AI and Agentic workflows to build and deploy models agents and coding pipelines. Ideally having experience in Gemini, Claude and Vertex AI., • Exposure to ML Frameworks such as PyTorch or TensorFlow., • Agent orchestration using frameworks such as Langgraph or CrewAI, • Understanding of ML Ops such as automated model, testing,monitoring and tuning., • Excellent interpersonal and communication skills, enabling effective collaboration with analytical and commercial teams to turn data into actionable insights., • Proactive and self-driven, capable of managing multiple data projects in a high tempo setting while meeting tight deadlines., • A continuous learner with a diligent approach to data engineering including data privacy (e.g., GDPR compliance) and evolving best practices. Diversity, Equity and Inclusion • We champion diversity in our teams and in our reporting. As a growing and global brand, we must have a workforce that’s more representative of our readers, viewers, clients and partners, and a workplace that creates a sense of belonging for everyone, • We are committed to hiring and developing a diverse workforce regardless of background, and we support our people to thrive in their careers here. The Independent is an equal opportunities employer, if you require any reasonable adjustments to complete your application, please do not hesitate to advise us accordingly. Key Responsibilities and Accountabilities - what is delivered • Design and Maintain Data Pipelines: Develop and maintain robust, scalable, and efficient data pipeline architecture to support current and future business needs., • Engineering and Integration: Assemble large, complex datasets from a variety of structured and unstructured sources, ensuring they meet functional requirements., • Process Automation and Optimisation: Identify, design, and implement improvements to automate manual processes, enhance data delivery performance, and re-architect infrastructure for improved scalability and resilience., • AI Agentic Workflows: Designs and deploy agentic workflows to increase efficiency in support of Engineering and the Analytics function., • AI adoption: Build on the current usage of AI across the business rolling out new processes and models., • Machine Learning: Take the lead in building out our capability in this area with new models and recommender systems., • ETL Development and Infrastructure Building: Build and manage the infrastructure necessary for optimal ETL or ELT of data using Python, SQL, and Google Cloud Platform (GCP) big data technologies, such as BigQuery, Dataflow, Dataproc and Cloud Storage., • Business Intelligence Enablement: Prepare and transform pipeline data to support downstream analytics and feed BI tools (DOMO), enabling data-driven decision-making across the organization., • Cross-Functional Collaboration: Partner with internal stakeholders—ranging from Data, Commercial, and Editorial teams to executive leadership—to address data-related technical challenges and support their infrastructure needs., • Enhance Data System Functionality: Collaborate with the Data Team to continuously improve the functionality, flexibility, and performance of data systems and platforms., • Data Governance and Compliance: Ensure all data is handled responsibly, securely, and in full compliance with the Data Protection Act, GDPR regulations, and the Company’s Code of Conduct. Skills and Experience • SQL and Database Expertise: Strong working knowledge of SQL with hands-on experience querying and managing relational databases, alongside familiarity with a variety of database technologies (e.g., PostgreSQL, MySQL, BigQuery)., • Big Data Engineering: Proven experience designing, building, and optimizing ‘big data’ pipelines, architectures, and datasets, enabling efficient data processing at scale., • Analytical Problem Solving: Skilled in performing root cause analysis on complex internal and external data sources and business processes to resolve issues and uncover opportunities for operational or strategic improvements., • Unstructured Data Handling: Strong analytical capability for working with unstructured and semi-structured datasets, transforming raw information into actionable insights., • Data Workflow Development: Expertise in developing and maintaining data transformation processes, managing data structures, metadata, workload dependencies, and orchestration frameworks., • Large-scale Data Processing: A demonstrated history of manipulating, processing, and extracting value from large, diverse, and disconnected datasets in fast-moving environments., • Project Management & Collaboration: Excellent project management and organizational skills, with experience supporting and collaborating with cross-functional teams in dynamic and evolving settings., • Education & Professional Background: Holds a graduate degree in Computer Science, STEM related quantitative field, with 2+ years of hands-on experience in a data engineering role. Tools & Technologies: • Databases: Proficient in SQL databases., • Workflow Management Tools: Experience with orchestration platforms such as Apache Airflow., • Programming Languages: Skilled in one or more of the following languages, i.e.: Python, Java, Go., • Cloud Infrastructure: Strong understanding of cloud infrastructure such as GCP and tools within the respective platform. Our values – you will deliver across all our values Inclusive: We champion diversity in our teams and in our reporting. Working as a team, we put transparency and effective communication at the heart of everything we do. Innovative: From the very beginning, The Independent has been breaking the mould. We take risks and are always looking to try new ideas in pursuit of excellence. Independent: Nobody tells us what to think; we make up our own minds and aren’t afraid to do things differently. Like our readers, we value honesty and integrity above outside influences.