Director of Data Engineering
2 days ago
City of London
Director of Data Engineering: AI & Data Platforms Location: London (Hybrid – 1–2 days per week onsite) Salary: Up to £140,000 + equity & benefits Reports to: CTO Overview of the Role This is a rare opportunity to shape and lead the evolution of a next-generation, AI-native data platform within a high-growth, well-funded scale-up. As Director of Data Engineering, you will operate at the intersection of strategy and execution partnering closely with the CTO to define the long-term architecture, build a high-performing team, and transition the organisation from a traditional data platform to an agentic, AI-driven ecosystem. You will inherit a capable team, but more importantly, you will define its future: how it scales, what it builds, and where it leverages external innovation. This role combines hands-on technical leadership (circa 50%) with team and organisational leadership (circa 50%), making it ideal for a builder-leader who thrives on both architecture and people. Key Responsibilities Strategic Leadership & Platform Vision • Partner with the CTO to define and execute the AI and data platform strategy, including critical build vs buy decisions, • Establish a clear approach to partner vs upskill, ensuring the team leverages external innovation while building core internal capability, • Shape the long-term vision for an agentic, AI-native data ecosystem AI-Driven Data Platform Development • Lead the design of a unified AI search layer, combining vector, keyword, and graph-based approaches (e.g. GraphRAG), • Architect and scale agent-based systems and human-in-the-loop workflows, • Oversee the development of a knowledge graph and data enrichment pipelines to unlock proprietary data value, • Drive AI/ML Ops maturity, including LLM deployment, RAG pipelines, and evaluation frameworks Engineering & Architecture • Define scalable, cloud-native architectures across GCP (preferred) and AWS, • Lead cross-cloud data orchestration, ensuring seamless data flow from ingestion to intelligence layers, • Guide technical decisions on tooling, frameworks, and platform evolution Team Leadership & Growth • Lead and develop a team of data and AI engineers, including senior and staff-level contributors, • Build a culture of engineering excellence, accountability, and continuous improvement, • Hire, mentor, and scale the team in line with business growth Delivery & Impact • Ensure engineering effort is focused on high-value, domain-specific problems, • Improve velocity through modern engineering practices and AI-assisted development, • Balance innovation with pragmatism, avoiding unnecessary reinvention while maintaining competitive advantage Key Requirements Leadership & Experience • Proven experience as a Director or Senior Engineering Manager leading data/platform teams, • Strong track record of scaling teams and mentoring engineers (typically 5+ years in leadership roles), • Comfortable operating in a hands-on leadership capacity Data Platform & Cloud Expertise • Experience building and scaling high-volume data platforms, • Strong knowledge of GCP (BigQuery preferred) and exposure to AWS, • Expertise in data pipelines, distributed processing, and Python-based data services AI & Emerging Technologies • Exposure to or strong interest in agentic systems, LLMs, and AI-driven architectures, • Experience with AI search (vector databases, hybrid search, GraphRAG) is highly desirable, • Familiarity with knowledge graphs, ontologies, or complex data modelling Strategic & Commercial Thinking • Experience making build vs buy and technology investment decisions, • Ability to evaluate and integrate third-party tools, platforms, and partnerships, • Strong alignment with business outcomes, not just technical delivery Additional Information • Working Pattern: Hybrid (1–2 days per week in London office), • Salary: Up to £140,000 + equity + benefits (private healthcare, pension), • Team Size: ~4–5 engineers currently, scaling further, • Career Progression: Clear path towards VP Engineering / CTO, • Environment:, • Stable, well-funded scale-up (not early-stage chaos), • Highly unique, proprietary datasets, • Strong investment in modern AI tooling and practices Interview Process (3 Stages) • Introductory conversation with senior leadership, • Technical/design interview (architecture-focused), • Final interview with executive stakeholders (in-person) This role is ideal for a senior engineering leader who wants to build, shape, and scale—not just maintain. If you’re motivated by cutting-edge AI, complex data challenges, and genuine strategic influence, this is an opportunity to make a lasting impact.