Databricks jobs in RichmondCreate job alerts

AI Engineer Job Tenure: Full-time, permanentSalary: CompetitiveCompany Description Ready to join a team thats leading the way in reshaping the future of insurance? Here at esure Group, we are on a mission to revolutionise insurance for good! We’ve been providing Home and Motor Insurance since 2000, with over 2 million customers trusting us to keep them covered through our esure and Sheilas’ Wheels brands. With a bold dedication for digital innovation, were transforming the way the industry operates and putting customers at the heart of everything we do. Having completed our recent multi-year digital transformation, we’re now leveraging advanced technology and data-driven insights alongside exceptional service, to deliver personalised experiences that meet our customers ever-changing needs today and in the future. Job Description We currently have a phenomenal opportunity for an AI Engineer to join our us as we enhance our customer experience by harnessing the power of NLP, large language models, and flawless integration with third-party GenAI services. In this dynamic role, youll be involved in every stage of the AI lifecycle, from brainstorming ideas and crafting prompts to fine-tuning models and deploying them into production. Success in this role is measured through rigorous A/B testing, ensuring our solutions deliver real value. Youll have the opportunity to work on a variety of impactful projects across the entire enterprise. We are seeking an innovative individual who shares our vision to fix insurance for good. What you’ll do: Collaborate with business partners to identify use-cases where GenAI can provide measurable benefits. Support the data science and AI team in deploying scalable AI applications across the enterprise. Work with a multi-functional team (DevOps, data engineers, developers, testers, Infosec) to productionize AI services on AWS. Enhance our tech stack, including MLOps, CI/CD pipelines, UI, and AI Python libraries. Develop prompts, fine-tune models, and track results. Evaluate third-party GenAI and LLM technologies for cost-benefit analysis. Deliver projects from scoping and R&D to production deployment and benefit measurement. Follow and contribute to standards for code quality, R&D, and model development. Engineer data for optimal AI application performance. Advise peers and senior management on business and customer impact. Write production-grade Python code, leveraging CI/CD, Databricks, and ensuring appropriate test coverage; proficiency in SQL. Liaise with third-party partners and stay updated on field advancements and regulations. Collaborate with data governance and data management teams on data requirements. Qualifications What we’d love you to bring: You are a self-starter with experience in delivering research + models to production to drive tangible commercial benefits and improved customer experiences. You thrive in working autonomously and collaboratively to drive impact in a fast-paced multi-functional business. PhD or MSc experience in a numerate subject: Machine Learning, Computer Science, Statistics, or equivalent, Demonstrable experience of applying, researching, and developing sophisticated NLP models, prompt engineering + fine-tuning LLMs to drive value in a commercial setting, Able to work independently and efficiently to deliver projects on time, taking project briefs and refining them to strong results. Experienced in engaging with non-technical stakeholders to scope, design and build an appropriate ML solution. Proficient with Python data science stack, e.g., pandas, scikit-learn, Jupyter etc., and version control, e.g., Git. Exposure to LLMOps, model monitoring principles, CI/CD and associated tech, e.g., Docker, MLflow, k8s, FastAPI etc. Knowledge of Langchain, huggingface and LLM offerings by 3rd party vendors. Knowledge of OO programming and software design, i.e., SOLID principles. Knowledge and working experience of AGILE methodologies. Familiarity with Databricks, and RAG application architecture a plus Experience with latency optimisation and quantisation preferred Additional Information What’s in it for you?: Competitive salary that reflects your skills, experience and potential. Discretionary bonus scheme that recognises your hard work and contributions to esure’s success. 25 days annual leave, plus 8 flexible days and the ability to buy and sell further holiday. Our flexible benefits platform is loaded with perks to choose from, so you can build a personal toolkit to support your health, wellbeing, lifestyle, and finances. Company funded private medical insurance for qualifying colleagues. Fantastic discounts on our insurance products! 50% off for yourself and spouse/partner and 10% off for direct family members. We’ll elevate your career with hands-on training, mentoring, access to our exclusive academies, regular career conversations, and expert partner resources. Driving good in the world couldn’t be more important to us. Our colleagues can use 2 volunteering days per year to support their local communities. Join our internal networks and communities to connect, learn, and share ideas with likeminded colleagues. We’re a proud supporter of the ABI’s ‘Make Flexible Work’ campaign and welcome you to ask about the flexibility you need. Our hybrid working approach also puts you in the driving seat of how and where you do your best work. And much more; See a full overview of our benefits here Reward and benefits | Esure Group PLC We are committed to creating an inclusive and diverse workplace where everyone feels valued, respected, and empowered. We celebrate individuality and create spaces where unique backgrounds and experiences can come together. We believe that diverse perspectives drive innovation, in turn enabling us to better serve our customers, community and build a stronger organisation. Our commitment to inclusion extends to every part of our business, from hiring practices to professional growth opportunities, ensuring equal access and support for all.

Data Architect - £70-90K base (DOE) - West London (hybrid)** We are recruiting a Data Architect for one of our clients based in West London on a permanent basis. The Data Architect is responsible for leading the definition, standardization, and governance of data architecture across platforms and products. This role balances technical leadership, data architecture, and collaboration with engineering, product, and security teams to ensure scalable, reliable, and secure systems. Key responsibilities: Enforce data architectural guidelines and consistency across development teams and services Support established Data Governance and Data Quality frameworks, including tooling, policy enforcement, and stewardship models Ensure robust metadata management, lineage tracking, and data cataloguing using business glossaries and modern catalog tools Review and approve data architecture for major features, platforms, and technical initiatives Collaborate with technical leads and DevOps on system scalability, performance, and reliability Ensure data platforms are AI/ML-ready, with scalable infrastructure and clean, well-structured data pipelines Collaborate with data science and analytics teams to enable model deployment, automation, and MLOps best practices Promote innovation in generative AI, predictive analytics, and real-time decision support Align data architecture with security, compliance, and data governance requirements Lead the evolution of technical architecture documentation, models, and decision records Conduct architecture and design reviews with cross-functional teams Guide teams in the adoption of best practices in API design, modularity, cloud-native patterns, and event-driven systems Recommend data management best practices, covering data flows, architecture patterns, retention, archival, and purging strategies Coach and mentor engineers on data design, refactoring, and architectural reasoningEssential skills and experience: Proven experience designing and scaling enterprise-grade cloud data platforms (AWS preferred) Deep experience with AWS, Databricks, Power Platform, and Redshift (Snowflake a plus) Proficiency in AWS Glue, Qlik Talend, DBT, Airflow, and modern data integration tools. Excellent knowledge of Python, SQL, PowerQuery (M), and preferably Scala or PySpark Working knowledge with enterprise architecture frameworks (e.g., TOGAF), MLOps, and BI tools like Power BI and QuickSight Experience of generative AI platforms (e.g., Amazon Bedrock, Anthropic) Familiarity with infrastructure as code (Terraform), CI/CD practices (Jenkins, GitHub Actions), and observability (Grafana, Kibana) Proficiency in scripting and automation using Bash, Groovy, or equivalent Ability to balance long-term architectural vision with immediate delivery constraintsDamia Group Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this permanent job, you accept our Data Protection Policy which can be found on our website. Please note that no terminology in this advert is intended to discriminate on the grounds of a person's gender, marital status, race, religion, colour, age, disability or sexual orientation. Every candidate will be assessed only in accordance with their merits, qualifications and ability to perform the duties of the job. Damia Group is acting as an Employment Business in relation to this vacancy and in accordance to Conduct Regulations 2003. The advertised salary range is dependent on experience and the required qualifications. Damia Group Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept our Data Protection Policy which can be found on our website. Please note that no terminology in this advert is intended to discriminate on the grounds of a person's gender, marital status, race, religion, colour, age, disability or sexual orientation. Every candidate will be assessed only in accordance with their merits, qualifications and ability to perform the duties of the job. Damia Group is acting as an Employment Business in relation to this vacancy and in accordance to Conduct Regulations 2003

We are looking for an experienced Data Engineering Architect / Consultant with strong Databricks expertise to support a private equity-backed international organisation in shaping the early-stage architecture of a federated data platform. Data Consultant (Databricks) / Data Engineering Architect.

Senior Data Engineer- Spark & Databricks Optimisation- Up to £55K. Databricks (but AWS, Azure or GCP.

This is a hands-on architecture role suited to someone who has spent the last 2-3 years working directly with Databricks at an architectural level and is ready to progress towards programmes such as the Databricks DPP. Data Architect - Databricks (Hybrid, UK). About the RoleWe are seeking an experie

Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse. Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services. Develop and maintain infrastructure-as-code (eg, Terraform, ARM) to provision and ma

Build and test high-performance ELT pipelines in Databricks. What We’re Looking For - 4 years expertise in Databricks, Spark, and cloud data architecture - Coding skills in Python and SQL. Experience with orchestration tools like Azure Data Factory and Databricks Workflows.

A global information and analytics company seeks a Software Engineer III to develop Java-based backend services and implement Scala/Spark pipelines for metric computations.This role emphasizes collaboration within a cross-functional team and requires experience with RESTful APIs.Flexible working hou

What We’re Looking For Technical Expertise - Strong experience with Databricks, Apache Spark, and cloud-based data architectures (preferably Azure). Experience orchestrating pipelines with Azure Data Factory and Databricks Workflows. Lead Data Engineer - Databricks Location: Remote UK or hybrid

Valcon is growing its Databricks community and we’re looking for a Databricks Architect to help clients turn data into measurable business results. Experience designing data Lakehouse architectures with Databricks at the core with security, privacy, and governance at its core. You’ll own the des

A leading data and AI company is seeking an experienced employee relations professional to support complex HR issues and conduct corporate investigations across Europe.The successful candidate will partner closely with management to develop strategies that enhance employee performance and maintain p

This mid-senior level position requires experience with Databricks, SQL, Python, and PySpark.

Join a forward-thinking technology consultancy that champions innovation and collaboration.This exciting opportunity involves developing scalable data pipelines, utilizing cutting-edge technologies like AWS and Python, and working closely with cross-functional teams to drive data solutions.As part o

An innovative technology firm seeks a Data Engineer to design and enhance cloud data platforms using Azure Data Services and Databricks.

Databricks Consultant / Databricks Engineering Architect. We are looking for an experienced Data Engineering Architect / Consultant with strong Databricks expertise to support a private equity–backed international organisation in shaping the early-stage architecture of a federated data platform. D

A leading data and AI company based in London is seeking a Technical Advisor to engage with senior stakeholders in the startup ecosystem.This role involves building relationships with CTOs and VPs of Engineering while driving awareness of the company's offerings.The ideal candidate has over 10 years

Operate as an expert solution architect and trusted advisor for significant data analytics architecture, design, and adoption of the Databricks Lakehouse platform. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the

The role involves driving the migration of enterprise data platforms from Palantir Foundry to Databricks, optimizing architectures, and ensuring data integrity. Ideal candidates will possess advanced Databricks expertise, proven migration experience, and strong skills in PySpark and SQL.

A leading technology firm seeks a Lead Data Engineer to drive the adoption of Databricks and build a modern Lakehouse architecture. Ideal candidates have over 5 years in data engineering, strong AWS and Databricks experience, and excellent Python and SQL skills.

At Databricks, we are on a mission to empower our customers to solve the world’s toughest data problems by utilizing the Databricks Data Intelligence Platform. It is commercial in the sense that you will drive growth in your assigned customers and use cases through leading your customers’ stakeh

Databricks: Must have recent experience.

At Databricks, we are on a mission to empower our customers to solve the world's toughest data problems by utilizing the the Databricks Data Intelligence Platform. It is commercial in the sense that you will drive growth in your assigned customers and use cases through leading your customers' stakeh

Strong proficiency in Python, SQL, FastAPI, and Databricks is required.

You will develop "technical champions" within our top Partners, providing enablement on technical matters related to the Databricks product. Provide partners with the level of enablement they need to assist their clients in evaluating and adopting Databricks, including hands‑on Apache Spark™ pro

DATAPAO, meaning that you'll get access to Databricks' public and internal courses to learn all the tricks of Distributed Data Processing, MLOps, Apache Spark, Databricks, and Cloud Migration from the best. We are backed by Databricks - the creators of Apache Spark, and act as a delivery partner and

A leading energy trading company in Greater London is seeking a skilled PowerBI + Databricks Engineer to functional design and develop applications.

A leading data and AI company in London is seeking a professional with strong technical pre-sales expertise.The ideal candidate will develop customer engagement strategies, influence stakeholders on data analytics solutions, and coach junior team members.Applicants should have hands-on experience in

A leading recruitment agency is looking for a Senior Azure Databricks Data Engineer specializing in energy and commodities trading. The ideal candidate will have strong Python skills, familiarity with Azure, and experience with tools like Snowflake and Databricks.

The ideal candidate will have strong skills in Python and SQL, along with hands-on experience in Azure Databricks.

A leading data and AI company is seeking an experienced Employee Relations professional to guide and support a dynamic team.This role involves partnering with management, conducting critical investigations, and managing relationships with European Works Councils.Ideal candidates will possess signifi

Ideal candidates should possess strong skills in data engineering, Azure Data Factory, Databricks, and proficiency in Python for automation.

The role focuses on Data Platform solutions, particularly with Databricks.

A leading cloud solutions provider seeks a Solutions Architect to enhance product adoption and engagement across various client sectors.With a focus on transactional database expertise and technical credibility, the successful candidate will collaborate with account teams to implement strategies ali

A leading consulting firm is seeking a Senior Azure Data Engineer to design and develop scalable data solutions on the Databricks platform.

The ideal candidate should have a solid foundation in data engineering and strong SQL skills, preferably with experience in Databricks or Snowflake.

As Databricks Associate Partner, you will own and grow Databricks-led revenue for Valcon in the UK. Enable Sales and Delivery teams on Databricks value propositions. In February 2026, we achieved Databricks Gold Partner status.

A leading data and AI company is seeking an exceptional Enterprise Account Executive to drive success in the Defence sector.This role involves building relationships with major Defence contractors and managing complex accounts.The ideal candidate will have strong experience in big data, cloud, and s

Industry experience with Databricks. We are looking for an experienced Databricks Lead Engineer and Architect to join and support the Delta Lake and Delta Table re‑engineering projects.

Technical Fluency: Advanced technical fluency; 2 Databricks Associate Certifications or 1 Professional-level Certification required within 6 months of hire. Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of

The ideal candidate will have hands-on expertise in Azure Databricks, ADF, ADLS Gen2, and strong programming skills in Python and SQL, along with solid data modeling knowledge.

This hybrid role in Wimbledon involves leading a team to develop high-performance data models in Databricks and collaborating with various stakeholders to enhance data-driven decision-making.

Build and maintain data processing and transformation workflows using Databricks. Strong hands‑on Databricks experience in production environments. SC Cleared – Cloud Engineer (Databricks / Azure).

Develop workflows and ELT pipelines using Python and Databricks. Good working knowledge of AWS, Databricks, and Python.

At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. Individuals looking for employment at Databricks are considered without regard to a

As the domain expert for Banking and Insurance (FSI) within Databricks UK&I, you will work in coordination with the Industry Leads to set the vision, build executive relationships with customers and partners, and collaborate with cross‑functional teams across Databricks in UK&I. As a Trusted Techn

Ideal candidates will have 3+ years of experience, strong skills in Python, Databricks, and SQL, and a good understanding of NoSQL databases.

A leading data and AI company in the UK is seeking a senior executive in Retail and Consumer Packaged Goods.This role involves setting the vision for technology interactions among clients, providing strategic guidance, and building executive relationships.Ideal candidates will have over 5 years of e

A leading data solutions firm in the UK is seeking a highly experienced Databricks Architect to lead the design and development of scalable data platforms. This role involves managing Databricks Lakehouse solutions, enforcing best practices, and collaborating closely with business leaders. The ideal