Artificial Intelligence Engineer
hace 5 días
Liverpool
Job Title: Artificial Intelligence Engineer (Databricks) Rate: DOE (outside IR35) Location: Remote Contract Length: 6 months A consultancy client of ours have secured a project requiring a Databricks focused Artificial Intelligence Engineer. This is an exciting opportunity to work on cutting-edge machine learning projects, building scalable ML pipelines and cloud-based systems that deliver real-world impact. Key Responsibilities: • Lead the design, development, and optimisation of scalable machine learning workflows using Azure Databricks, • Build and deploy robust ML pipelines leveraging Delta Lake, MLflow, notebooks, and Databricks Jobs, • Apply advanced knowledge of Databricks architecture and performance tuning to support production-grade ML solutions, • Collaborate with data scientists, data engineers, and analysts to operationalise machine learning models at scale, • Champion the use of Databricks-native features (e.g., Unity Catalog, MLflow Model Registry, AutoML) to improve model lifecycle management, • Migrate legacy model training and scoring workflows into unified Databricks-based pipelines, • Ensure best practices in model reproducibility, governance, monitoring, and security within the Databricks environment, • Act as a subject matter expert on Databricks ML capabilities, advising on architecture, tools, and integrations, • Mentor peers and junior engineers on ML engineering practices, with a focus on MLOps and Databricks workflows, • Continuously improve the machine learning platform, tooling, and deployment practices to accelerate delivery Experience and Qualifications Required: • Deep hands-on experience with Azure Databricks, particularly in developing and deploying machine learning solutions using Delta Lake, MLflow, and Spark ML/PyTorch/TensorFlow integrations, • Strong programming skills in Python (including ML libraries like scikit-learn, pandas, PySpark) and experience using SQL for data preparation and analysis, • Experience orchestrating end-to-end ML pipelines, including data preprocessing, model training, validation, and deployment, • Solid understanding of MLOps principles, including model versioning, monitoring, and CI/CD for ML workflows, • Familiarity with Azure cloud services, including Azure Data Lake, Azure Machine Learning, and Data Factory, • Experience with feature engineering, model management, and automated retraining in production environments, • Knowledge of data governance, security, and regulatory compliance in the context of ML workflows, • Strong problem-solving skills, with the ability to debug and optimise distributed ML pipelines, • Proven track record of delivering machine learning models in production within enterprise-scale environments, • Excellent communication and collaboration skills, with experience engaging both technical and business stakeholders, • Experience mentoring others and promoting best practices in ML engineering and Databricks usage If this sounds like an exciting opportunity please apply with your CV.