DevOps Engineer (SC Cleared) - Contract Inside IR35
2 days ago
London
About the Role: We are seeking a highly skilled and experienced Platform Engineer to build and maintain the core infrastructure and tooling that empowers our data scientists, economists, and developers working on our cutting-edge Azure Databricks Economic Data Platform. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Platform Engineer will focus on creating a self-service, scalable, and reliable platform that streamlines development workflows, simplifies infrastructure management, and enhances overall productivity. This role requires a strong understanding of cloud computing (specifically Azure), infrastructure-as-code (IaC), DevOps practices, containerisation, orchestration, and a passion for building developer-friendly platforms. Key Responsibilities:Platform Design & Development: • Design, develop, and maintain the core platform infrastructure on Azure, including networking, compute, storage, security, and identity management., • Implement infrastructure-as-code (IaC) using tools like Terraform, ARM templates, or Bicep to automate infrastructure provisioning and management., • Deeply integrate Azure Databricks into the platform, ensuring seamless access and efficient resource utilisation., • Implement automation for Databricks workspace setup, cluster configuration, and job deployments., • Implement and manage containerised applications and services using Docker and Kubernetes (or Azure Kubernetes Service - AKS)., • Design and implement robust CI/CD pipelines for building, testing, and deploying platform components and user applications., • Automate build processes, unit tests, integration tests, and deployment processes., • Implement comprehensive monitoring, logging, and alerting systems to proactively identify and address performance issues, errors, and security threats., • Use tools like Azure Monitor, Prometheus, Grafana, or similar to collect and analyse metrics, logs, and traces., • Implement security best practices and controls within the platform infrastructure and CI/CD pipelines., • Ensure compliance with relevant security standards and regulations. Develop and maintain comprehensive documentation for the platform, including API documentation, user guides, and troubleshooting guides. * Provide support to users of the platform. Collaboration & Communication: • Collaborate closely with data scientists, economists, developers, and other stakeholders to understand their needs and gather feedback on the platform., • 5+ years of experience in platform engineering, DevOps engineering, or a related role., • Strong experience with Azure cloud platform and services - Data Factory, Databricks, WebApp Service, API Management & APIOps., • Good experience with IaC tools like Terraform, ARM templates, or Bicep., • Solid understanding of CI/CD principles and experience with CI/CD tools like Github Actions, Azure DevOps, Jenkins, or GitLab CI., • Preferred 2+ years of experience with Github Actions, • Strong experience with containerisation technologies like Docker and orchestration tools like Kubernetes (or AKS)., • Experience with monitoring and logging tools., • Scripting skills in PowerShell, Bash, YAML and Python., • Good understanding of networking concepts and security best practices., • Excellent problem-solving and troubleshooting skills., • Strong communication and collaboration skills., • Experience with Azure Databricks and its integration with platform tooling., • Experience with configuration management tools like Ansible, Puppet, or Chef., • Experience building and maintaining internal developer platforms., • Experience working in a regulated industry (eg, financial services)., • Azure certifications (eg, Azure DevOps Engineer Expert, Azure Administrator Associate, Azure Solutions Architect Expert).