Databricks Architect
8 days ago
Sheffield
Job Title: Databricks Architect Location: UK/Europe - Remote Engagement Type: Contract (Outside IR35) Role Overview We are seeking a Senior Databricks Architect with 15+ years of experience in Data Engineering/Analytics to design and deliver scalable, secure, and cost-efficient data solutions on AWS. This is a leadership role requiring strong hands-on expertise, proactive risk management, and excellent stakeholder communication skills. Key Responsibilities β’ Architect and implement data solutions using Databricks on AWS, PySpark, and Python, β’ Lead cloud migration initiatives, including on-prem to AWS migrations, β’ Drive GenAI-enabled automation for data asset migration, β’ Develop scalable data models and deliver actionable business insights, β’ Design and optimize ETL pipelines (Informatica or similar tools), β’ Build and deploy PySpark applications for large-scale data processing, β’ Work on LLM pretraining, fine-tuning, augmentation, and optimization, β’ Ensure high data quality through cleaning, transformation, and validation (Python/SQL), β’ Automate data workflows to enhance scalability and efficiency, β’ Maintain clear technical documentation and best practices Required Experience β’ 15+ years in Data Analyst/Data Engineer/Data Architect roles, β’ Strong expertise in Databricks on AWS, β’ Advanced Python (Pandas, Scikit-learn) and SQL skills, β’ Proven experience in large-scale cloud data migrations, β’ Hands-on experience with LLMs and GenAI-based automation, β’ Strong experience designing secure, high-performance data architectures Key Competencies β’ Proactive risk & issue management, β’ Strong requirement clarification and stakeholder engagement skills, β’ Excellent communication β able to translate technical insights into business value, β’ Critical thinker with strong problem-solving capabilities, β’ Self-driven, autonomous, and collaborative approach